Skip to content

Commit 2a63ea5

Browse files
vankovapvankovap
authored
kafka (#253)
1 parent 0861e3b commit 2a63ea5

File tree

6 files changed

+299
-77
lines changed

6 files changed

+299
-77
lines changed

‎apps/docs/content/homepage.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,8 +50,8 @@ export const databases = [
5050
{ name: "Meilisearch", link: "/meilisearch/overview", icon: <Icons.meilisearch/> },
5151
{ name: "Qdrant", link: "/qdrant/overview", icon: <Icons.qdrant/> },
5252
{ name: "NATS", link: "/nats/overview", icon: <Icons.nats/> },
53+
{ name: "Kafka", link: "/kafka/overview", icon: <Icons.kafka/> },
5354
{ name: "KeyDB", link: "/keydb/overview", icon: <Icons.keydb/> },
54-
{ name: "Kafka", icon: <Icons.kafka/> },
5555
]
5656

5757
export const storages = [

‎apps/docs/content/kafka/overview.mdx

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
---
2+
title: Apache Kafka
3+
desc: Production-ready Apache Kafka on Zerops platform with automated scaling and enterprise reliability. Deploy, manage, and scale with zero infrastructure overhead.
4+
---
5+
6+
import UnorderedList from '@site/src/components/UnorderedList';
7+
import UnorderedCodeList from '@site/src/components/UnorderedCodeList';
8+
import data from '@site/static/data.json';
9+
10+
Zerops provides a fully managed [Apache Kafka](https://kafka.apache.org/) messaging platform with automated scaling and zero infrastructure overhead, letting developers focus entirely on development.
11+
12+
## Supported Versions
13+
14+
Currently supported Kafka version:
15+
<UnorderedList data={data.kafka.readable}/>
16+
17+
Import configuration version:
18+
<UnorderedCodeList data={data.kafka.import}/>
19+
20+
## Service Configuration
21+
22+
Our Kafka implementation features optimized default settings designed for common use cases.
23+
24+
### Key Configuration
25+
26+
* **Client Connections:** Data brokers available on port `9092`
27+
* **Authentication:** Secure SASL PLAIN with automatically generated credentials
28+
* **Data Persistence:** Topic data stored indefinitely (no time or size limit)
29+
* **Performance:** Optimized settings for reliability and throughput
30+
31+
### Resource Allocation
32+
33+
Zerops automatically allocates resources to your Kafka service based on demand:
34+
35+
* **Memory:** Up to 40GB RAM for high-performance message processing
36+
* **Storage:** Up to 250GB for persistent storage of messages and logs
37+
* **Auto-scaling:** Resources scale up and down automatically based on workload
38+
39+
## Deployment Modes
40+
41+
:::important
42+
Deployment mode is selected during service creation and cannot be changed later.
43+
:::
44+
45+
### High-Availability (HA) Setup
46+
47+
The recommended solution for production workloads and mission-critical data:
48+
49+
* Creates a multi-node Kafka cluster with 3 broker nodes
50+
* Configures 6 partitions across the cluster
51+
* Implements replication factor of 3 (each broker node has a copy of each partition)
52+
* Default topic replication is also 3 (overridable by user application)
53+
* Zerops automatically attempts to repair the cluster and data replication in case of a node failure
54+
55+
### Single Node Instance
56+
57+
Suitable for development and testing environments:
58+
59+
* Consists of 1 broker node
60+
* Configures 3 partitions
61+
* No data replication
62+
* Lower resource requirements
63+
64+
:::note
65+
Use for development or non-critical data only, as data loss may occur due to container volatility.
66+
:::
67+
68+
## Authentication Management
69+
70+
Authentication credentials are automatically generated and managed by the platform using SASL PLAIN authentication.
71+
72+
**Access your credentials through:**
73+
* The service access details in the Zerops GUI
74+
* Environment variables in your service configuration:
75+
* `user` - Username for authentication
76+
* `password` - Generated secure password
77+
* `port` - Kafka port (value: `9092`)
78+
79+
## Client Access
80+
81+
Client implementations differ, please refer to your chosen client's configuration manual for specific details.
82+
83+
### Access Methods
84+
85+
#### Seed Broker Connection
86+
Connect to the Kafka cluster using the "seed" (or "bootstrap") broker server:
87+
```
88+
<hostname>:9092
89+
```
90+
91+
#### Specific Broker Access
92+
To access a single specific broker or a list of all/some brokers:
93+
```
94+
node-stable-1.db.<hostname>.zerops:9092,node-stable-2.db.<hostname>.zerops:9092,...
95+
```
96+
97+
## Best Practices
98+
99+
### Production Workloads
100+
* Use HA mode for all production deployments
101+
* Configure proper retention policies for your topics based on your data requirements
102+
* Monitor consumer lag to ensure messages are being processed efficiently
103+
* Use consumer groups to distribute processing load
104+
105+
### Development Environments
106+
* Single node instances are suitable for development and testing
107+
* Be aware of potential data loss in non-HA deployments
108+
* Consider using smaller message sizes during development to reduce resource usage
109+
110+
## Support
111+
112+
For advanced configurations or custom requirements:
113+
* Join our [Discord community](https://discord.gg/zerops)
114+
* Contact support via [email](mailto:support@zerops.io)

‎apps/docs/sidebars.js

Lines changed: 29 additions & 74 deletions
Original file line numberDiff line numberDiff line change
@@ -416,6 +416,15 @@ module.exports = {
416416
},
417417
className: "homepage-sidebar-item service-sidebar-item",
418418
},
419+
{
420+
type: "ref",
421+
id: "kafka/overview",
422+
label: "Kafka",
423+
customProps: {
424+
sidebar_icon: "kafka",
425+
},
426+
className: "homepage-sidebar-item service-sidebar-item",
427+
},
419428
{
420429
type: 'ref',
421430
id: 'keydb/overview',
@@ -425,15 +434,6 @@ module.exports = {
425434
},
426435
className: 'homepage-sidebar-item service-sidebar-item',
427436
},
428-
// {
429-
// type: "ref",
430-
// id: "kafka/overview",
431-
// label: "Kafka",
432-
// customProps: {
433-
// sidebar_icon: "kafka",
434-
// },
435-
// className: "homepage-sidebar-item service-sidebar-item",
436-
// },
437437
],
438438
},
439439
{
@@ -2294,71 +2294,26 @@ module.exports = {
22942294
},
22952295
},
22962296
],
2297-
// rabbitmq: [
2298-
// {
2299-
// type: "ref",
2300-
// id: "homepage",
2301-
// label: "Back to home",
2302-
// customProps: {
2303-
// sidebar_is_back_link: true,
2304-
// sidebar_icon: "back-arrow",
2305-
// },
2306-
// },
2307-
// {
2308-
// type: "doc",
2309-
// id: "rabbitmq/overview",
2310-
// label: "Zerops RabbitMQ Service",
2311-
// customProps: {
2312-
// sidebar_is_title: true,
2313-
// sidebar_icon: "rabbitmq",
2314-
// },
2315-
// },
2316-
// {
2317-
// type: "category",
2318-
// label: "How-to",
2319-
// collapsible: false,
2320-
// customProps: {
2321-
// sidebar_is_group_headline: true,
2322-
// },
2323-
// items: [
2324-
// {
2325-
// type: "doc",
2326-
// id: "rabbitmq/how-to/create",
2327-
// label: "Create RabbitMQ service",
2328-
// },
2329-
// {
2330-
// type: "doc",
2331-
// id: "rabbitmq/how-to/connect",
2332-
// label: "Connect to RabbitMQ",
2333-
// },
2334-
// {
2335-
// type: "doc",
2336-
// id: "rabbitmq/how-to/manage",
2337-
// label: "Manage users and databases",
2338-
// },
2339-
// {
2340-
// type: "doc",
2341-
// id: "rabbitmq/how-to/export-import-data",
2342-
// label: "Export and import data",
2343-
// },
2344-
// {
2345-
// type: "doc",
2346-
// id: "rabbitmq/how-to/scale",
2347-
// label: "Scale RabbitMQ service",
2348-
// },
2349-
// {
2350-
// type: "doc",
2351-
// id: "rabbitmq/how-to/control",
2352-
// label: "Stop and start RabbitMQ service",
2353-
// },
2354-
// {
2355-
// type: "doc",
2356-
// id: "rabbitmq/how-to/delete",
2357-
// label: "Delete RabbitMQ service",
2358-
// },
2359-
// ],
2360-
// },
2361-
// ],
2297+
kafka: [
2298+
{
2299+
type: 'ref',
2300+
id: 'homepage',
2301+
label: 'Back to home',
2302+
customProps: {
2303+
sidebar_is_back_link: true,
2304+
sidebar_icon: 'back-arrow',
2305+
},
2306+
},
2307+
{
2308+
type: 'doc',
2309+
id: 'kafka/overview',
2310+
label: 'Zerops Kafka Service',
2311+
customProps: {
2312+
sidebar_is_title: true,
2313+
sidebar_icon: 'kafka',
2314+
},
2315+
},
2316+
],
23622317
sharedstorage: [
23632318
{
23642319
type: 'ref',

‎apps/docs/static/llms-full.txt

Lines changed: 77 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14185,8 +14185,8 @@ export const databases = [
1418514185
{ name: "Meilisearch", link: "/meilisearch/overview", icon: },
1418614186
{ name: "Qdrant", link: "/qdrant/overview", icon: },
1418714187
{ name: "NATS", link: "/nats/overview", icon: },
14188+
{ name: "Kafka", link: "/kafka/overview", icon: },
1418814189
{ name: "KeyDB", link: "/keydb/overview", icon: },
14189-
{ name: "Kafka", icon: },
1419014190
]
1419114191
export const storages = [
1419214192
{ name: "Object storage", link: "/object-storage/overview", icon: },
@@ -15950,6 +15950,82 @@ Have you build something that others might find useful? Don't hesitate to share
1595015950

1595115951
----------------------------------------
1595215952

15953+
# Kafka > Overview
15954+
15955+
Zerops provides a fully managed [Apache Kafka](https://kafka.apache.org/) messaging platform with automated scaling and zero infrastructure overhead, letting developers focus entirely on development.
15956+
## Supported Versions
15957+
Currently supported Kafka version:
15958+
Import configuration version:
15959+
## Service Configuration
15960+
Our Kafka implementation features optimized default settings designed for common use cases.
15961+
### Key Configuration
15962+
* **Client Connections:** Data brokers available on port `9092`
15963+
* **Authentication:** Secure SASL PLAIN with automatically generated credentials
15964+
* **Data Persistence:** Topic data stored indefinitely (no time or size limit)
15965+
* **Performance:** Optimized settings for reliability and throughput
15966+
### Resource Allocation
15967+
Zerops automatically allocates resources to your Kafka service based on demand:
15968+
* **Memory:** Up to 40GB RAM for high-performance message processing
15969+
* **Storage:** Up to 250GB for persistent storage of messages and logs
15970+
* **Auto-scaling:** Resources scale up and down automatically based on workload
15971+
## Deployment Modes
15972+
:::important
15973+
Deployment mode is selected during service creation and cannot be changed later.
15974+
:::
15975+
### High-Availability (HA) Setup
15976+
The recommended solution for production workloads and mission-critical data:
15977+
* Creates a multi-node Kafka cluster with 3 broker nodes
15978+
* Configures 6 partitions across the cluster
15979+
* Implements replication factor of 3 (each broker node has a copy of each partition)
15980+
* Default topic replication is also 3 (overridable by user application)
15981+
* Zerops automatically attempts to repair the cluster and data replication in case of a node failure
15982+
### Single Node Instance
15983+
Suitable for development and testing environments:
15984+
* Consists of 1 broker node
15985+
* Configures 3 partitions
15986+
* No data replication
15987+
* Lower resource requirements
15988+
:::note
15989+
Use for development or non-critical data only, as data loss may occur due to container volatility.
15990+
:::
15991+
## Authentication Management
15992+
Authentication credentials are automatically generated and managed by the platform using SASL PLAIN authentication.
15993+
**Access your credentials through:**
15994+
* The service access details in the Zerops GUI
15995+
* Environment variables in your service configuration:
15996+
* `user` - Username for authentication
15997+
* `password` - Generated secure password
15998+
* `port` - Kafka port (value: `9092`)
15999+
## Client Access
16000+
Client implementations differ, please refer to your chosen client's configuration manual for specific details.
16001+
### Access Methods
16002+
#### Seed Broker Connection
16003+
Connect to the Kafka cluster using the "seed" (or "bootstrap") broker server:
16004+
```
16005+
:9092
16006+
```
16007+
#### Specific Broker Access
16008+
To access a single specific broker or a list of all/some brokers:
16009+
```
16010+
node-stable-1.db..zerops:9092,node-stable-2.db..zerops:9092,...
16011+
```
16012+
## Best Practices
16013+
### Production Workloads
16014+
* Use HA mode for all production deployments
16015+
* Configure proper retention policies for your topics based on your data requirements
16016+
* Monitor consumer lag to ensure messages are being processed efficiently
16017+
* Use consumer groups to distribute processing load
16018+
### Development Environments
16019+
* Single node instances are suitable for development and testing
16020+
* Be aware of potential data loss in non-HA deployments
16021+
* Consider using smaller message sizes during development to reduce resource usage
16022+
## Support
16023+
For advanced configurations or custom requirements:
16024+
* Join our [Discord community](https://discord.gg/zerops)
16025+
* Contact support via [email](mailto:support@zerops.io)
16026+
16027+
----------------------------------------
16028+
1595316029
# Keydb > Getting Started
1595416030

1595516031
This quick start allows you to get hands-on experience of Zerops by running a predefined application consisting of a KeyDB service and a runtime service that handles the SQL queries.

0 commit comments

Comments
 (0)