Skip to content

Commit af51bc1

Browse files
shiyuhang0shiyuhang0
authored
cloud: add changefeed docs for TiDB Cloud Essential (#22244)
1 parent 3c1e259 commit af51bc1

6 files changed

Lines changed: 518 additions & 18 deletions

‎TOC-tidb-cloud-essential.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -260,6 +260,10 @@
260260
- [Vector Index](/vector-search/vector-search-index.md)
261261
- [Improve Performance](/vector-search/vector-search-improve-performance.md)
262262
- [Limitations](/vector-search/vector-search-limitations.md)
263+
- Stream Data ![BETA](/media/tidb-cloud/blank_transparent_placeholder.png)
264+
- [Changefeed Overview](/tidb-cloud/essential-changefeed-overview.md)
265+
- [Sink to MySQL](/tidb-cloud/essential-changefeed-sink-to-mysql.md)
266+
- [Sink to Apache Kafka](/tidb-cloud/essential-changefeed-sink-to-kafka.md)
263267
- Security
264268
- [Security Overview](/tidb-cloud/security-overview.md)
265269
- Identity Access Control

‎tidb-cloud/changefeed-sink-to-apache-kafka.md‎

Lines changed: 1 addition & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -11,15 +11,7 @@ This document describes how to create a changefeed to stream data from TiDB Clou
1111

1212
> **Note:**
1313
>
14-
> - To use the changefeed feature, make sure that your TiDB Cloud Dedicated cluster version is v6.1.3 or later.
15-
> - For [{{{ .starter }}}](/tidb-cloud/select-cluster-tier.md#starter) and [{{{ .essential }}}](/tidb-cloud/select-cluster-tier.md#essential) clusters, the changefeed feature is unavailable.
16-
17-
</CustomContent>
18-
<CustomContent plan="premium">
19-
20-
> **Note:**
21-
>
22-
> For [{{{ .starter }}}](/tidb-cloud/select-cluster-tier.md#starter) and [{{{ .essential }}}](/tidb-cloud/select-cluster-tier.md#essential) clusters, the changefeed feature is unavailable.
14+
> To use the changefeed feature, make sure that your TiDB Cloud Dedicated cluster version is v6.1.3 or later.
2315
2416
</CustomContent>
2517

‎tidb-cloud/changefeed-sink-to-mysql.md‎

Lines changed: 1 addition & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -11,15 +11,7 @@ This document describes how to stream data from TiDB Cloud to MySQL using the **
1111

1212
> **Note:**
1313
>
14-
> - To use the changefeed feature, make sure that your TiDB Cloud Dedicated cluster version is v6.1.3 or later.
15-
> - For [{{{ .starter }}}](/tidb-cloud/select-cluster-tier.md#starter) and [{{{ .essential }}}](/tidb-cloud/select-cluster-tier.md#essential) clusters, the changefeed feature is unavailable.
16-
17-
</CustomContent>
18-
<CustomContent plan="premium">
19-
20-
> **Note:**
21-
>
22-
> For [{{{ .starter }}}](/tidb-cloud/select-cluster-tier.md#starter) and [{{{ .essential }}}](/tidb-cloud/select-cluster-tier.md#essential) clusters, the changefeed feature is unavailable.
14+
> To use the changefeed feature, make sure that your TiDB Cloud Dedicated cluster version is v6.1.3 or later.
2315
2416
</CustomContent>
2517

‎tidb-cloud/essential-changefeed-overview.md‎

Lines changed: 186 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,186 @@
1+
---
2+
title: Changefeed (Beta)
3+
summary: TiDB Cloud changefeed helps you stream data from TiDB Cloud to other data services.
4+
---
5+
6+
# Changefeed (Beta)
7+
8+
TiDB Cloud changefeed helps you stream data from TiDB Cloud to other data services. Currently, TiDB Cloud supports streaming data to Apache Kafka and MySQL.
9+
10+
> **Note:**
11+
>
12+
> - Currently, TiDB Cloud only allows up to 10 changefeeds per {{{ .essential }}} cluster.
13+
> - For [{{{ .starter }}}](/tidb-cloud/select-cluster-tier.md#starter) clusters, the changefeed feature is unavailable.
14+
15+
## Supported regions
16+
17+
The changefeed feature is available in the following regions:
18+
19+
| Cloud provider | Supported regions |
20+
| --- | --- |
21+
| AWS | <ul><li>`us-east-1`</li><li>`ap-southeast-1`</li></ul> |
22+
| Alibaba Cloud | <ul><li>`ap-southeast-1`</li><li>`cn-hongkong`</li><li>`ap-southeast-5`</li></ul> |
23+
24+
Additional regions will be supported in the future. For immediate support in a specific region, contact [TiDB Cloud Support](/tidb-cloud/tidb-cloud-support.md).
25+
26+
## View the Changefeed page
27+
28+
To access the changefeed feature, take the following steps:
29+
30+
1. Log in to the [TiDB Cloud console](https://tidbcloud.com/) and navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project.
31+
32+
> **Tip:**
33+
>
34+
> You can use the combo box in the upper-left corner to switch between organizations, projects, and clusters.
35+
36+
2. Click the name of your target cluster to go to its overview page, and then click **Data** > **Changefeed** in the left navigation pane. The changefeed page is displayed.
37+
38+
On the **Changefeed** page, you can create a changefeed, view a list of existing changefeeds, and operate the existing changefeeds (such as pausing, resuming, editing, and deleting a changefeed).
39+
40+
## Create a changefeed
41+
42+
To create a changefeed, refer to the tutorials:
43+
44+
- [Sink to Apache Kafka](/tidb-cloud/essential-changefeed-sink-to-kafka.md)
45+
- [Sink to MySQL](/tidb-cloud/essential-changefeed-sink-to-mysql.md)
46+
47+
## View a changefeed
48+
49+
You can view a changefeed using the TiDB Cloud console or the TiDB Cloud CLI.
50+
51+
<SimpleTab>
52+
<div label="Console">
53+
54+
1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB cluster.
55+
2. Locate the corresponding changefeed you want to view, and click **...** > **View** in the **Action** column.
56+
3. You can see the details of a changefeed, including its configuration, status, and metrics.
57+
58+
</div>
59+
60+
<div label="CLI">
61+
62+
Run the following command:
63+
64+
```bash
65+
ticloud serverless changefeed get --cluster-id <cluster-id> --changefeed-id <changefeed-id>
66+
```
67+
68+
</div>
69+
</SimpleTab>
70+
71+
## Pause or resume a changefeed
72+
73+
You can pause or resume a changefeed using the TiDB Cloud console or the TiDB Cloud CLI.
74+
75+
<SimpleTab>
76+
<div label="Console">
77+
78+
1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB cluster.
79+
2. Locate the corresponding changefeed you want to pause or resume, and click **...** > **Pause/Resume** in the **Action** column.
80+
81+
</div>
82+
83+
<div label="CLI">
84+
85+
To pause a changefeed, run the following command:
86+
87+
```bash
88+
ticloud serverless changefeed pause --cluster-id <cluster-id> --changefeed-id <changefeed-id>
89+
```
90+
91+
To resume a changefeed:
92+
93+
```
94+
ticloud serverless changefeed resume -c <cluster-id> --changefeed-id <changefeed-id>
95+
```
96+
97+
</div>
98+
</SimpleTab>
99+
100+
## Edit a changefeed
101+
102+
> **Note:**
103+
>
104+
> TiDB Cloud currently only allows editing changefeeds in the paused status.
105+
106+
You can edit a changefeed using the TiDB Cloud console or the TiDB Cloud CLI.
107+
108+
<SimpleTab>
109+
<div label="Console">
110+
111+
1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB cluster.
112+
2. Locate the changefeed you want to pause, and click **...** > **Pause** in the **Action** column.
113+
3. When the changefeed status changes to `Paused`, click **...** > **Edit** to edit the corresponding changefeed.
114+
115+
TiDB Cloud populates the changefeed configuration by default. You can modify the following configurations:
116+
117+
- Apache Kafka sink: all configurations except **Destination**, **Connection**, and **Start Position**
118+
- MySQL sink: all configurations except **Destination**, **Connection** and **Start Position**
119+
120+
4. After editing the configuration, click **...** > **Resume** to resume the corresponding changefeed.
121+
122+
</div>
123+
124+
<div label="CLI">
125+
126+
Edit a changefeed with an Apache Kafka sink:
127+
128+
```bash
129+
ticloud serverless changefeed edit --cluster-id <cluster-id> --changefeed-id <changefeed-id> --name <new-displayName> --kafka <full-specified-kafka> --filter <full-specified-filter>
130+
```
131+
132+
Edit a changefeed with a MySQL sink:
133+
134+
```bash
135+
ticloud serverless changefeed edit --cluster-id <cluster-id> --changefeed-id <changefeed-id> --name <new-displayName> --mysql <full-specified-mysql> --filter <full-specified-filter>
136+
```
137+
138+
</div>
139+
</SimpleTab>
140+
141+
## Duplicate a changefeed
142+
143+
1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB cluster.
144+
2. Locate the changefeed that you want to duplicate. In the **Action** column, click **...** > **Duplicate**.
145+
3. TiDB Cloud automatically populates the new changefeed configuration with the original settings. You can review and modify the configuration as needed.
146+
4. After confirming the configuration, click **Submit** to create and start the new changefeed.
147+
148+
## Delete a changefeed
149+
150+
You can delete a changefeed using the TiDB Cloud console or the TiDB Cloud CLI.
151+
152+
<SimpleTab>
153+
<div label="Console">
154+
155+
1. Navigate to the [**Changefeed**](#view-the-changefeed-page) page of your target TiDB cluster.
156+
2. Locate the changefeed you want to delete, and click **...** > **Delete** in the **Action** column.
157+
158+
</div>
159+
160+
<div label="CLI">
161+
162+
Run the following command:
163+
164+
```bash
165+
ticloud serverless changefeed delete --cluster-id <cluster-id> --changefeed-id <changefeed-id>
166+
```
167+
168+
</div>
169+
</SimpleTab>
170+
171+
## Changefeed billing
172+
173+
Changefeeds are free of charge during the beta phase.
174+
175+
## Changefeed states
176+
177+
During the running process, changefeeds might fail with errors, or be manually paused or resumed. These behaviors can lead to changes of the changefeed state.
178+
179+
The states are described as follows:
180+
181+
- `CREATING`: the changefeed is being created.
182+
- `CREATE_FAILED`: the changefeed creation fails. You need to delete the changefeed and create a new one.
183+
- `RUNNING`: the changefeed runs normally and the checkpoint-ts proceeds normally.
184+
- `PAUSED`: the changefeed is paused.
185+
- `WARNING`: the changefeed returns a warning. The changefeed cannot continue due to some recoverable errors. The changefeed in this state keeps trying to resume until the state transfers to `RUNNING`. The changefeed in this state blocks [GC operations](https://docs.pingcap.com/tidb/stable/garbage-collection-overview).
186+
- `RUNNING_FAILED`: the changefeed fails. Due to some errors, the changefeed cannot resume and cannot be recovered automatically. If the issues are resolved before the garbage collection (GC) of the incremental data, you can manually resume the failed changefeed. The default Time-To-Live (TTL) duration for incremental data is 24 hours, which means that the GC mechanism does not delete any data within 24 hours after the changefeed is interrupted.

0 commit comments

Comments
 (0)