Skip to content

Commit fc3c558

Browse files
committed
more updates
1 parent 19b59c0 commit fc3c558

File tree

9 files changed

+15
-35
lines changed

9 files changed

+15
-35
lines changed

Extensions/AzureTableAPI/README.md

+2
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22

33
This extension is built to facilitate both a data Source and Sink for the migration tool to be able to read and write to Azure Table API. This covers both the Azure Storage Table API and Azure Cosmos DB Table API, since they both adhere to the same API spec.
44

5+
> **Note**: When specifying the AzureTableAPI extension as the Source or Sink property in configuration, utilize the name **AzureTableAPI**.
6+
>
57
## Configuration Settings
68

79
The AzureTableAPI has a couple required and optional settings for configuring the connection to the Table API you are connecting to. This applies to both Azure Storage Table API and Azure Cosmos DB Table API.

Extensions/Cosmos/README.md

+2
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22

33
The Cosmos data transfer extension provides source and sink capabilities for reading from and writing to containers in Cosmos DB using the Core (SQL) API. Source and sink both support string, number, and boolean property values, arrays, and hierarchical nested object structures.
44

5+
> **Note**: When specifying the JSON extension as the Source or Sink property in configuration, utilize the name **Cosmos-nosql**.
6+
57
## Settings
68

79
Source and sink settings both require multiple parameters to specify and provide access to the data location within a Cosmos DB account:

Extensions/Json/README.md

+2
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22

33
The JSON data transfer extension provides source and sink capabilities for reading from and writing to JSON files. Source and sink both support string, number, and boolean property values, arrays, and hierarchical nested object structures.
44

5+
> **Note**: When specifying the JSON extension as the Source or Sink property in configuration, utilize the name **JSON**.
6+
57
## Settings
68

79
Source and sink settings both require a `FilePath` parameter, which should specify a path to a JSON file or folder containing JSON files. The path can be either absolute or relative to the application. The JSON files can contain either an array of JSON objects or a single object. Sink also supports an optional `Indented` parameter (`false` by default) and an optional `IncludeNullFields` parameter (`false` by default) to control the formatting of the JSON output.

Extensions/Mongo/README.md

+2
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22

33
The Mongo data transfer extension provides source and sink capabilities for reading from and writing to a Mongo database.
44

5+
> **Note**: When specifying the Mongo extension as the Source or Sink property in configuration, utilize the name **Mongo**.
6+
>
57
## Settings
68

79
Source and sink settings require both `ConnectionString` and `DatabaseName` parameters. The source takes an optional `Collection` parameter (if this parameter is not set, it will read from all collections). The sink requires the `Collection` parameter and will insert all records received from a source into that collection, as well as an optional `BatchSize` parameter (default value is 100) to batch the writes into the collection.

Extensions/SqlServer/README.md

+2
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22

33
The SQL Server data transfer extension provides source and sink capabilities for reading from and writing to table data in SQL Server. Hierarchical data is not fully supported but the Sink extension will write a nested object from a hierarchical source (i.e. JSON, Cosmos) into a SQL column as a JSON string.
44

5+
> **Note**: When specifying the SQL Server extension as the Source or Sink property in configuration, utilize the name **SqlServer**.
6+
57
## Settings
68

79
Source and sink settings both require a `ConnectionString` parameter.

README.md

+5-35
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,9 @@ This tutorial outlines how to use the Cosmos DB Data Migration tool to move JSON
108108

109109
```json
110110
{
111-
"SourceSettings": {
111+
"Source": "JSON",
112+
"Sink": "Cosmos-nosql",
113+
"SourceSettings": {
112114
"FilePath": "C:\\btcdata\\simple_json.json"
113115
},
114116
"SinkSettings": {
@@ -126,41 +128,9 @@ This tutorial outlines how to use the Cosmos DB Data Migration tool to move JSON
126128

127129
3. Ensure the **Cosmos.DataTransfer.Core** project is set as the startup project then press <kbd>F5</kbd> to run the application.
128130

129-
4. The application will load the provided extensions, then prompt for a **Source**. Select **JSON**. When prompted to load settings from a file enter `n` and press <kbd>Enter</kbd>. The application provides the ability to configure extensions in a separate file versus having them located in the **migrationsettings.json** file. When a separate file is not specified, the settings are read from **migrationsettings.json**. When prompted to provide the Configuration section name, press <kbd>Enter</kbd> to accept the default: **JSONSourceSettings**.
130-
131-
![The Data Migration application displays with a menu prompt for Source.](media/app_source_prompt.png "Select Source")
132-
133-
>**Note**: Alternatively, to avoid this prompt add the **Source** property to the configuration with the value **Json** (matching the **DisplayName** property of the extension)
134-
135-
5. The application will then prompt for a **Sink**, choose **Cosmos-nosql**. When prompted to load settings from a file enter `n` and press <kbd>Enter</kbd>. The application provides the ability to configure extensions in a separate file versus having them located in the **migrationsettings.json** file. When a separate file is not specified, the settings are read from **migrationsettings.json**. When prompted to provide the Configuration section name, press <kbd>Enter</kbd> to accept the default: **Cosmos-NoSqlSinkSettings**.
131+
4. The application then performs the data migration. After a few moments the process will indicate **Done.**.
136132

137-
![The Data migration application displays with a menu prompt for Sink.](media/app_sink_prompt.png "Select Sink")
138-
139-
>**Note**: Alternatively, to avoid this prompt add the **Sink** property to the configuration with the value **Cosmos-nosql** (matching the **DisplayName** property of the extension)
140-
141-
6. The application then performs the data migration. After a few moments the process will indicate **Done.**.
142-
143-
![The Data migration application displays with the full output of the migration including the Done message.](media/app_final.png "Data Migration completes")
144-
145-
Example of `migrationsettings.json` with **Source** and **Sink** configured.
146-
147-
```json
148-
{
149-
"Source": "JSON",
150-
"Sink": "Cosmos-nosql",
151-
"SourceSettings": {
152-
"FilePath": "C:\\btcdata\\simple_json.json"
153-
},
154-
"SinkSettings": {
155-
"ConnectionString": "AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDj...",
156-
"Database": "datamigration",
157-
"Container": "btcdata",
158-
"PartitionKeyPath": "/id",
159-
"RecreateContainer": false,
160-
"IncludeMetadataFields": false
161-
}
162-
}
163-
```
133+
>**Note**: The `Source` and `Sink` properties should match the **DisplayName** set in the code for the extensions.
164134

165135
## Using the command line
166136

media/app_final.png

-11.9 KB
Binary file not shown.

media/app_sink_prompt.png

-5.12 KB
Binary file not shown.

media/app_source_prompt.png

-5.84 KB
Binary file not shown.

0 commit comments

Comments
 (0)