Adding example for OData API #98
Conversation
fivetran-sahilkhirwal
requested review from
varundhall,
ameetfivetran and
Copilot
|
This pull request has been linked to 1 task:
|
There was a problem hiding this comment.
Pull Request Overview
This PR adds comprehensive examples for syncing data from OData APIs using the Connector SDK. It introduces examples for OData versions 2 and 4, demonstrates usage of both pyodata and python_odata libraries, and covers advanced scenarios such as incremental sync, expanded relationships, multiple entity operations, and batch operations.
Reviewed Changes
Copilot reviewed 9 out of 13 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| examples/source_examples/odata_api/odata_version_4/README.md | Adds detailed instructions for syncing data using OData v4 with state tracking and advanced query options. |
| examples/source_examples/odata_api/odata_version_2/README.md | Provides similar usage examples for OData v2, including incremental sync and expanded relationships. |
| examples/source_examples/odata_api/odata_version_4/connector.py | Implements the connector for syncing data from multiple OData services using different client libraries. |
| examples/source_examples/odata_api/using_python_libraries/using_python_odata/connector.py | Demonstrates fetching and filtering data from OData v4 via python_odata. |
| examples/source_examples/odata_api/odata_version_2/connector.py | Contains a connector implementation tailored for OData v2 with incremental sync and batch processing. |
| examples/source_examples/odata_api/using_python_libraries/using_pyodata/connector.py | Provides an example for syncing data from OData v2 using the pyodata library. |
| examples/source_examples/odata_api/odata_version_2/ODataClient.py | Implements the client class for interacting with OData v2 services using pyodata. |
| README.md | Updates the overview to include the new OData API example. |
Files not reviewed (4)
- examples/source_examples/odata_api/odata_version_2/requirements.txt: Language not supported
- examples/source_examples/odata_api/odata_version_4/requirements.txt: Language not supported
- examples/source_examples/odata_api/using_python_libraries/using_pyodata/requirements.txt: Language not supported
- examples/source_examples/odata_api/using_python_libraries/using_python_odata/requirements.txt: Language not supported
examples/source_examples/odata_api/odata_version_2/ODataClient.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Pull Request Overview
This pull request adds example implementations for syncing data from various versions of OData APIs using different Python libraries. The changes include new README guides and connector implementations for OData version 2, version 4, and examples using both python_odata and pyodata, as well as an update to the main README file to list the odata_api example.
Reviewed Changes
Copilot reviewed 9 out of 13 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| examples/source_examples/odata_api/odata_version_2/README.md | Added reference guide for OData version 2 syncing |
| examples/source_examples/odata_api/odata_version_4/README.md | Added reference guide for OData version 4 syncing |
| examples/source_examples/odata_api/odata_version_4/connector.py | New connector implementation for OData version 4 |
| examples/source_examples/odata_api/odata_version_2/connector.py | New connector implementation for OData version 2 |
| examples/source_examples/odata_api/using_python_libraries/using_python_odata/connector.py | New connector using python_odata library |
| examples/source_examples/odata_api/using_python_libraries/using_pyodata/connector.py | New connector using pyodata library |
| examples/source_examples/odata_api/odata_version_2/ODataClient.py | New ODataClient implementation for version 2 |
| README.md | Updated example list to include odata_api |
Files not reviewed (4)
- examples/source_examples/odata_api/odata_version_2/requirements.txt: Language not supported
- examples/source_examples/odata_api/odata_version_4/requirements.txt: Language not supported
- examples/source_examples/odata_api/using_python_libraries/using_pyodata/requirements.txt: Language not supported
- examples/source_examples/odata_api/using_python_libraries/using_python_odata/requirements.txt: Language not supported
Comments suppressed due to low confidence (1)
examples/source_examples/odata_api/odata_version_2/README.md:3
- [nitpick] There is a minor grammatical issue: the word 'This' should not be capitalized after the comma. Consider changing it to 'this'.
if you still need to sync data from the OData version 2 API, This document provides a reference guide for using the ODataClient class in your Fivetran connector:
examples/source_examples/odata_api/using_python_libraries/using_pyodata/connector.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Pull Request Overview
This PR adds a suite of examples and documentation for syncing data from OData APIs using Connector SDK, covering multiple versions (v2, v4) with different Python libraries. Key changes include:
- An example using python_odata for OData v4.
- An example using pyodata for OData v2.
- Updated documentation and a shared ODataClient implementation for OData v2.
Reviewed Changes
Copilot reviewed 9 out of 13 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| examples/source_examples/odata_api/using_python_libraries/using_python_odata/connector.py | Example for syncing using the python_odata library for OData API v4. |
| examples/source_examples/odata_api/using_python_libraries/using_pyodata/connector.py | Example for syncing using the pyodata library for OData API v2. |
| examples/source_examples/odata_api/odata_version_4/connector.py | Example for OData v4 using ODataClient for advanced query and sync features. |
| examples/source_examples/odata_api/odata_version_4/README.md | Documentation for using the OData v4 connector. |
| examples/source_examples/odata_api/odata_version_2/connector.py | Example for OData v2 using ODataClient built on pyodata. |
| examples/source_examples/odata_api/odata_version_2/README.md | Documentation for using the OData v2 connector. |
| examples/source_examples/odata_api/odata_version_2/ODataClient.py | Implementation of ODataClient for OData v2 interactions. |
| README.md | Updated main README to list the new OData API examples. |
Files not reviewed (4)
- examples/source_examples/odata_api/odata_version_2/requirements.txt: Language not supported
- examples/source_examples/odata_api/odata_version_4/requirements.txt: Language not supported
- examples/source_examples/odata_api/using_python_libraries/using_pyodata/requirements.txt: Language not supported
- examples/source_examples/odata_api/using_python_libraries/using_python_odata/requirements.txt: Language not supported
Comments suppressed due to low confidence (2)
examples/source_examples/odata_api/using_python_libraries/using_pyodata/connector.py:198
- [nitpick] The variable name 'service_v4' may be misleading in the context of an OData v2 example using pyodata. Consider renaming it to 'service_v2' or a more descriptive name to improve clarity.
service_v4 = setup_odata_service()
examples/source_examples/odata_api/odata_version_2/ODataClient.py:150
- [nitpick] Consider deduplicating the select paths before applying them to the query, as duplicate fields in the select clause could lead to inefficiencies or unclear query definitions.
def _apply_all_select_path(self, select_paths, query, query_options):
| @@ -0,0 +1,183 @@ | |||
| ## Syncing data from OData version 2 | |||
|
|
|||
| The OData version 2 API is a legacy API that is no longer maintained. It is recommended to use the OData version 4 API instead. However, if you still need to sync data from the OData version 2 API, This document provides a reference guide for using the ODataClient class in your Fivetran connector: | |||
There was a problem hiding this comment.
can we also add what section of the example the customers need to change to make it work for their source?
There was a problem hiding this comment.
if we want to use this example as template, what part of this template needs edit
| @@ -0,0 +1,225 @@ | |||
| # This is an example for how to work with the fivetran_connector_sdk module. | |||
There was a problem hiding this comment.
Need README for these examples too. We need to mark and identify the parts of connector.py code that customer should edit to get their source data.
Closes https://fivetran.height.app/user/sahilk/T-899687
What example does