Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for forward and backward compatibility in DCL schemes #540

Merged
merged 6 commits into from
Feb 20, 2024
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update schema compatibility strategies
akarabashov committed Feb 12, 2024

Unverified

The email in this signature doesn’t match the committer email.
commit a202f16f5fbfc1851894d085337973c71e1f0273
29 changes: 25 additions & 4 deletions docs/design/schema-compatibility.md
Original file line number Diff line number Diff line change
@@ -6,7 +6,7 @@ Schema changes can cover a wide range of modifications with varying impacts on a

### Strategy for Compatible Changes

For changes that are backward-compatible, such as adding optional fields or extending enumerations:
For changes that are backward-compatible, such as adding optional fields or extending enumerations.

**Description:**

@@ -28,7 +28,7 @@ This strategy is straightforward and quick to implement, but only suitable for c

### Strategy for Non-Compatible Changes

For significant changes that directly impact compatibility, such as adding mandatory fields or removing fields, splitting or merging schemas, changing enumerations:
For significant changes that directly impact compatibility, such as adding mandatory fields or removing fields, splitting or merging schemas, changing enumerations.

#### Option 1: Separate Schemas for Each Version

@@ -65,7 +65,7 @@ While offering a robust solution for handling radical changes, this method requi

### Strategy for Compatible changes (Not keeping backward compatibility in API)

For changes that are backward-compatible, such as adding optional fields or extending enumerations:
For changes that are backward-compatible, such as adding optional fields or extending enumerations.

**Description:**

@@ -80,7 +80,7 @@ This strategy focuses on updating the schema without ensuring backward compatibi

### Strategy for Non-Compatible changes (Not keeping backward compatibility in API)

For changes that affect compatibility, like adding mandatory fields
For changes that affect compatibility, like adding mandatory fields.

**Description:**

@@ -93,6 +93,27 @@ This strategy focuses on updating the schema without ensuring backward compatibi
- Update transactions and queries if needed.
- Add a new transaction to fulfill new required fields (essentially this is a manual migration via transactions)

### Strategy for Non-Compatible changes (Keeping backward compatibility in API)

For changes that affect compatibility but can be converted, adding mandatory fields and changing enumerations.

**Description:**

The main idea of this strategy is the dynamically converting newer schemas into older ones. However, this method is only possible if there is compatibility between the newer and legacy schemas, allowing them to be converted to each other. Due to the on-the-fly data conversion, this approach does not support the Light Client in legacy APIs because the converted data is not stored in the state, preventing the generation of proofs.

**Strategy steps:**

- For each update:
- Create a new version of a Schema and state (a new .proto file)
- Migrate older states to newer schema version.
- Remove the states associated with the older schema versions.
- Implement transactions and queries for the new schema version.
- Update older transactions and queries to converting data between the latest and older schema version, ensuring backward compatibility.
- There will be separated API for each version of the schema, for example::
- models/vid/pid
- modelsV2/vid/pid
- modelsV3/vid/pid

## Conclusion

To lay the foundation for future compatibility improvements, it's a good idea to start by adding a version field to each schema. For subsequent changes, we will then select the most appropriate strategy based on the nature of these changes.