JetStream Object Store : Add conditional writes

[udf2457]

Proposed change

NATS should introduce a conditional write feature in order to protect against race conditions in various scenarios.

Use case

This probably explains it bettter than I could: https://www.exoscale.com/syslog/exoscale-sos-conditional-write/
(The Exoscale implementation is an implementation of the original concept from AWS: https://aws.amazon.com/about-aws/whats-new/2024/08/amazon-s3-conditional-writes/)

Contribution

No response

[Jarema]

While NATS JetStream supports some mechnisms that conditional writes require, and those are utilized in Key Value store, it is much harder to achieve the same in Object Store API, as Object Store PUT operations are not atomic - they publishing chunks and metadata messages into the stream. That in turn, makes introducing feature like conditional writes quite unreliable.

You could technically use KV with TTL'd messages as a distriubuted lock to achieve similar goals.

[udf2457]

Thanks for the clarification @Jarema .

I guess that makes sense although I suppose I would ask how come even though the operation is not atomic the result (ack-equivalent) of a PUT can be trusted ?

[Jarema]

@udf2457 the operation is successful only if all publishes to the Stream (chunks and metadata) were successful (client got acks for all of them).

[udf2457]

@Jarema , thanks.

Maybe at some point a kind soul could update the docs with this sort of useful background info ? It provides reassurance as to data integrity ! The docs are presently as they were during the beta/experimental-feature stages.