[QUESTION] How does one go about implementing a custom Output connector?

[weakphish]

What is your question or problem? Please describe.
Hi! For our use-case, we'd like to implement a connector that writes to S3 along with some other operations. The docs on custom connectors only go over how to create an input, but not an output.

Diving into the source code, it looks like existing outputs packaged with Pathway use the table.to method, but I'm not sure if this is meant to be used by users who are implementing custom connectors. I also saw the ability to create subscribers in Python code, but it wasn't clear to me if that was the idiomatic / preferred way to do custom writers.

Describe what you would like to happen
I'd like an example or documentation on creating a custom output / writer connector.

[voodoo11]

Hi. Please take a look at this tutorial.

I must admit it could use an update since "subscribe" now has more options, but it should still be a good starting point.

API docs here

[weakphish]

@voodoo11 Thank you for the reply and information! Loving Pathway so far :)

One note - the docs for connectors lists Python as both input and output, right above "Python output" - I think this was a source of confusion for me.

Additionally, does on_time_end refer to window functions when it says the following?

the callback function to be called on each closed time of computation.

[weakphish]

And a follow up question - thank you for your patience :) - will state be retained for a subscriber?

What I mean is, if I want to patch changes, could I define a list of changes, and in my on_change callback append to that list? With that list retaining the state. I assume that might be tricky with the Rust engine interop.