Vector retry function failed ??

[PejeDK]

Hi All

I am running vector01 (sink) -> Vector02 (source, port 1111) on a system.
No healthcheck on the sink.
Both running version 0.16.1.

The other day the vector02 had en networking error, i am still not sure what happend, where the node did not response on TCP 1111. The vector01 logged a lot of errors in the log regarding this (obviously). Vector02 server was fixed again after 1 hour and the port was up and responding on a netcat test.

But the vector01 never connected and started sending logs again through the pipeline.

Does anyone know how the retry function works in a vector sink ?
Is heatlthcheck mandatory for retry to function correct ?

Thanks in advance
\Peter

[jszwedko]

Hi @PejeDK !

Could you share the source and sink configs? I'm especially curious if you were using the version 1 or 2 of the protocol (0.16.1 defaults to v1). Either protocol version should be retrying connections though.

[PejeDK]

Hi

With what you say, then i am using version 1.

`
[sources.syslog_forwarder_1101]
type = "vector" # required
address = "[::0]:1101" # required

[sinks.syslog_forwarder_upstream]
type = "vector" # required
inputs = ["syslog_forwarder_1101"] # required
address = "XXXXXXXX:1101" # required
healthcheck = false # optional, default
`

I cannot find the differences on version 1 and two. Do you know if there are any difference that are nice to know ?

Thanks.
\Peter

[jszwedko]

Gotcha, I might recommend trying version = "2" on your vector source and sink. v1 should also be retrying but this version of the protocol is deprecated and so unlikely to see active development at this point. If you are still seeing issues with retries on the v2 source/sink we can dig in futher.

v2 of the Vector protocol is a gRPC-based protocol. You can see this https://vector.dev/highlights/2021-08-24-vector-source-sink/ for more details about this change.