pipe: add flb_pipe_error #10017

braydonk · 2025-02-26T21:13:09Z

On Windows, the flb_pipe_r and flb_pipe_w macros do not set errno on failure, meaning calling flb_errno in error scenarios is insufficient. This PR adds a new macro that will check the correct place, WSAGetLastError, and output a similar error message. On Linux this will still be flb_errno, meaning messages should work the same as they always did, but now on Windows we will get actual error messages.

Fixes #3146

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

Example configuration file for the change
Debug log output from testing the change

I had trouble finding a scenario wherein a proper winsock.h failure is propagated. In #3146 they mentioned sending to an elasticsearch host that is not available, but the pattern for how the http client handles that error appears to have changed such that a direct pipe read error isn't propagated.

To test the log format, I made this local change on Windows:

--- a/src/flb_utils.c
+++ b/src/flb_utils.c
@@ -494,7 +494,9 @@ int flb_utils_timer_consume(flb_pipefd_t fd)
     int ret;
     uint64_t val;

-    ret = flb_pipe_r(fd, &val, sizeof(val));
+    // ret = flb_pipe_r(fd, &val, sizeof(val));
+    ret = -1;
+    WSASetLastError(WSAEADDRINUSE);
     if (ret == -1) {
         flb_pipe_error();
         return -1;

Resulting log:

[2025/02/27 18:01:58] [error] [C:\Users\braydonk\Git\fluent-bit\src\flb_utils.c:501 WSAGetLastError=10048] Only one usage of each socket address (protocol/network address/port) is normally permitted.

On Linux I made the following similar local change:

--- a/src/flb_utils.c
+++ b/src/flb_utils.c
@@ -494,7 +494,9 @@ int flb_utils_timer_consume(flb_pipefd_t fd)
     int ret;
     uint64_t val;
 
-    ret = flb_pipe_r(fd, &val, sizeof(val));
+    // ret = flb_pipe_r(fd, &val, sizeof(val));
+    ret = -1;
+    errno = EADDRINUSE;
     if (ret == -1) {
         flb_pipe_error();
         return -1;

And this is the resulting error message:

[2025/02/28 13:57:17] [error] [/usr/local/google/home/braydonk/Git/fluent-bit/src/flb_utils.c:501 errno=98] Address already in use

Attached Valgrind output that shows no leaks or memory corruption was found

Documentation

Docs PR should not be required.

Backporting

Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

On Windows, the `flb_pipe_r` and `flb_pipe_w` macros do not set errno on failure, meaning calling `flb_errno` in error scenarios is insufficient. This PR adds a new macro that will check the correct place, `WSAGetLastError`, and output a similar error message. On Linux this will still be `flb_errno`, meaning messages should work the same as they always did, but now on Windows we will get actual error messages. Signed-off-by: braydonk <braydonk@google.com>

braydonk · 2025-02-28T14:00:10Z

That s390x failure doesn't look related to this PR.

On Windows, the `flb_pipe_r` and `flb_pipe_w` macros do not set errno on failure, meaning calling `flb_errno` in error scenarios is insufficient. This PR adds a new macro that will check the correct place, `WSAGetLastError`, and output a similar error message. On Linux this will still be `flb_errno`, meaning messages should work the same as they always did, but now on Windows we will get actual error messages. Signed-off-by: braydonk <braydonk@google.com>

leonardo-albertovich · 2025-03-10T15:07:33Z

include/fluent-bit/flb_log.h

@@ -232,11 +232,18 @@ static inline int flb_log_suppress_check(int log_suppress_interval, const char *
 int flb_log_worker_init(struct flb_worker *worker);
 int flb_log_worker_destroy(struct flb_worker *worker);
 int flb_errno_print(int errnum, const char *file, int line);
+int flb_WSAGetLastError_print(int errnum, const char *file, int line);


Could you please change these names to follow the coding style?

leonardo-albertovich · 2025-03-10T15:07:43Z

include/fluent-bit/flb_log.h


 #ifdef __FLB_FILENAME__
 #define flb_errno() flb_errno_print(errno, __FLB_FILENAME__, __LINE__)
+#ifdef WIN32
+#define flb_WSAGetLastError() flb_WSAGetLastError_print(WSAGetLastError(), __FLB_FILENAME__, __LINE__)


Could you please change these names to follow the coding style?

I think this function should be renamed, otherwise the current name suggests it would return the error code rather than printing it.

This is just matching the same pattern as flb_errno currently. How should the name change? And should the name of flb_errno change given that it provides the same potential confusion?

I'd appreciate it if you named it flb_wsa_get_last_error.

I agree with you that flb_errno presents the same issue in terms of its name but as you might imagine changing its name would involve a boat load of changes which even if minimal would be super noisy so it might not be the right time for that.

What do you think of having both flb_errno and flb_wsa_get_last_error_print return the error code in addition to printing it just as a first step?

I understand that this would basically eat the error code that the function itself could generate (if error conditions were handled) which could make this a step in the wrong direction so let me know what you think about that.

The "critical" thing to me is changing those names from camel case to snake case.

Although no callers will currently use that return value, I don't see any actual harm in simply returning the errnum value. Did that along with the rename in 429bbe1

leonardo-albertovich · 2025-03-10T15:10:14Z

src/flb_log.c

@@ -121,7 +126,7 @@ static inline int log_read(flb_pipefd_t fd, struct flb_log *log)
    bytes = flb_pipe_read_all(fd, &msg, sizeof(struct log_message));

    if (bytes <= 0) {
-        flb_errno();
+        flb_pipe_error();


In this case there is an flb_error call in the second error exitt code path inside of flb_pipe_read_all which could cause confusion, could you remove it?

Good catch, fixed in 7a1429c

leonardo-albertovich · 2025-03-10T15:10:40Z

src/flb_log.c

+                  buf, sizeof(buf), NULL);
+    flb_error("[%s:%i WSAGetLastError=%i] %s", file, line, errnum, buf);
+#endif
+}


Please add a return value or change the return type to void

Added a return value to keep it lined up with flb_errno in 7a1429c

leonardo-albertovich · 2025-03-10T15:13:03Z

src/flb_log.c

@@ -745,6 +750,17 @@ int flb_errno_print(int errnum, const char *file, int line)
    return 0;
 }

+int flb_WSAGetLastError_print(int errnum, const char *file, int line)
+{
+#ifdef WIN32


Would it be better to make the body of the function or the whole function conditional?
This function is only referenced by flb_WSAGetLastError if our target is Windows so I don't think making the whole thing conditional would be a problem.

Yep fair assessment. Done in 7a1429c

…flb_pipe_error

Signed-off-by: braydonk <braydonk@google.com>

braydonk requested review from PettitWesley, edsiper, niedbalski, leonardo-albertovich, fujimotos and koleini as code owners February 26, 2025 21:13

github-actions bot added the docs-required label Feb 26, 2025

braydonk temporarily deployed to pr February 26, 2025 21:13 — with GitHub Actions Inactive

braydonk force-pushed the flb_pipe_error branch from eb05f25 to 864216c Compare February 26, 2025 21:17

braydonk temporarily deployed to pr February 26, 2025 21:18 — with GitHub Actions Inactive

braydonk temporarily deployed to pr February 26, 2025 21:40 — with GitHub Actions Inactive

braydonk temporarily deployed to pr February 26, 2025 21:41 — with GitHub Actions Inactive

edsiper added this to the Fluent Bit v4.0.0 milestone Feb 28, 2025

braydonk force-pushed the flb_pipe_error branch from 864216c to c66851c Compare March 4, 2025 21:26

braydonk temporarily deployed to pr March 4, 2025 21:27 — with GitHub Actions Inactive

braydonk temporarily deployed to pr March 4, 2025 21:48 — with GitHub Actions Inactive

leonardo-albertovich requested changes Mar 10, 2025

View reviewed changes

Merge branch 'flb_pipe_error' of github.com:braydonk/fluent-bit into …

05f391c

…flb_pipe_error

log: address wsa error function comments

7a1429c

Signed-off-by: braydonk <braydonk@google.com>

braydonk temporarily deployed to pr March 13, 2025 15:50 — with GitHub Actions Inactive

braydonk temporarily deployed to pr March 13, 2025 16:15 — with GitHub Actions Inactive

braydonk temporarily deployed to pr March 13, 2025 16:16 — with GitHub Actions Inactive

log: rename get last error print

429bbe1

Signed-off-by: braydonk <braydonk@google.com>

braydonk temporarily deployed to pr March 14, 2025 12:47 — with GitHub Actions Inactive

braydonk temporarily deployed to pr March 14, 2025 13:09 — with GitHub Actions Inactive

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pipe: add flb_pipe_error #10017

pipe: add flb_pipe_error #10017

braydonk commented Feb 26, 2025 •

edited

Loading

braydonk commented Feb 28, 2025

leonardo-albertovich Mar 10, 2025

leonardo-albertovich Mar 10, 2025

leonardo-albertovich Mar 10, 2025

braydonk Mar 13, 2025

leonardo-albertovich Mar 14, 2025

braydonk Mar 14, 2025

leonardo-albertovich Mar 10, 2025

braydonk Mar 13, 2025

leonardo-albertovich Mar 10, 2025

braydonk Mar 13, 2025

leonardo-albertovich Mar 10, 2025

braydonk Mar 13, 2025

pipe: add flb_pipe_error #10017

Are you sure you want to change the base?

pipe: add flb_pipe_error #10017

Conversation

braydonk commented Feb 26, 2025 • edited Loading

braydonk commented Feb 28, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

braydonk commented Feb 26, 2025 •

edited

Loading