-
|
Hello, I have this mysql slow query log content bebow I try to parse the log but couldn't have good result. Here is my vector config: Here is the result in console: Questions:
If you have a better way to parse mysql slow query log, please tell me. I tried to search but could get the info I need.... |
Beta Was this translation helpful? Give feedback.
Answered by
milkrage
Oct 22, 2024
Replies: 1 comment 3 replies
-
|
Improved version should be. I got all the info I need. |
Beta Was this translation helpful? Give feedback.
3 replies
-
|
Why i can't. MySQL version is 8.0.35. |
Beta Was this translation helpful? Give feedback.
-
What are your logs? Can you try to show some example entries? |
Beta Was this translation helpful? Give feedback.
-
|
The slow query log file can contain not only information about slow queries, but also other service information. Entries that are not related to slow queries will cause parsing errors. As a solution, you can use In the example I use the Example: sources:
slow_log_source:
type: "file"
data_dir: .
include:
- slow.log
read_from: "end"
multiline:
condition_pattern: "(^#.*|.*;$)"
start_pattern: "^# Time.*"
mode: continue_through
timeout_ms: 1000
transforms:
slow_log:
inputs:
- slow_log_source
type: "remap"
drop_on_error: true
source: |-
.content, err = parse_regex(.message, r'^# Time: (?<time>[^ ]+)\n# User@Host: (?<user>[^*]+) @ (?<host>[^*]+) \[\] Id: (?<id>[^*]+)\n# Schema: Last_errno: (?<errno>[^ ]+) Killed: (?<killed>[^ ]+)\n# Query_time: (?<query_time>[^ ]+) Lock_time: (?<lock_time>[^ ]+) Rows_sent: (?<rows_sent>[^ ]+) Rows_examined: (?<rows_examined>[^ ]+) Rows_affected: (?<rows_affected>[^ ]+) Bytes_sent: (?<bytes_sent>[^*]+)\nSET timestamp=(?<timestamp>[^ ]+);\n(?<sql>[^*]*);$')
if err != null {
abort
}
.timestamp = parse_timestamp!(.content.time, format: "%+")
.user = .content.user
.host = .content.host
.query_time = parse_float!(.content.query_time)
.lock_time = parse_float!(.content.lock_time)
.rows_sent = parse_int!(.content.rows_sent)
.rows_examined = parse_int!(.content.rows_examined)
.rows_affected = parse_int!(.content.rows_affected)
.bytes_sent = parse_int!(.content.bytes_sent)
.sql = .content.sql
del(.content)
del(.message)
del(.file)
del(.source_type)
sinks:
stdout_slow:
type: console
inputs:
- slow_log
target: stdout
encoding:
codec: jsonFor testing I used the following entry: echo "# Time: 2024-10-20T00:00:05.330194Z
# User@Host: root[root] @ localhost [] Id: 1590103
# Schema: Last_errno: 0 Killed: 0
# Query_time: 2.000299 Lock_time: 0.000000 Rows_sent: 1 Rows_examined: 1 Rows_affected: 0 Bytes_sent: 56
SET timestamp=1729382403;
select sleep(2);
2024-09-12T12:17:16.578671Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started." >> slow.log |
Beta Was this translation helpful? Give feedback.
Answer selected by
jszwedko
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The slow query log file can contain not only information about slow queries, but also other service information. Entries that are not related to slow queries will cause parsing errors. As a solution, you can use
drop_on_errorandifwithabort.In the example I use the
continue_throughmodeExample: