You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: pipeline/filters/log_to_metrics.md
+16-4
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@ description: Generate metrics from logs
8
8
9
9
## Log To Metrics
10
10
11
-
The _Log To Metrics Filter_ plugin allows you to generate log-derived metrics. It currently supports modes to count records, provide a gauge for field values or create a histogram. You can also match or exclude specific records based on regular expression patterns for values or nested values. This filter plugin does not actually act as a record filter and does not change or drop records. All records will pass this filter untouched and generated metrics will be emitted into a seperate metric pipeline.
11
+
The _Log To Metrics Filter_ plugin allows you to generate log-derived metrics. It currently supports modes to count records or field values, provide a gauge for field values or create a histogram. You can also match or exclude specific records based on regular expression patterns for values or nested values. This filter plugin does not actually act as a record filter and does not change or drop records. All records will pass this filter untouched and generated metrics will be emitted into a seperate metric pipeline.
12
12
13
13
_Please note that this plugin is an experimental feature and is not recommended for production use. Configuration parameters and plugin functionality are subject to change without notice._
14
14
@@ -25,7 +25,7 @@ The plugin supports the following configuration parameters:
25
25
| bucket | Defines a bucket for `histogram`| Yes, for mode `histogram`| e.g. 0.75 |
26
26
| add\_label | Add a custom label NAME and set the value to the value of KEY |||
27
27
| label\_field | Includes a record field as label dimension in the metric. || Name of record key. Supports [Record Accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) notation for nested fields. |
28
-
| value\_field | Specify the record field that holds a numerical value | Yes, for modes \[`gauge` and `histogram`]| Name of record key. Supports [Record Accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) notation for nested fields. |
28
+
| value\_field | Specify the record field that holds a numerical value | Yes, for modes \[`gauge` and `histogram`, optional for `counter`]| Name of record key. Supports [Record Accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) notation for nested fields. |
29
29
| kubernetes\_mode | If enabled, it will automatically put pod\_id, pod\_name, namespace\_name, docker\_id and container\_name into the metric as labels. This option is intended to be used in combination with the [kubernetes](kubernetes.md) filter plugin, which fills those fields. |||
30
30
| Regex | Include records in which the content of KEY matches the regular expression. || KEY REGEX |
31
31
| Exclude | Exclude records in which the content of KEY matches the regular expression. || KEY REGEX |
@@ -34,7 +34,7 @@ The plugin supports the following configuration parameters:
34
34
35
35
### Getting Started
36
36
37
-
The following example takes records from two dummy inputs and counts all messages passing through the `log_to_metrics` filter. It then generates metric records which are provided to the `prometheus_exporter`:
37
+
The following example takes records from two dummy inputs and generates two different counter metrics: It counts all messages passing through the `log_to_metrics` filter and it counts the total time spend according to the duration field of each message. It then generates metric records which are provided to the `prometheus_exporter`:
38
38
39
39
#### Configuration - Counter
40
40
@@ -61,6 +61,15 @@ The following example takes records from two dummy inputs and counts all message
61
61
metric_name count_all_dummy_messages
62
62
metric_description This metric counts dummy messages
63
63
64
+
[FILTER]
65
+
name log_to_metrics
66
+
match dummy.log*
67
+
tag test_metric
68
+
metric_mode counter
69
+
metric_name total_duration
70
+
metric_description This metric counts the total time spend
71
+
value_field duration
72
+
64
73
[OUTPUT]
65
74
name prometheus_exporter
66
75
match *
@@ -76,7 +85,10 @@ You can then use e.g. curl command to retrieve the generated metric:
76
85
77
86
# HELP log_metric_counter_count_all_dummy_messages This metric counts dummy messages
78
87
# TYPE log_metric_counter_count_all_dummy_messages counter
79
-
log_metric_counter_count_all_dummy_messages 49
88
+
log_metric_counter_count_all_dummy_messages 24
89
+
# HELP log_metric_counter_total_duration This metric counts the total time spend
0 commit comments