How to specify logline/content in loki sink?

[9Lucky9]

Hi, how to specify logline/content in loki sink?

Current config works labels are applied, but all json lines is written to logline and i only need .message. I know i could delete some fields by using del(), but i can't delete .app, .hostname, .level as they are used as lables later in the sink.

Here's my values.yaml file.

role: Agent

logLevel: "debug"

customConfig:
  data_dir: /vector-data-dir
  api:
    enabled: true
    address: 0.0.0.0:8686
    playground: false
  sources:
    vitta_app_logs:
      type: kubernetes_logs
      extra_label_selector: "developed-by=lucky"
  transforms:
    vitta_app_logs_transform:
      type: remap
      inputs:
        - vitta_app_logs
      source: |
        .app = .kubernetes.pod_labels."app.kubernetes.io/name"
        .hostname = .kubernetes.node_labels."kubernetes.io/hostname"
        .messageJson = parse_json!(.message)
        .level = .messageJson.Level
        .message = .messageJson.Message

  sinks:
    console:
      type: console
      inputs:
        - vitta_app_logs_transform
      encoding:
        codec: "json"
    vitta_app_loki_sink:
      type: loki
      inputs:
        - vitta_app_logs_transform
      compression: snappy
      encoding:
        codec: "json"
      endpoint: http://redacted.redacted:9999
      labels:
        app: "{{`{{.app}}`}}"
        hostname: "{{`{{.hostname}}`}}"
        level: "{{`{{.level}}`}}"
      path: /loki/loki/api/v1/push
      remove_timestamp: true
      tenant_id: fake

resources:
  requests:
    cpu: 200m
    memory: 64Mi
  limits:
    cpu: 200m
    memory: 256Mi

[9Lucky9]

So i figured this out by doing this:

1. I deleted another fields using a del() function like that:

del(.file)
del(.kubernetes)
del(.source_type)
del(.messageJson)
del(.stream)

2. I added field remove_label_fields: true to loki sink.
3. I changed encoding.codec to "text"

If this is not the preferred/best way of doing this, correct me.

Final values.yaml config:

role: Agent

logLevel: "debug"

customConfig:
  data_dir: /vector-data-dir
  api:
    enabled: true
    address: 0.0.0.0:8686
    playground: false
  sources:
    vitta_app_logs:
      type: kubernetes_logs
      extra_label_selector: "developed-by=lucky"
  transforms:
    vitta_app_logs_transform:
      type: remap
      inputs:
        - vitta_app_logs
      source: |
        .app = .kubernetes.pod_labels."app.kubernetes.io/name"
        .hostname = .kubernetes.node_labels."kubernetes.io/hostname"
        .messageJson = parse_json!(.message)
        .level = .messageJson.Level
        .message = .messageJson.Message
        del(.file)
        del(.kubernetes)
        del(.source_type)
        del(.messageJson)
        del(.stream)
  sinks:
    console:
      type: console
      inputs:
        - vitta_app_logs_transform
      encoding:
        codec: "json"
    vitta_app_loki_sink:
      type: loki
      inputs:
        - vitta_app_logs_transform
      compression: snappy
      encoding:
        codec: "text"
      endpoint: http://redacted.redacted:9999
      labels:
        app: "{{`{{.app}}`}}"
        hostname: "{{`{{.hostname}}`}}"
        level: "{{`{{.level}}`}}"
      path: /loki/loki/api/v1/push
      remove_label_fields: true
      remove_timestamp: true
      tenant_id: fake

resources:
  requests:
    cpu: 200m
    memory: 64Mi
  limits:
    cpu: 200m
    memory: 256Mi

[pront]

What you did above seems reasonable. Thank you for sharing your solution. Other users might benefit from this in the future.