-
Notifications
You must be signed in to change notification settings - Fork 3.8k
Labels
Description
Is your feature request related to a problem? Please describe.
When loki queriers scale due to an increase in requests for reads, it takes 30s before the querier pods are marked as ready and start taking traffic.
readinessProbe:
failureThreshold: 3
httpGet:
path: /ready
port: http-metrics
scheme: HTTP
initialDelaySeconds: 30
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
This causes delays in the ability to scale out to handle the increase in searches.
Describe the solution you'd like
The readiness probe configurations for loki queriers should be configurable to allow fields such as initialDelaySeconds
can be customized. Currently the configuration is using a default global configuration:
- https://github.com/grafana/loki/blob/main/production/helm/loki/values.yaml#L68-L73
- https://github.com/grafana/loki/blob/main/production/helm/loki/templates/querier/deployment-querier.yaml#L104-L111
Introduce a new loki.querier.readinessProbe
value that would be merged with the default loki.readinessProbe
to allow this customization
Describe alternatives you've considered
None
Additional context