Skip to content

Prepare MSQ/Dart to execute QueryDefinitions planned upfront #17895

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

kgyrtkirk
Copy link
Member

@kgyrtkirk kgyrtkirk commented Apr 9, 2025

These changes aim to center the MSQ/Dart execution around the QueryDefinition - and prepares for a possible future removal of the Query object from the MSQSpec object.

  • renames old MSQSpec to LegacyMSQSpec
  • new MSQSpec contains QueryDefinition
  • migrates most places to use the new MSQSpec ; important exception is that the task retains the old
  • makes Context#makeQueryKitSpec to be a factory instead

@github-actions github-actions bot added Area - Batch Ingestion Area - Querying Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262 labels Apr 9, 2025
return query;
}

public String getId()

Check notice

Code scanning / CodeQL

Missing Override annotation Note

This method overrides
MSQSpec.getId
; it is advisable to add an Override annotation.

public interface MSQTerminalStageSpecFactory
{
/**
* Creates a {@link TerminalStageSpec} which determines the final of a query.
*/
TerminalStageSpec createTerminalStageSpec(DruidQuery druidQuery, PlannerContext plannerContext);
TerminalStageSpec createTerminalStageSpec(PlannerContext plannerContext);

Check notice

Code scanning / CodeQL

Useless parameter Note

The parameter 'plannerContext' is never used.
@@ -236,17 +244,26 @@

// This flag is to ensure backward compatibility, as brokers are upgraded after indexers/middlemanagers.
nativeQueryContextOverrides.put(MultiStageQueryContext.WINDOW_FUNCTION_OPERATOR_TRANSFORMATION, true);
boolean isReindex = MSQControllerTask.isReplaceInputDataSourceTask(druidQuery.getQuery(), destination);

Check warning

Code scanning / CodeQL

Dereferenced variable may be null Warning

Variable
druidQuery
may be null at this access as suggested by
this
null guard.
Variable
druidQuery
may be null at this access as suggested by
this
null guard.
Comment on lines +255 to +256
LegacyMSQSpec.builder()
.query(druidQuery==null?null:druidQuery.getQuery().withOverriddenContext(nativeQueryContextOverrides))

Check notice

Code scanning / CodeQL

Deprecated method or constructor invocation Note

Invoking
Builder.query
should be avoided because it has been deprecated.
@kgyrtkirk kgyrtkirk marked this pull request as draft April 9, 2025 16:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area - Batch Ingestion Area - MSQ For multi stage queries - https://github.com/apache/druid/issues/12262 Area - Querying
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant