Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KAFKA-18894: Add KIP-877 support for ConfigProvider #19397

Open
wants to merge 9 commits into
base: trunk
Choose a base branch
from

Conversation

Yunyung
Copy link
Contributor

@Yunyung Yunyung commented Apr 7, 2025

Add KIP-877 support for ConfigProvider.

Cleanup: Collections.singletonMap() -> Map.of()

Jira: https://issues.apache.org/jira/browse/KAFKA-18894

@m1a2st
Copy link
Collaborator

m1a2st commented Apr 7, 2025

Thanks for this patch, please use ./gradlew spotlessApply fix build error

@Yunyung Yunyung marked this pull request as draft April 8, 2025 02:50
@Yunyung Yunyung marked this pull request as ready for review April 8, 2025 05:31
Copy link
Collaborator

@m1a2st m1a2st left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Yunyung for this patch, left some comments

Comment on lines +591 to +596
* returns a map of config provider name and its instance wrapped in a {@link org.apache.kafka.common.internals.Plugin}.
*
* @param indirectConfigs The map of potential variable configs
* @param providerConfigProperties The map of config provider configs
* @param classNameFilter Filter for config provider class names
* @return map of config provider name and its instance.
* @return map of config provider name and its instance wrapped in a {@link org.apache.kafka.common.internals.Plugin}.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could keep the original document, FYI #19050 (comment)

Copy link
Contributor Author

@Yunyung Yunyung Apr 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. I'll update all of them later.

for (Map.Entry<String, String> entry : providerMap.entrySet()) {
try {
String prefix = CONFIG_PROVIDERS_CONFIG + "." + entry.getKey() + CONFIG_PROVIDERS_PARAM;
Map<String, ?> configProperties = configProviderProperties(prefix, providerConfigProperties);
ConfigProvider provider = Utils.newInstance(entry.getValue(), ConfigProvider.class);
provider.configure(configProperties);
configProviderInstances.put(entry.getKey(), provider);
Plugin<ConfigProvider> providerPlugin = Plugin.wrapInstance(provider, null, CONFIG_PROVIDERS_CONFIG);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to wrap provider into a Plugin at this point, since the metrics are null and it doesn't seem to serve any practical purpose.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is for ConfigTransformer#transform, so we need to update the path where the transform method is used.

Comment on lines -276 to +279
CONFIG_PROVIDERS_CONFIG + "." + name,
Plugins.ClassLoaderUsage.PLUGINS
name,
Plugins.ClassLoaderUsage.PLUGINS,
null
Copy link
Collaborator

@m1a2st m1a2st Apr 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto: I don't think we need to wrap provider into a Plugin at this point, since the metrics are null and it doesn't seem to serve any practical purpose.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

Copy link
Contributor Author

@Yunyung Yunyung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review! PTAL again.

for (Map.Entry<String, String> entry : providerMap.entrySet()) {
try {
String prefix = CONFIG_PROVIDERS_CONFIG + "." + entry.getKey() + CONFIG_PROVIDERS_PARAM;
Map<String, ?> configProperties = configProviderProperties(prefix, providerConfigProperties);
ConfigProvider provider = Utils.newInstance(entry.getValue(), ConfigProvider.class);
provider.configure(configProperties);
configProviderInstances.put(entry.getKey(), provider);
Plugin<ConfigProvider> providerPlugin = Plugin.wrapInstance(provider, null, CONFIG_PROVIDERS_CONFIG);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is for ConfigTransformer#transform, so we need to update the path where the transform method is used.

Comment on lines -276 to +279
CONFIG_PROVIDERS_CONFIG + "." + name,
Plugins.ClassLoaderUsage.PLUGINS
name,
Plugins.ClassLoaderUsage.PLUGINS,
null
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

Comment on lines +591 to +596
* returns a map of config provider name and its instance wrapped in a {@link org.apache.kafka.common.internals.Plugin}.
*
* @param indirectConfigs The map of potential variable configs
* @param providerConfigProperties The map of config provider configs
* @param classNameFilter Filter for config provider class names
* @return map of config provider name and its instance.
* @return map of config provider name and its instance wrapped in a {@link org.apache.kafka.common.internals.Plugin}.
Copy link
Contributor Author

@Yunyung Yunyung Apr 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. I'll update all of them later.

@github-actions github-actions bot removed the triage PRs from the community label Apr 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants