Skip to content

ENH: Add user-configurable parameter override system for model completion parameters #1676

@albertvillanova

Description

@albertvillanova

Problem

Currently, the library uses hardcoded logic in supports_stop_parameter() to determine which models support the stop parameter. This approach has several limitations:

  1. Maintenance burden: Every new model that doesn't support stop requires updating the hardcoded regex pattern
  2. Inflexibility: Users cannot override the library's assumptions about their specific model's capabilities
  3. Limited scope: The same problem exists for other parameters beyond just stop
  4. Breaking changes: Users with custom deployments or newer models may hit errors until the library is updated

For example, when GPT-5 was released, we had to update the regex pattern:

Proposed solution

Implement a flexible parameter override system that allows users to:

  1. Explicitly control stop parameter behavior
  2. Override any completion parameter
  3. Remove parameters completely

Benefits

  • No breaking changes: Existing code continues to work unchanged
  • Future-proof: No need to update hardcoded lists when new models are released
  • User control: Users can override library assumptions based on their specific needs
  • Flexible: System works for any parameter, not just stop
  • Pythonic: Clean, intuitive API using dictionaries and sentinel values

Checklist

  • I have searched the existing issues and have not found a similar feature request.
  • I have verified that this feature is not already implemented in the latest version.
  • I am willing to work on this feature and submit a pull request. (optional)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions