-
Notifications
You must be signed in to change notification settings - Fork 2k
Closed
Labels
enhancementNew feature or requestNew feature or request
Description
Problem
Currently, the library uses hardcoded logic in supports_stop_parameter() to determine which models support the stop parameter. This approach has several limitations:
- Maintenance burden: Every new model that doesn't support stop requires updating the hardcoded regex pattern
- Inflexibility: Users cannot override the library's assumptions about their specific model's capabilities
- Limited scope: The same problem exists for other parameters beyond just stop
- Breaking changes: Users with custom deployments or newer models may hit errors until the library is updated
For example, when GPT-5 was released, we had to update the regex pattern:
- Fix error for GPT-5 not supporting the
stop
parameter #1662- from:
pattern = r"^(o3[-\d]*|o4-mini[-\d]*)$"
- to:
pattern = r"^(o3[-\d]*|o4-mini[-\d]*|gpt-5(-mini|-nano)?[-\d]*)$"
- from:
Proposed solution
Implement a flexible parameter override system that allows users to:
- Explicitly control
stop
parameter behavior - Override any completion parameter
- Remove parameters completely
Benefits
- No breaking changes: Existing code continues to work unchanged
- Future-proof: No need to update hardcoded lists when new models are released
- User control: Users can override library assumptions based on their specific needs
- Flexible: System works for any parameter, not just stop
- Pythonic: Clean, intuitive API using dictionaries and sentinel values
Checklist
- I have searched the existing issues and have not found a similar feature request.
- I have verified that this feature is not already implemented in the latest version.
- I am willing to work on this feature and submit a pull request. (optional)
kutsushitaneko
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request