Refactor message extraction to support reasoning content #62
Conversation
• Introduce CompletionDeltaModel to encapsulate reasoning and standard content. • Update OpenAIChatCompletionEventSourceListener to wrap reasoning text with <think> tags and properly transition to normal content. • Update JSON deserialization in OpenAIChatCompletionResponseChoiceDelta to extract reasoning_content. • Bump version to 0.8.39.
| } | ||
|
|
||
| @Override | ||
| protected ErrorDetails getErrorDetails(String data) throws JsonProcessingException { | ||
| return OBJECT_MAPPER.readValue(data, ApiResponseError.class).getError(); | ||
| } | ||
|
|
||
| private String getMessageFromDeltaModel(CompletionDeltaModel completionDeltaModel) { |
There was a problem hiding this comment.
This seems like an ugly hack to make it compliant with how ProxyAI parses the thinking output.
Perhaps we can change the logic of how and where the respected thinking output is taken. For example, similar support (new param) was just recently added for Google Gemini requests as well, but as far as I know, it's not accepted in the UI logic yet.
<think> tags were DeepSeek's invention and the first ever model displaying its thinking process, hence the logic in the plugin repo.
There was a problem hiding this comment.
I tried to do this by replacing the return type from a string to a class with two fields (message, reasoningContent). but it turned out that a lot of changes had to be made on the plug-in side, because the OpenAI format is used not only by CustomOpenAI, but also by many other providers. it didn't seem advisable without prior refactoring.
There was a problem hiding this comment.
Looking ahead, I think there will be a lot of changes when mcp integration begins, which will require rewriting the integration of the llm client library.
I'm waiting for a resolution, leave it that way, or still make the transition to the class in the OpenAI response.
| @@ -7,7 +7,7 @@ plugins { | |||
| } | |||
|
|
|||
| group = "ee.carlrobert" | |||
| version = "0.8.38" | |||
| version = "0.8.39" | |||
There was a problem hiding this comment.
Can we remove this? I usually increment it when I modify the changelog.
• Introduce CompletionDeltaModel to encapsulate reasoning and standard content. • Update OpenAIChatCompletionEventSourceListener to wrap reasoning text with tags and properly transition to normal content. • Update JSON deserialization in OpenAIChatCompletionResponseChoiceDelta to extract reasoning_content. • Bump version to 0.8.39.