feat: add image text model provider separation and fal.ai integration #650
+129
−12
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Relates to:
Risks
Low - Adds optional fal.ai integration for image generation. Existing functionality remains unchanged when new optional imageModelProvider is not set.
Background
What does this PR do?
FAL_API_KEY
andFAL_AI_LORA_PATH
environment variablesWhat kind of change is this?
Features (non-breaking change which adds functionality)
Documentation changes needed?
My changes require a change to the project documentation to document:
Testing
Where should a reviewer start?
agent/src/index.ts
- Updated to include image generation plugin should one of the valid image generation model api keys be included.packages/core/src/generation.ts
- Review fal.ai integration. Changed to look at model provider on runtime instance.if modelProvider = imageModelProvider
, use token otherwise use one of the image env vars set.packages/core/src/models.ts
- Added new entry for FAL modelpackages/core/src/types.ts
- Check newimageModelProvider
type additionspackages/core/src/runtime.ts
- ReviewimageModelProvider
implementation. Updated to setimageModelProvider
if one is provided in character config otherwise default tomodelProvider
.Detailed testing steps
fal ai api testing:
FAL_API_KEY
to .envimageModelProvider
: "falai"FAL_AI_LORA_PATH
same
modelProvider
toimageModelProvider
tests:OPENAI_API_KEY
modelProvider
to "openai" in character configdifferent
modelProvider
toimageModelProvider
tests:OPENAI_API_KEY
&FAL_API_KEY
modelProvider
to "openai" andimageModelProvider
to "falai" in character configdifferent
modelProvider
toimageModelProvider
tests:OPENAI_API_KEY
&HEURIST_API_KEY
modelProvider
to "openai" andimageModelProvider
to "heurist" in character configDiscord username
yoniebans