You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is this a new feature, an improvement, or a change to existing functionality?
New Feature
How would you describe the priority of this feature request
Medium
Please provide a clear description of problem this feature solves
This feature will allow evaluating a workflow via the generate endpoint (this feature is mostly there and requires a few minor fixups to be re-enabled) -
Start workflow serve same as before - aiq serve --config_file=examples/simple/configs/config.yml
Run eval with a config file that has the ground truth dataset - aiq eval --config_file examples/simple/configs/eval_config.yml --help --endpoint http://localhost:8000/generate
Describe your ideal solution
Run workflow via serve
Evaluate via endpoint
Additional context
No response
Code of Conduct
I agree to follow this project's Code of Conduct
I have searched the open feature requests and have found no duplicates for this feature request
The text was updated successfully, but these errors were encountered:
Is this a new feature, an improvement, or a change to existing functionality?
New Feature
How would you describe the priority of this feature request
Medium
Please provide a clear description of problem this feature solves
This feature will allow evaluating a workflow via the generate endpoint (this feature is mostly there and requires a few minor fixups to be re-enabled) -
aiq serve --config_file=examples/simple/configs/config.yml
aiq eval --config_file examples/simple/configs/eval_config.yml --help --endpoint http://localhost:8000/generate
Describe your ideal solution
Additional context
No response
Code of Conduct
The text was updated successfully, but these errors were encountered: