Skip to content

Conversation

abbbe
Copy link

@abbbe abbbe commented Jun 10, 2025

I have customised your tool to add an option to target arbitrary LLM implementations, not only ones supported by LangChains. I have done this to be able to target an LLM exposed via a custom REST API.

To use this mode user has to provide a custom python module implementing two methods. Please see ps_fuzz/custom.py for sample implementation (it uses a local transformer-based model for testing).

To test: prompt-security-fuzzer --target-provider custom --target-model ps_fuzz.custom -b

If you find it useful and willing to merge in -- please let me know. I will adjust README and provide sample client for REST API.

@abbbe abbbe changed the title Implement custom clients in addition to ones supported by ChainLang Implement custom clients in addition to ones supported by LangChain Jun 10, 2025
@vitaly-ps
Copy link
Contributor

Hi @abbbe

It seems I am not able to add PR to your cloned repo (it doesn't allow anonymous contributions).

So please, update your PR: add this line to setup.py in the install_requires=[] list (after "fastparquet" line):

+        "setuptools>=80.0.0,<81.0.0"

This is lacking for your addition of the new "plugin-based" system.

Also, if you can, please add a short section in README describing your new "plugins" feature.

Also, suggest you pull the latest commits from the upstream repo (https://github.com/prompt-security
ps-fuzz), it contains some bugfixes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants