Improve docs #79
Improve docs #79
Conversation
|
Warning Rate limit exceeded@Agent-Hellboy has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 13 minutes and 7 seconds before requesting another review. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📒 Files selected for processing (14)
WalkthroughDocumentation overhaul adds a new Runtime Management layer (ProcessManager, ProcessWatchdog, AsyncFuzzExecutor), expands Strategy System and configuration (YAML/TOML, profiles, validation), updates getting started and index, introduces a Process Management Guide, and extends reference and examples. Some sections are duplicated in examples and reference. No code changes. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor User
participant CLI as CLI
participant Client as Fuzzer Client
participant Exec as AsyncFuzzExecutor
participant PM as ProcessManager
participant WD as ProcessWatchdog
participant Target as Target Server
User->>CLI: Run fuzz command
CLI->>Client: Initialize session/config
Client->>WD: Start watchdog
Client->>PM: start_process(command, config)
PM->>WD: register_process(pid, activity_cb)
PM->>Target: Spawn subprocess (async)
Client->>Exec: execute_batch(tasks, max_concurrency)
loop For each fuzz task
Exec->>Target: Send fuzz input (async)
Target-->>Exec: Response/error
Exec-->>Client: Result/exception
PM->>WD: update_activity(pid)
end
alt Graceful stop
Client->>PM: stop_all_processes()
PM->>Target: SIGTERM (proc group)
else Force stop
WD-->>PM: Hang detected
PM->>Target: SIGKILL (proc group)
end
Client->>WD: stop()
Client-->>CLI: Summary & reports
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Assessment against linked issues
Assessment against linked issues: Out-of-scope changes
Possibly related PRs
Poem
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Documentation PreviewYour documentation changes have been built successfully! Preview URL: https://Agent-Hellboy.github.io/mcp-server-fuzzer/ The documentation will be automatically deployed to the main site when this PR is merged. This comment was automatically generated by the documentation workflow. |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #79 +/- ##
==========================================
- Coverage 61.44% 60.52% -0.93%
==========================================
Files 55 55
Lines 4762 4889 +127
Branches 813 838 +25
==========================================
+ Hits 2926 2959 +33
- Misses 1606 1698 +92
- Partials 230 232 +2 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 10
🧹 Nitpick comments (21)
docs/reference.md (5)
103-224: Deduplicate Runtime Management content in ReferenceThe “Runtime Management API Reference” block (Lines 103–224) and the later “## Runtime” section (Lines 882–902+) substantially overlap. Keep one canonical section and link to it to avoid drift.
Apply this minimal replacement to point readers to the canonical API section:
-## Runtime - -The runtime layer provides robust, asynchronous subprocess lifecycle management for transports and target servers under test. - -- Components: - - `ProcessManager` (async): start/stop processes, send signals, await exit, collect stats; integrates with the watchdog. - - `ProcessWatchdog`: monitors registered PIDs for hangs/inactivity and terminates them based on policy. - - All operations are now fully asynchronous using native asyncio. +## Runtime + +This section duplicates the Runtime Management API Reference above. See “Runtime Management API Reference” for the authoritative, non-duplicated content.Also applies to: 882-902
113-117: Clarify ProcessManager constructor parameter name/typeConstructor shows
config: Optional[WatchdogConfig]. Name itwatchdog_configto avoid implying a manager-specific config object.-class ProcessManager: - def __init__(self, config: Optional[WatchdogConfig] = None): - """Initialize the async process manager.""" +class ProcessManager: + def __init__(self, watchdog_config: Optional[WatchdogConfig] = None): + """Initialize the async process manager."""
291-305: Example uses asyncio without importAdd the import so the snippet is copy-paste runnable.
-from mcp_fuzzer.fuzz_engine.runtime.watchdog import ProcessWatchdog, WatchdogConfig +import asyncio +from mcp_fuzzer.fuzz_engine.runtime.watchdog import ProcessWatchdog, WatchdogConfig
366-381: Missing imports for examples using asyncio/randomEnsure examples run as-is.
-from mcp_fuzzer.fuzz_engine.executor import AsyncFuzzExecutor +import asyncio +import random +from mcp_fuzzer.fuzz_engine.executor import AsyncFuzzExecutor
1091-1100: Unify safety class naming: SystemBlocker vs SystemCommandBlockerElsewhere you reference
SystemBlocker; here it saysSystemCommandBlocker. Use one name consistently (prefer the actual class name in code).-- System-level blocking (`mcp_fuzzer.safety_system.system_blocker.SystemCommandBlocker`): +- System-level blocking (`mcp_fuzzer.safety_system.system_blocker.SystemBlocker`):docs/process-management-guide.md (4)
91-100: Add missing asyncio import in snippetThe periodic updater uses asyncio.
-# Or use a periodic updater +# Or use a periodic updater +import asyncio async def periodic_activity_update(manager, pid, interval=5.0):
219-238: Add missing asyncio import in standalone watchdog exampleWithout it, the snippet won’t run.
-async def standalone_watchdog_example(): +import asyncio +async def standalone_watchdog_example():
402-410: Call out optional psutil dependencyReaders may hit ImportError; add an install note.
-import psutil +import psutil # pip install psutil
438-443: Note on cross-platform signalsOn Windows, SIGKILL semantics differ. Add a short note to avoid user confusion.
- Verify process group signaling - Use force kill when needed - Implement proper signal handlers - Review process cleanup code +> Note: Signal handling varies by OS (e.g., Windows lacks POSIX SIGKILL semantics). Adjust strategies accordingly.docs/runtime-management.md (3)
376-389: Import random for retry exampleSnippet uses random.
-async def executor_with_retry(): +import random +async def executor_with_retry():
478-505: Import time for complete example
time.time()is used.-import asyncio +import asyncio +import time from mcp_fuzzer.fuzz_engine.runtime.manager import ProcessManager, ProcessConfig
537-544: Safer command parsing for endpoint stringsUse
shlex.splitto preserve quoted args.- config = ProcessConfig( - command=self.endpoint.split(), + import shlex + config = ProcessConfig( + command=shlex.split(self.endpoint), name="stdio_server", timeout=30.0 )docs/architecture.md (4)
310-310: Fix section numbering: Invariants should be 6Avoid duplicate “5”.
-### 5. Invariants System +### 6. Invariants System
326-326: Bump subsequent section numbers accordingly (Safety → 7)-### 6. Safety System +### 7. Safety System
342-342: Bump Authentication to 8-### 7. Authentication System +### 8. Authentication System
359-359: Bump Reporting to 9-### 8. Reporting System +### 9. Reporting Systemdocs/examples.md (3)
467-505: Add missing import for asyncio in example.The snippet uses asyncio but doesn't import it. Add the import to avoid confusion when readers copy-paste.
-```python -import time +```python +import time +import asyncio
705-721: Standardize --export-safety-data usage.Examples alternate between using a filename and no argument. Clarify whether the flag accepts an optional path and show both forms consistently, or pick one canonical usage.
-# Export safety data to JSON -mcp-fuzzer ... --export-safety-data +# Export safety data to default filename +mcp-fuzzer ... --export-safety-data + +# Or export to a specific file +mcp-fuzzer ... --export-safety-data safety_data.json
801-851: Unused imports in reporter example.ConsoleFormatter, JSONFormatter, TextFormatter aren’t used. Either show usage or remove imports to reduce noise.
-from mcp_fuzzer.reports.formatters import ConsoleFormatter, JSONFormatter, TextFormatterdocs/getting-started.md (2)
79-101: Avoid duplication with “Reporting and Output” section below.This new “Generate Reports” section repeats commands later (lines 248–282). Consider consolidating to one canonical section and link from Quick Start.
-### 4. Generate Reports -... -# (commands) +### 4. Generate Reports +See Reporting and Output for full details. Quick examples: +```bash +mcp-fuzzer --mode tools --protocol stdio --endpoint "python test_server.py" --runs 10 +mcp-fuzzer --mode tools --protocol stdio --endpoint "python test_server.py" --runs 10 --output-dir "my_reports" +mcp-fuzzer --mode tools --protocol stdio --endpoint "python test_server.py" --runs 10 --safety-report --export-safety-data +```
90-95: Standardize flag semantics for --export-safety-data.As in examples.md, show whether it accepts an optional filename and keep usage consistent across docs.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (8)
docs/architecture.md(4 hunks)docs/configuration.md(1 hunks)docs/examples.md(2 hunks)docs/getting-started.md(1 hunks)docs/index.md(1 hunks)docs/process-management-guide.md(1 hunks)docs/reference.md(1 hunks)docs/runtime-management.md(1 hunks)
🧰 Additional context used
🪛 LanguageTool
docs/process-management-guide.md
[grammar] ~338-~338: There might be a mistake here.
Context: ...### 1. Process Not Starting Symptoms: - Process fails to start - Immediate exit ...
(QB_NEW_EN)
[grammar] ~357-~357: There might be a mistake here.
Context: ....cwd} does not exist") ``` Solutions: - Verify command path and arguments - Chec...
(QB_NEW_EN)
[grammar] ~366-~366: There might be a mistake here.
Context: ...gs #### 2. Process Hanging Symptoms: - Process appears to hang - No response to...
(QB_NEW_EN)
[grammar] ~386-~386: There might be a mistake here.
Context: ...g(level=logging.DEBUG) ``` Solutions: - Implement activity callbacks - Increase ...
(QB_NEW_EN)
[grammar] ~395-~395: There might be a mistake here.
Context: ...#### 3. High Resource Usage Symptoms: - High CPU usage - High memory usage - Sys...
(QB_NEW_EN)
[grammar] ~396-~396: There might be a mistake here.
Context: ...ce Usage Symptoms: - High CPU usage - High memory usage - System slowdown **D...
(QB_NEW_EN)
[grammar] ~397-~397: There might be a mistake here.
Context: ...:** - High CPU usage - High memory usage - System slowdown Diagnosis: ```pytho...
(QB_NEW_EN)
[grammar] ~412-~412: There might be a mistake here.
Context: ... 1024 / 1024:.1f} MB") ``` Solutions: - Reduce concurrency limits - Increase che...
(QB_NEW_EN)
[grammar] ~421-~421: There might be a mistake here.
Context: ...# 4. Signal Handling Issues Symptoms: - Processes not terminating - Zombie proce...
(QB_NEW_EN)
[grammar] ~422-~422: There might be a mistake here.
Context: ...Symptoms:* - Processes not terminating - Zombie processes - Signal errors **Diag...
(QB_NEW_EN)
[grammar] ~423-~423: There might be a mistake here.
Context: ...esses not terminating - Zombie processes - Signal errors Diagnosis: ```python ...
(QB_NEW_EN)
[grammar] ~437-~437: There might be a mistake here.
Context: ...g process group: {e}") ``` Solutions: - Verify process handles SIGTERM - Check p...
(QB_NEW_EN)
docs/reference.md
[grammar] ~138-~138: There might be a mistake here.
Context: ...ted subprocess object - Automatically registers process with watchdog - `async stop_pr...
(QB_NEW_EN)
[grammar] ~138-~138: There might be a mistake here.
Context: ...ect - Automatically registers process with watchdog - `async stop_process(pid: in...
(QB_NEW_EN)
[grammar] ~142-~142: There might be a mistake here.
Context: ...acefully or forcefully - Returns True if process was stopped successfully - Us...
(QB_NEW_EN)
[grammar] ~152-~152: There might be a mistake here.
Context: ...for a specific process - Returns None if process is not managed - Includes sta...
(QB_NEW_EN)
[grammar] ~183-~183: There might be a mistake here.
Context: ...l types: "timeout", "force", "interrupt" - Returns True if signal was sent successf...
(QB_NEW_EN)
[grammar] ~184-~184: There might be a mistake here.
Context: ..., "force", "interrupt" - Returns True if signal was sent successfully - `async ...
(QB_NEW_EN)
[grammar] ~273-~273: There might be a mistake here.
Context: ...istered for monitoring - Returns True if process is being monitored - `async ge...
(QB_NEW_EN)
[grammar] ~360-~360: There might be a mistake here.
Context: ... complete - Cancels outstanding tasks if timeout is exceeded - Ensures proper ...
(QB_NEW_EN)
[grammar] ~653-~653: There might be a mistake here.
Context: ...uote-char """ ``` CSV output includes: - Tool name - Run number - Success status ...
(QB_NEW_EN)
[grammar] ~654-~654: There might be a mistake here.
Context: ..."" ``` CSV output includes: - Tool name - Run number - Success status - Response t...
(QB_NEW_EN)
[grammar] ~655-~655: There might be a mistake here.
Context: ...utput includes: - Tool name - Run number - Success status - Response time - Excepti...
(QB_NEW_EN)
[grammar] ~656-~656: There might be a mistake here.
Context: ... Tool name - Run number - Success status - Response time - Exception message (if an...
(QB_NEW_EN)
[grammar] ~657-~657: There might be a mistake here.
Context: ... number - Success status - Response time - Exception message (if any) - Arguments u...
(QB_NEW_EN)
[grammar] ~658-~658: There might be a mistake here.
Context: ...sponse time - Exception message (if any) - Arguments used - Timestamp ### XML Expo...
(QB_NEW_EN)
[grammar] ~659-~659: There might be a mistake here.
Context: ...eption message (if any) - Arguments used - Timestamp ### XML Export Export fuzzin...
(QB_NEW_EN)
[grammar] ~711-~711: There might be a mistake here.
Context: ...ions | Option | Default | Description | |--------|---------|-------------| | `--...
(QB_NEW_EN)
[grammar] ~712-~712: There might be a mistake here.
Context: ...ion | |--------|---------|-------------| | --csv-delimiter | "," | Field delimi...
(QB_NEW_EN)
[grammar] ~713-~713: There might be a mistake here.
Context: ...csv-delimiter| "," | Field delimiter | |--csv-quote-char` | """ | Quote char...
(QB_NEW_EN)
[grammar] ~714-~714: There might be a mistake here.
Context: ...v-quote-char| "\"" | Quote character | |--csv-escape-char` | "\" | Escape ch...
(QB_NEW_EN)
[grammar] ~715-~715: There might be a mistake here.
Context: ...escape-char| "\\" | Escape character | |--csv-line-terminator` | "\n" | Line...
(QB_NEW_EN)
[grammar] ~720-~720: There might be a mistake here.
Context: ...ions | Option | Default | Description | |--------|---------|-------------| | `--...
(QB_NEW_EN)
[grammar] ~721-~721: There might be a mistake here.
Context: ...ion | |--------|---------|-------------| | --xml-indent | 2 | Indentation space...
(QB_NEW_EN)
[grammar] ~722-~722: There might be a mistake here.
Context: ...--xml-indent| 2 | Indentation spaces | |--xml-encoding` | "utf-8" | Character...
(QB_NEW_EN)
[grammar] ~723-~723: There might be a mistake here.
Context: ...coding| "utf-8" | Character encoding | |--xml-root-name` | "fuzzing_results" ...
(QB_NEW_EN)
[grammar] ~724-~724: There might be a mistake here.
Context: ... "fuzzing_results" | Root element name | | --xml-attribute-quotes | "double" | ...
(QB_NEW_EN)
[grammar] ~729-~729: There might be a mistake here.
Context: ...ions | Option | Default | Description | |--------|---------|-------------| | `--...
(QB_NEW_EN)
[grammar] ~730-~730: There might be a mistake here.
Context: ...ion | |--------|---------|-------------| | --html-template | "default" | HTML t...
(QB_NEW_EN)
[grammar] ~731-~731: There might be a mistake here.
Context: ...te| "default" | HTML template to use | |--html-title` | "Fuzzing Results" | P...
(QB_NEW_EN)
[grammar] ~732-~732: There might be a mistake here.
Context: ...itle| "Fuzzing Results" | Page title | |--html-css` | "default" | CSS style t...
(QB_NEW_EN)
[grammar] ~733-~733: There might be a mistake here.
Context: ...-css| "default" | CSS style to apply | |--html-js` | "default" | JavaScript t...
(QB_NEW_EN)
[grammar] ~738-~738: There might be a mistake here.
Context: ...ions | Option | Default | Description | |--------|---------|-------------| | `--...
(QB_NEW_EN)
[grammar] ~739-~739: There might be a mistake here.
Context: ...ion | |--------|---------|-------------| | --markdown-style | "default" | Markd...
(QB_NEW_EN)
[grammar] ~740-~740: There might be a mistake here.
Context: ... Markdown style (github, gitlab, etc.) | | --markdown-toc | false | Include tab...
(QB_NEW_EN)
[grammar] ~741-~741: There might be a mistake here.
Context: ...c| false | Include table of contents | |--markdown-toc-depth` | 3 | TOC depth...
(QB_NEW_EN)
[grammar] ~742-~742: There might be a mistake here.
Context: ...--markdown-toc-depth | 3 | TOC depth | | --markdown-code-style | "fenced" | C...
(QB_NEW_EN)
[grammar] ~747-~747: There might be a mistake here.
Context: ...son | Format | Use Case | Pros | Cons | |--------|----------|------|------| | **...
(QB_NEW_EN)
[grammar] ~748-~748: There might be a mistake here.
Context: ...ns | |--------|----------|------|------| | JSON | API integration, programmat...
(QB_NEW_EN)
[grammar] ~749-~749: There might be a mistake here.
Context: ...readable | Verbose, not human-readable | | CSV | Spreadsheet analysis, data s...
(QB_NEW_EN)
[grammar] ~750-~750: There might be a mistake here.
Context: ...orted | Limited structure, no metadata | | XML | Enterprise integration, comp...
(QB_NEW_EN)
[grammar] ~751-~751: There might be a mistake here.
Context: ... extensible | Verbose, complex parsing | | HTML | Web reporting, human-readab...
(QB_NEW_EN)
[grammar] ~752-~752: There might be a mistake here.
Context: ...ng, interactive | Not machine-readable | | Markdown | Documentation, GitHub i...
(QB_NEW_EN)
[grammar] ~753-~753: There might be a mistake here.
Context: ... control friendly | Limited formatting | | Text | Simple reporting, logs | Si...
(QB_NEW_EN)
docs/runtime-management.md
[grammar] ~9-~9: There might be a mistake here.
Context: ...Fully async process lifecycle management - ProcessWatchdog: Automated monitoring ...
(QB_NEW_EN)
[grammar] ~10-~10: There might be a mistake here.
Context: ...ing and termination of hanging processes - AsyncFuzzExecutor: Controlled concurre...
(QB_NEW_EN)
[grammar] ~19-~19: There might be a mistake here.
Context: ..._exec` for non-blocking process spawning - Process Registration: Automatically re...
(QB_NEW_EN)
[grammar] ~20-~20: There might be a mistake here.
Context: ...ocesses with the watchdog for monitoring - Signal Handling: Supports graceful ter...
(QB_NEW_EN)
[grammar] ~21-~21: There might be a mistake here.
Context: ...l (SIGKILL) with process-group signaling - Status Tracking: Maintains comprehensi...
(QB_NEW_EN)
[grammar] ~22-~22: There might be a mistake here.
Context: ...ng start time, status, and configuration - Cleanup Management: Automatic cleanup ...
(QB_NEW_EN)
[grammar] ~145-~145: There might be a mistake here.
Context: ...acefully or forcefully - Returns True if process was stopped successfully - `as...
(QB_NEW_EN)
[grammar] ~153-~153: There might be a mistake here.
Context: ...for a specific process - Returns None if process is not managed - `async list_p...
(QB_NEW_EN)
[grammar] ~181-~181: There might be a mistake here.
Context: ...activity through callbacks or timestamps - Hang Detection: Identifies processes t...
(QB_NEW_EN)
[grammar] ~182-~182: There might be a mistake here.
Context: ...en active for configured timeout periods - Automatic Termination: Can automatical...
(QB_NEW_EN)
[grammar] ~183-~183: There might be a mistake here.
Context: ...y kill hanging processes based on policy - Configurable Thresholds: Separate thre...
(QB_NEW_EN)
[grammar] ~304-~304: There might be a mistake here.
Context: ...monitoring - Activity callback should return timestamp of last activity - `async un...
(QB_NEW_EN)
[grammar] ~304-~304: There might be a mistake here.
Context: ...tivity callback should return timestamp of last activity - `async unregister_proc...
(QB_NEW_EN)
[grammar] ~324-~324: There might be a mistake here.
Context: ...semaphore to limit concurrent operations - Task Tracking: Maintains set of runnin...
(QB_NEW_EN)
[grammar] ~325-~325: There might be a mistake here.
Context: ...set of running tasks for proper shutdown - Batch Operations: Execute multiple ope...
(QB_NEW_EN)
[grammar] ~330-~330: There might be a mistake here.
Context: ...rable timeouts for individual operations - Retry Logic: Exponential backoff retry...
(QB_NEW_EN)
[grammar] ~331-~331: There might be a mistake here.
Context: ...ff retry mechanism for failed operations - Exception Collection: Collects and cat...
(QB_NEW_EN)
[grammar] ~466-~466: There might be a mistake here.
Context: ... complete - Cancels outstanding tasks if timeout is exceeded ## Integration Exa...
(QB_NEW_EN)
[grammar] ~574-~574: There might be a mistake here.
Context: ...tart or immediately exits Solutions: - Check command path and arguments - Verif...
(QB_NEW_EN)
[grammar] ~583-~583: There might be a mistake here.
Context: ... hang and doesn't respond Solutions: - Check watchdog configuration and timeout...
(QB_NEW_EN)
[grammar] ~591-~591: There might be a mistake here.
Context: ... High CPU or memory usage during fuzzing Solutions: - Reduce `max_concurrency...
(QB_NEW_EN)
[grammar] ~592-~592: There might be a mistake here.
Context: ...mory usage during fuzzing Solutions: - Reduce max_concurrency in AsyncFuzzExe...
(QB_NEW_EN)
[grammar] ~600-~600: There might be a mistake here.
Context: ...ms**: Processes not terminating properly Solutions: - Check if process handle...
(QB_NEW_EN)
[grammar] ~601-~601: There might be a mistake here.
Context: ... not terminating properly Solutions: - Check if process handles SIGTERM correct...
(QB_NEW_EN)
docs/configuration.md
[grammar] ~9-~9: There might be a mistake here.
Context: ...nd-line arguments** (highest precedence) 2. Configuration files (YAML/TOML) 3. **E...
(QB_NEW_EN)
[grammar] ~10-~10: There might be a mistake here.
Context: ...) 2. Configuration files (YAML/TOML) 3. Environment variables (lowest preceden...
(QB_NEW_EN)
[grammar] ~286-~286: There might be a mistake here.
Context: ...on | Variable | Default | Description | |----------|---------|-------------| | `...
(QB_NEW_EN)
[grammar] ~287-~287: There might be a mistake here.
Context: ...n | |----------|---------|-------------| | MCP_FUZZER_TIMEOUT | 30.0 | Default ...
(QB_NEW_EN)
[grammar] ~288-~288: There might be a mistake here.
Context: ...0 | Default timeout for all operations | | MCP_FUZZER_LOG_LEVEL | INFO | Defaul...
(QB_NEW_EN)
[grammar] ~289-~289: There might be a mistake here.
Context: ..._LOG_LEVEL| INFO | Default log level | |MCP_FUZZER_VERBOSE` | false | Enable ...
(QB_NEW_EN)
[grammar] ~290-~290: There might be a mistake here.
Context: ...BOSE| false | Enable verbose logging | |MCP_FUZZER_OUTPUT_DIR` | reports | De...
(QB_NEW_EN)
[grammar] ~295-~295: There might be a mistake here.
Context: ...on | Variable | Default | Description | |----------|---------|-------------| | `...
(QB_NEW_EN)
[grammar] ~296-~296: There might be a mistake here.
Context: ...n | |----------|---------|-------------| | MCP_FUZZER_SAFETY_ENABLED | false | ...
(QB_NEW_EN)
[grammar] ~297-~297: There might be a mistake here.
Context: ...alse | Enable safety system by default | | MCP_FUZZER_FS_ROOT | ~/.mcp_fuzzer |...
(QB_NEW_EN)
[grammar] ~298-~298: There might be a mistake here.
Context: ...r | Default filesystem root for safety | | MCP_FUZZER_AUTO_KILL | true | Auto-k...
(QB_NEW_EN)
[grammar] ~299-~299: There might be a mistake here.
Context: ...| true | Auto-kill hanging processes | |MCP_FUZZER_RETRY_WITH_SAFETY` | false...
(QB_NEW_EN)
[grammar] ~304-~304: There might be a mistake here.
Context: ...on | Variable | Default | Description | |----------|---------|-------------| | `...
(QB_NEW_EN)
[grammar] ~305-~305: There might be a mistake here.
Context: ...n | |----------|---------|-------------| | MCP_FUZZER_MAX_CONCURRENCY | 5 | Max...
(QB_NEW_EN)
[grammar] ~306-~306: There might be a mistake here.
Context: ...Y| 5 | Maximum concurrent operations | |MCP_FUZZER_RETRY_COUNT` | 1 | Number ...
(QB_NEW_EN)
[grammar] ~307-~307: There might be a mistake here.
Context: ...umber of retries for failed operations | | MCP_FUZZER_RETRY_DELAY | 1.0 | Delay...
(QB_NEW_EN)
[grammar] ~312-~312: There might be a mistake here.
Context: ...on | Variable | Default | Description | |----------|---------|-------------| | `...
(QB_NEW_EN)
[grammar] ~313-~313: There might be a mistake here.
Context: ...n | |----------|---------|-------------| | MCP_FUZZER_HTTP_TIMEOUT | 30.0 | HTT...
(QB_NEW_EN)
[grammar] ~314-~314: There might be a mistake here.
Context: ...MEOUT| 30.0 | HTTP transport timeout | |MCP_FUZZER_SSE_TIMEOUT` | 30.0 | SSE ...
(QB_NEW_EN)
[grammar] ~315-~315: There might be a mistake here.
Context: ...IMEOUT| 30.0 | SSE transport timeout | |MCP_FUZZER_STDIO_TIMEOUT` | 30.0 | St...
(QB_NEW_EN)
[grammar] ~320-~320: There might be a mistake here.
Context: ...nt Variables | Variable | Description | |----------|-------------| | `MCP_API_KE...
(QB_NEW_EN)
[grammar] ~321-~321: There might be a mistake here.
Context: ...Description | |----------|-------------| | MCP_API_KEY | API key for authentica...
(QB_NEW_EN)
[grammar] ~322-~322: There might be a mistake here.
Context: ..._API_KEY| API key for authentication | |MCP_HEADER_NAME` | Header name for AP...
(QB_NEW_EN)
[grammar] ~323-~323: There might be a mistake here.
Context: ...e for API key (default: Authorization) | | MCP_USERNAME | Username for basic au...
(QB_NEW_EN)
[grammar] ~324-~324: There might be a mistake here.
Context: ...E| Username for basic authentication | |MCP_PASSWORD` | Password for basic au...
(QB_NEW_EN)
[grammar] ~325-~325: There might be a mistake here.
Context: ...D| Password for basic authentication | |MCP_OAUTH_TOKEN` | OAuth token for au...
(QB_NEW_EN)
[grammar] ~330-~330: There might be a mistake here.
Context: ...on | Variable | Default | Description | |----------|---------|-------------| | `...
(QB_NEW_EN)
[grammar] ~331-~331: There might be a mistake here.
Context: ...n | |----------|---------|-------------| | MCP_FUZZER_CHECK_INTERVAL | 1.0 | Ho...
(QB_NEW_EN)
[grammar] ~332-~332: There might be a mistake here.
Context: ...| 1.0 | How often to check processes | |MCP_FUZZER_PROCESS_TIMEOUT` | 30.0 | ...
(QB_NEW_EN)
[grammar] ~333-~333: There might be a mistake here.
Context: ...e before process is considered hanging | | MCP_FUZZER_EXTRA_BUFFER | 5.0 | Extr...
(QB_NEW_EN)
[grammar] ~334-~334: There might be a mistake here.
Context: ...R| 5.0 | Extra time before auto-kill | |MCP_FUZZER_MAX_HANG_TIME` | 60.0 | Ma...
(QB_NEW_EN)
docs/architecture.md
[grammar] ~265-~265: There might be a mistake here.
Context: ... send signals, await exit, collect stats - watchdog.py: ProcessWatchdog for monitoring registe...
(QB_NEW_EN)
[grammar] ~266-~266: There might be a mistake here.
Context: ...ity and terminating them based on policy - executor.py: AsyncFuzzExecutor providing concurrenc...
(QB_NEW_EN)
[grammar] ~395-~395: There might be a mistake here.
Context: ...ollowing capabilities: Core Features: - Async Process Creation: Uses `asyncio....
(QB_NEW_EN)
[grammar] ~402-~402: There might be a mistake here.
Context: ...ent resource leaks Process Lifecycle: 1. Start: Process is created with asyncio...
(QB_NEW_EN)
[grammar] ~408-~408: There might be a mistake here.
Context: ... from tracking Configuration Options: - command: List of command and arguments - cwd:...
(QB_NEW_EN)
[grammar] ~409-~409: There might be a mistake here.
Context: ...command: List of command and arguments - cwd: Working directory for the process - `e...
(QB_NEW_EN)
[grammar] ~410-~410: There might be a mistake here.
Context: ...cwd: Working directory for the process - env: Environment variables (merged with cur...
(QB_NEW_EN)
[grammar] ~411-~411: There might be a mistake here.
Context: ...iables (merged with current environment) - timeout: Default timeout for process operations...
(QB_NEW_EN)
[grammar] ~412-~412: There might be a mistake here.
Context: ...: Default timeout for process operations - auto_kill: Whether to automatically kill hanging ...
(QB_NEW_EN)
[grammar] ~413-~413: There might be a mistake here.
Context: ... to automatically kill hanging processes - name: Human-readable name for logging and id...
(QB_NEW_EN)
[grammar] ~414-~414: There might be a mistake here.
Context: ...able name for logging and identification - activity_callback: Optional callback to report process ac...
(QB_NEW_EN)
[grammar] ~421-~421: There might be a mistake here.
Context: ...nging processes: Monitoring Features: - Activity Tracking: Monitors process ac...
(QB_NEW_EN)
[grammar] ~427-~427: There might be a mistake here.
Context: ...and force kill Configuration Options: - check_interval: How often to check processes (default:...
(QB_NEW_EN)
[grammar] ~428-~428: There might be a mistake here.
Context: ...o check processes (default: 1.0 seconds) - process_timeout: Time before process is considered hang...
(QB_NEW_EN)
[grammar] ~429-~429: There might be a mistake here.
Context: ...nsidered hanging (default: 30.0 seconds) - extra_buffer: Extra time before auto-kill (default: ...
(QB_NEW_EN)
[grammar] ~430-~430: There might be a mistake here.
Context: ... before auto-kill (default: 5.0 seconds) - max_hang_time: Maximum time before force kill (defaul...
(QB_NEW_EN)
[grammar] ~431-~431: There might be a mistake here.
Context: ...efore force kill (default: 60.0 seconds) - auto_kill: Whether to automatically kill hanging ...
(QB_NEW_EN)
[grammar] ~441-~441: There might be a mistake here.
Context: ...zing operations: Concurrency Control: - Bounded Concurrency: Uses semaphore to...
(QB_NEW_EN)
[grammar] ~446-~446: There might be a mistake here.
Context: ...ith result collection Error Handling: - Timeout Management: Configurable timeo...
(QB_NEW_EN)
[grammar] ~447-~447: There might be a mistake here.
Context: ...rable timeouts for individual operations - Retry Logic: Exponential backoff retry...
(QB_NEW_EN)
[grammar] ~448-~448: There might be a mistake here.
Context: ...ff retry mechanism for failed operations - Exception Collection: Collects and cat...
(QB_NEW_EN)
[grammar] ~451-~451: There might be a mistake here.
Context: ...tch operations Configuration Options: - max_concurrency: Maximum number of concurrent operation...
(QB_NEW_EN)
[grammar] ~452-~452: There might be a mistake here.
Context: ...er of concurrent operations (default: 5) - timeout: Default timeout for operations (defaul...
(QB_NEW_EN)
[grammar] ~453-~453: There might be a mistake here.
Context: ...t for operations (default: 30.0 seconds) - retry_count: Number of retries for failed operation...
(QB_NEW_EN)
[grammar] ~454-~454: There might be a mistake here.
Context: ...tries for failed operations (default: 1) - retry_delay: Delay between retries (default: 1.0 se...
(QB_NEW_EN)
[grammar] ~457-~457: There might be a mistake here.
Context: ...default: 1.0 seconds) Usage Patterns: - Single Operations: Execute individual ...
(QB_NEW_EN)
🔇 Additional comments (7)
docs/reference.md (1)
40-45: Verify streamablehttp handshake implementation
- I see
mcp-session-id/mcp-protocol-versionheaders and 307/308 support in HTTPTransport, but no explicit “initialize” RPC before the first payload in StreamableHTTPTransport. Please confirm that the MCP initialize handshake is sent automatically on first use as the docs state.docs/configuration.md (1)
3-12: Docs coverage looks solidConfiguration precedence, YAML/TOML, profiles, and validation are well covered. Nice.
docs/architecture.md (1)
259-279: Architecture update meets objectivesRuntime Management layer and AsyncFuzzExecutor are clearly introduced and aligned with the rest of the docs.
docs/examples.md (2)
461-926: Ignore duplicate warning: the ‘## Runtime Management Examples’ header appears only once in docs/examples.md (line 461); no deduplication needed.Likely an incorrect or invalid review comment.
621-656: execute_batch returns “results” and “errors” as documented.
No updates to the example are needed.docs/index.md (1)
111-111: Good clarification on supported config formats.Explicit “(YAML/TOML)” improves discoverability.
docs/getting-started.md (1)
77-78: Nice addition: call out Safety Data explicitly.Helps users understand when extra output appears.
docs/configuration.md
Outdated
| [safety] | ||
| enabled = true | ||
| fs_root = "~/.mcp_fuzzer" | ||
| auto_kill = true | ||
| retry_with_safety_on_interrupt = false | ||
|
|
There was a problem hiding this comment.
🛠️ Refactor suggestion
Same fix in TOML: move auto_kill out of [safety]
[safety]
enabled = true
fs_root = "~/.mcp_fuzzer"
-auto_kill = true
retry_with_safety_on_interrupt = falseCommittable suggestion skipped: line range outside the PR's diff.
🤖 Prompt for AI Agents
In docs/configuration.md around lines 143 to 148 the example TOML places
auto_kill inside the [safety] table but the setting is meant to be a top-level
key; remove the auto_kill line from the [safety] block and add it at the root
level (before or after the [safety] table) with the same value, keeping
surrounding spacing/format consistent and updating any adjacent prose if it
references the old location.
| ```bash | ||
| # Validate configuration file | ||
| mcp-fuzzer --validate-config config.yaml | ||
| # Output: "Configuration file is valid" | ||
|
|
||
| # Validate with errors | ||
| mcp-fuzzer --validate-config invalid-config.yaml | ||
| # Output: "Configuration validation failed: | ||
| # - Line 5: 'timeout' must be a number, got 'invalid' | ||
| # - Line 10: 'protocol' must be one of ['http', 'sse', 'stdio', 'streamablehttp']" | ||
| ``` | ||
|
|
||
| ### Common Error Scenarios | ||
| ### Environment Variable Validation | ||
|
|
||
| - **File Not Found**: When the specified configuration file doesn't exist | ||
| - **Invalid Format**: When the file extension is not supported (only .yml and .yaml are supported) | ||
| - **Parsing Error**: When the YAML content cannot be parsed | ||
| - **Permission Error**: When there are permission issues reading the file | ||
| ```bash | ||
| # Check environment variables | ||
| mcp-fuzzer --check-env | ||
| # Output: "Environment variables check: | ||
| # ✓ MCP_FUZZER_TIMEOUT=30.0 | ||
| # ✗ MCP_FUZZER_LOG_LEVEL=INVALID (must be one of: DEBUG, INFO, WARNING, ERROR, CRITICAL) | ||
| # ✓ MCP_FUZZER_SAFETY_ENABLED=true" | ||
| ``` |
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
Verify config validation/check-env commands exist
Docs introduce --validate-config and --check-env. Confirm they’re implemented.
🏁 Script executed:
#!/bin/bash
rg -nC3 --type=py -e '--validate-config' -e '--check-env' -g 'cli/**'
rg -nC3 --type=py -e 'validate_config' -e 'check_env' -g 'cli/**'Length of output: 429
🏁 Script executed:
#!/bin/bash
# Search entire repo for config validation and environment check commands without filters
rg -nC2 'validate_config'
rg -nC2 'check_env'
rg -nC2 -- '--validate-config'
rg -nC2 -- '--check-env'Length of output: 1138
Ensure CLI flags --validate-config and --check-env are implemented
The flags documented in docs/configuration.md (around lines 428–449) have no corresponding code references or handlers—implement these commands in the CLI or remove their documentation.
🤖 Prompt for AI Agents
In docs/configuration.md around lines 428–449, the docs reference CLI flags
--validate-config and --check-env that are not implemented; implement these CLI
commands in the main CLI entrypoint (or remove the docs). Add two handlers: one
for --validate-config <file> that loads and validates the YAML/JSON config
against the existing config schema, prints either "Configuration file is valid"
or a detailed validation error list with line/field info and non-zero exit code
on failure; and one for --check-env that validates required environment
variables (types/allowed values), prints a summary with ✓/✗ markers and
explanations for failures, and exits non-zero if any checks fail. Wire both
flags into the argument parser/help output, reuse existing validation utilities,
add unit tests for success/failure cases, and ensure logging/exit codes follow
the CLI conventions used elsewhere in the project.
| ## Additional Export Formats | ||
|
|
||
| The MCP Server Fuzzer supports multiple export formats for reports: | ||
|
|
||
| ### CSV Export | ||
|
|
||
| Export fuzzing results to CSV format for analysis in spreadsheet applications: | ||
|
|
||
| ```bash | ||
| # Export to CSV format | ||
| mcp-fuzzer --mode tools --protocol http --endpoint http://localhost:8000 --runs 20 --export-csv results.csv | ||
|
|
||
| # Export with custom CSV configuration | ||
| mcp-fuzzer --mode tools --protocol http --endpoint http://localhost:8000 --runs 20 \ | ||
| --export-csv results.csv \ | ||
| --csv-delimiter "," \ | ||
| --csv-quote-char "\"" | ||
| ``` | ||
|
|
||
| CSV output includes: | ||
| - Tool name | ||
| - Run number | ||
| - Success status | ||
| - Response time | ||
| - Exception message (if any) | ||
| - Arguments used | ||
| - Timestamp | ||
|
|
||
| ### XML Export | ||
|
|
||
| Export fuzzing results to XML format for integration with XML-based tools: | ||
|
|
||
| ```bash | ||
| # Export to XML format | ||
| mcp-fuzzer --mode tools --protocol http --endpoint http://localhost:8000 --runs 20 --export-xml results.xml | ||
|
|
||
| # Export with custom XML configuration | ||
| mcp-fuzzer --mode tools --protocol http --endpoint http://localhost:8000 --runs 20 \ | ||
| --export-xml results.xml \ | ||
| --xml-indent 2 \ | ||
| --xml-encoding "utf-8" | ||
| ``` | ||
|
|
||
| ### HTML Export | ||
|
|
||
| Export results to HTML format for web-based reporting: | ||
|
|
||
| ```bash | ||
| # Export to HTML format | ||
| mcp-fuzzer --mode tools --protocol http --endpoint http://localhost:8000 --runs 20 --export-html results.html | ||
|
|
||
| # Export with custom HTML template | ||
| mcp-fuzzer --mode tools --protocol http --endpoint http://localhost:8000 --runs 20 \ | ||
| --export-html results.html \ | ||
| --html-template custom_template.html \ | ||
| --html-title "Fuzzing Results Report" | ||
| ``` | ||
|
|
||
| ### Markdown Export | ||
|
|
||
| Export results to Markdown format for documentation: | ||
|
|
||
| ```bash | ||
| # Export to Markdown format | ||
| mcp-fuzzer --mode tools --protocol http --endpoint http://localhost:8000 --runs 20 --export-markdown results.md | ||
|
|
||
| # Export with custom Markdown configuration | ||
| mcp-fuzzer --mode tools --protocol http --endpoint http://localhost:8000 --runs 20 \ | ||
| --export-markdown results.md \ | ||
| --markdown-style github \ | ||
| --markdown-toc true | ||
| ``` | ||
|
|
There was a problem hiding this comment.
💡 Verification agent
❓ Verification inconclusive
Confirm new export flags are wired in CLI
The docs introduce --export-csv, --export-xml, --export-html, --export-markdown and their options. Please verify these are parsed and handled.
🏁 Script executed:
#!/bin/bash
# Look for CLI args and handlers
rg -nC3 --type=py -e '--export-(csv|xml|html|markdown)' -e '--csv-|--xml-|--html-|--markdown-' cli args.py runner.py -g 'cli/**'
rg -nC3 --type=py -e 'export_(csv|xml|html|markdown)' -e 'write_csv|write_xml|write_html|write_markdown' -g 'reports/**'Length of output: 544
🏁 Script executed:
#!/bin/bash
# Search for export flags in CLI argument definitions
rg -n -C3 --type=py "add_argument.*--export-(csv|xml|html|markdown)"
# Search for corresponding handler functions
rg -n -C3 --type=py "def write_(csv|xml|html|markdown)"Length of output: 128
Implement missing CLI export flags and handlers
No definitions or handlers for --export-csv, --export-xml, --export-html, or --export-markdown were found in the CLI argument parsing or in report-writing code; add the corresponding add_argument (or @click.option) entries and implement the write_csv, write_xml, write_html, and write_markdown functions.
🧰 Tools
🪛 LanguageTool
[grammar] ~653-~653: There might be a mistake here.
Context: ...uote-char """ ``` CSV output includes: - Tool name - Run number - Success status ...
(QB_NEW_EN)
[grammar] ~654-~654: There might be a mistake here.
Context: ..."" ``` CSV output includes: - Tool name - Run number - Success status - Response t...
(QB_NEW_EN)
[grammar] ~655-~655: There might be a mistake here.
Context: ...utput includes: - Tool name - Run number - Success status - Response time - Excepti...
(QB_NEW_EN)
[grammar] ~656-~656: There might be a mistake here.
Context: ... Tool name - Run number - Success status - Response time - Exception message (if an...
(QB_NEW_EN)
[grammar] ~657-~657: There might be a mistake here.
Context: ... number - Success status - Response time - Exception message (if any) - Arguments u...
(QB_NEW_EN)
[grammar] ~658-~658: There might be a mistake here.
Context: ...sponse time - Exception message (if any) - Arguments used - Timestamp ### XML Expo...
(QB_NEW_EN)
[grammar] ~659-~659: There might be a mistake here.
Context: ...eption message (if any) - Arguments used - Timestamp ### XML Export Export fuzzin...
(QB_NEW_EN)
🤖 Prompt for AI Agents
In docs/reference.md around lines 634 to 706, the docs reference CLI flags
(--export-csv, --export-xml, --export-html, --export-markdown) and related
options but the CLI and report-writer lack these flags and handlers; add CLI
argument definitions (argparse.add_argument or click.option) for each export
flag and their configuration options (csv-delimiter, csv-quote-char, xml-indent,
xml-encoding, html-template, html-title, markdown-style, markdown-toc), wire
them into the main run function, and implement four exporter functions
write_csv, write_xml, write_html, write_markdown that accept the run results and
options and serialize the reports accordingly (CSV: delimiter, quote handling;
XML: encoding and indentation; HTML: template and title insertion; Markdown:
style & TOC generation), ensure they are unit-tested and invoked when the
corresponding flag is present, and add error handling and logging for file I/O
failures.
Agent-Hellboy
force-pushed
the
fix-54
branch
from
ce86364 to
dde974d
Compare
Documentation PreviewYour documentation changes have been built successfully! Preview URL: https://Agent-Hellboy.github.io/mcp-server-fuzzer/ The documentation will be automatically deployed to the main site when this PR is merged. This comment was automatically generated by the documentation workflow. |
Documentation PreviewYour documentation changes have been built successfully! Preview URL: https://Agent-Hellboy.github.io/mcp-server-fuzzer/ The documentation will be automatically deployed to the main site when this PR is merged. This comment was automatically generated by the documentation workflow. |
Documentation PreviewYour documentation changes have been built successfully! Preview URL: https://Agent-Hellboy.github.io/mcp-server-fuzzer/ The documentation will be automatically deployed to the main site when this PR is merged. This comment was automatically generated by the documentation workflow. |
fixes #54
Summary by CodeRabbit