Add Cerebros Dashboard: real-time search monitor, intensity explorer, and 3D network prototype

[Thunderblok]

Issue Type
feature-request-or-enhancement

Title (suggested)
Add Cerebros Dashboard: real-time search monitor, intensity explorer, and 3D network prototype

Describe the issue
We want to formalize and productionize the new Cerebros Dashboard prototype currently on branch dashboard-polish (file: cerebros_dashboard.py). The prototype provides a Tkinter-based GUI to monitor random search runs (via oracle.csv), visualize metrics in near real time, explore lateral connectivity intensity curves (λ * decay(k)), and (early prototype) animate a 3D dense-layer activation view.

Right now it’s a single monolithic script with inline logic for:

• Project discovery (oracle.csv parsing)
• Metric selection and plotting
• Lateral intensity curve exploration
• Demo project generator (synthetic Ames-like regression mini dataset)
• Optional 3D dense layer activation visualization
• Logo + theming logic (white theme + optional CPU-only mode)

We need to:

1. Move from prototype script to a maintainable module/subpackage (e.g. cerebros/dashboard/).
2. Add automated smoke tests (headless) for core non-GUI logic (metric parsing, intensity curve calc, oracle ingestion).
3. Document usage in README (and optionally docs/).
4. Make a clean PR base branch (e.g. feat/dashboard) for incremental enhancements instead of pushing to main.
5. Add optional dependencies segmentation (matplotlib, pillow) so base install isn’t forced to pull GUI stack.
6. Decide on roadmap items (multi-metric overlay, rolling stats, dark mode toggle, model graph inspection, etc.).

Steps to reproduce the current prototype behavior

1. Checkout branch dashboard-polish (local) containing cerebros_dashboard.py.
2. (Optional) Generate a smoke project: python smoke_run.py --mode cerebros --epochs 2 (creates oracle.csv).
3. Run dashboard CPU-only to avoid GPU PTX compile overhead:
   python cerebros_dashboard.py --cpu-only --project cerebros-smoke-project --refresh 1.0
4. Observe:
   • KPI cards populate (Best/Last/Average/Trials).
   • Metric line updates every refresh interval (default 3s, overridden with --refresh).
   • Change metric via combobox if multiple numeric columns exist (e.g. loss, val_loss).
   • Switch to “Intensity Curve” tab and adjust λ + decay to see bar chart update.
   • Click “Load Best Model Summary” to pull model .keras summary (lazy load).
   • Use “3D Network” tab → “Load From Best Model” → “Toggle Animate” to see prototype activation movement.
5. Optionally click “Load Demo (Ames)” instead of supplying a project to auto-generate a demo_ames_project folder.

Environment questions

• Were you running this on the Kubeflow framework?
  not running in Kubeflow (local shell experiment)

• If running on Kubeflow, which Jupyter notebook template?
  N/A

• If not on Kubeflow, where were you running this?
  Running Cerebros alone in a shell terminal (Linux)

• Data used?
  Synthetic generated data (via smoke_run.py or inline demo generator). For demo mode: artificial regression with small numeric feature set; for “cerebros” smoke mode: randomized features + engineered target (linear + sinusoidal components).

• Code / parameters?
  Prototype invocation example:

  python smoke_run.py --mode cerebros --epochs 2
  python cerebros_dashboard.py --cpu-only --project cerebros-smoke-project --refresh 1.0
  

• Errors observed?
  Without --cpu-only, TensorFlow attempts GPU PTX JIT on unsupported compute capability → long warnings + potential delay. (Mitigated by new --cpu-only flag.) No functional runtime exceptions for standard flows. 3D tab gracefully degrades if model lacks dense layers.

Expected behavior
A modular, documented dashboard package that:

• Starts quickly (lazy imports)
• Allows switching between existing and demo projects
• Clearly indicates which metric is displayed
• Smoothly updates plots without flicker
• Provides optional 3D structural/activation view with safe fallback
• Cleanly exits without orphan threads
• Supports theming (light now; dark later)
• Avoids GPU JIT overhead unless user opts in

Screenshots
(Attach a screenshot of the Overview tab with KPIs, the Intensity Curve tab, logo area, and 3D network tab once captured.)

Execution environment
Linux (distro: e.g. Fedora / Ubuntu; Python 3.10 or 3.13 venv), local shell.

Desktop
OS: Linux
Browser: N/A (Tkinter app)
Version: Python 3.10.x (CI) / 3.13 (local test), TensorFlow 2.20.0, matplotlib ~3.10.x

Additional context

• Dashboard is optional; should not be imported on core library path by default to avoid heavy GUI deps in headless CI.
• Might later allow writing metrics to a lightweight SQLite DB or WebSocket streaming for remote monitoring.
• Future synergy: parse exported graph spec (keras_export.py) and visualize node-degree distribution in a separate tab.

Optional additional items

Issue default title
Add Cerebros Dashboard (real-time metrics + intensity explorer + 3D prototype)

Suggested Labels
feature, enhancement, ui, observability, dashboard, good-first-followup

Proposed initial task breakdown (checklist)

• Create branch feat/dashboard from latest main (or updated upstream main)
• Move cerebros_dashboard.py → cerebros/dashboard/__init__.py + app.py structure
• Add extras_require entry (e.g. dashboard: matplotlib, pillow)
• Add headless tests for: oracle parsing, metric selection logic, intensity curve calculations
• Add README section “Dashboard Usage”
• Add CLI flag --dark-mode (stub; may ship disabled)
• Add metric overlay (loss vs val_loss) toggle
• Add rolling average line (window size configurable)
• Defer 3D advanced features (edge weight coloring) to follow-up issue
• Add GitHub Action (optional) to run headless tests (skip GUI)
• Link issue to PR when opened

Migration / backward compatibility
None; new optional feature. Existing code paths unaffected unless users import it.

Security considerations

• Reads local CSV and model files only.
• No network or shell invocation.
• Provide clear warning in docs not to point at untrusted model directories.

---

Branch naming suggestion
feat/dashboard (or if you want iterative: feat/dashboard-mvp, then feat/dashboard-phase2)

PR strategy

1. Open this issue.
2. Create branch off upstream/main:

   git checkout main
   git pull upstream main
   git checkout -b feat/dashboard
   

3. Move + refactor + tests.
4. Push: git push -u origin feat/dashboard
5. Open PR referencing this issue: “Closes #<issue_number>”