Agentic task generation

[kohankhaki]

PR Type

Feature

Short Description

This PR adds a new agentic system for task generation. It also introduces a structured prompt/response contract (JSON) to include thoughts and integrates Langfuse for logging LLM outputs and key events.

Tests Added

None

---

This change is 

[afkanpour]

@afkanpour reviewed 7 of 7 files at r1, all commit messages.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @kohankhaki)

---

src/utils/agentic_prompts.py line 209 at r1 (raw file):

Please return your proposal and your thoughts and reasoning in the following format:
{{
  "thought": "Your reasoning and thought process about the kind of tasks you're proposing",

"Thought": "Your reasoning and thought process for designing the tasks and ensuring diversity in content and difficulty of tasks"

Code quote:

"Your reasoning and thought process about the kind of tasks you're proposing"

---

src/utils/agentic_prompts.py line 211 at r1 (raw file):

  "thought": "Your reasoning and thought process about the kind of tasks you're proposing",
  "problems": {{
    "problem_0": "TASK_TEXT_1",

These could be replaced with "PROBLEM_1_DESCRIPTION"

Code quote:

TASK_TEXT_1

---

src/utils/agentic_prompts.py line 242 at r1 (raw file):

    "solution_1": "SOLUTION_TEXT_2",
    ...
  }}

We should give one problem at a time for solving. So I expect the solution json will contain only one solution.

We should add a sentence to the prompt asking for the final numerical solution, so parsing and verification becomes easy.

Code quote:

  "solutions": {{
    "solution_0": "SOLUTION_TEXT_1",
    "solution_1": "SOLUTION_TEXT_2",
    ...
  }}