änderungen von vorherigen Codexnutzung: sollte den Agent lassen die Aufgaben zu erzeugen

This commit is contained in:
Eugenius Naumenko
2026-09-22 16:14:41 +00:00
parent a7d8f72129
commit 74a4caa1c6
12 changed files with 457 additions and 239 deletions
+82 -86
View File
@@ -11,32 +11,27 @@ It is designed for the current Go backend adapter.
- Internal Flowise URL for the API container: `http://flowise:3000`
- Verified Flowise version on the existing instance: `3.0.11`
- The Go backend calls Flowise with `POST /api/v1/prediction/{FLOWISE_CHATFLOW_ID}`.
- The request body includes `question` as a JSON string and `form` as the same structured object.
- The request body includes `question` as one JSON string and `overrideConfig.sessionId`.
Build the first V2 flow with Start Node input type `Chat Input`; it is the simplest path because
all current tasks arrive as one JSON context. V2 `Form Input` is also possible now, but only if the
form fields match the backend payload fields for the task you want to handle.
the backend sends one JSON context as the chat input. V2 `Form Input` can come later if you split
the flow into typed task-specific flows.
The backend parses Flowise responses from `text`, `json`, or a raw JSON response. A plain JSON text
reply is still the easiest response format to debug in Flowise and in the app.
## Current backend tasks
The current backend can call the agent for these tasks:
The current backend sends only one task to Flowise:
1. `practice_feedback`
- Sent after Azure/mock pronunciation assessment.
- Context includes `target_text`, `recognized_text`, `scores`, `weak_words`, and rules.
- The agent must never create or modify pronunciation scores.
2. `start_conversation`
- Starts a short A1 roleplay.
- Context includes `target_language` and `cefr_level`.
3. `continue_conversation`
- Continues a short A1 roleplay.
- Context includes `target_language`, `cefr_level`, and `user_text`.
1. `next_exercise`
- Sent when dashboard or `POST /lessons/next` needs the next speaking exercise.
- Context includes learner profile, recent progress, weak words, target language, CEFR level, and rules.
- The agent creates exactly one A1 exercise. The backend stores it in PostgreSQL for that user.
The backend does not currently call Flowise to choose the next DB exercise. The prompt
`prompts/exercise_generation.md` is useful for a later backend extension, but it is not used now.
Flowise owns the exercise-generation logic. PostgreSQL remains the durable system of record for
users, profiles, generated exercises, attempts, and progress. Pronunciation scoring stays in Azure
Speech/the backend. Feedback and the simple conversation screen are local fallback logic for now.
## Recommended V2 graph
@@ -65,7 +60,8 @@ Start Node:
LLM/Agent Node:
- Input Message: `{{ $flow.input }}`
- Memory: off for now. The backend does not pass a Flowise `sessionId` yet.
- Memory: on or Flowise chat memory enabled if you want per-user continuity.
- The backend sends `overrideConfig.sessionId` in the shape `user_id:target_language:practice`.
- Output: JSON text only.
Direct Reply Node:
@@ -83,55 +79,47 @@ You receive one JSON object as the user message. It contains a field named "task
Global rules:
- Return valid JSON only. No Markdown. No code fences. No commentary outside JSON.
- Use exactly this JSON shape:
- Read the input JSON field "task".
- If task is "next_exercise", use exactly this JSON shape:
{
"exercise_type": "repeat_sentence",
"target_text": "English text the learner should say next",
"teacher_feedback": "Short German feedback",
"tts_text": "Text for speech synthesis",
"ssml": "",
"next_recommendation": "One short next step in German"
"target_language": "en-US",
"cefr_level": "A1",
"reference_text": "English sentence the learner must pronounce",
"display_prompt": "Short German instruction",
"tts_text": "English text for speech synthesis",
"focus_points": ["one or two words or sounds to practise"],
"success_criteria": ["concrete success criterion"],
"reason_for_choice": "Short German reason based on learner progress"
}
- Allowed exercise_type values: "repeat_sentence", "conversation".
- If the task is missing or unknown, still return one safe A1 repeat_sentence exercise in this same shape.
- Allowed exercise_type value for now: "repeat_sentence".
- Keep the learner at CEFR A1 unless the input explicitly says otherwise.
- Explain in German. Target sentences and roleplay turns are in English.
- Use short, everyday A1 English. Prefer 3 to 9 words for target_text when possible.
- Give at most two corrections per turn.
- Use German for display_prompt and reason_for_choice.
- Use English for reference_text and tts_text.
- Use short, everyday A1 English. Prefer 3 to 9 words for reference_text when possible.
- Do not invent pronunciation scores. Scores come only from Azure Speech or the backend mock.
- Do not change score values. Use scores only to decide feedback.
- Prefer repetition, minimal pairs, shadowing, short roleplay, and one clear next step.
- Keep ssml as an empty string unless the backend later asks for SSML.
- Use recent weak_words if present.
- Prefer repetition, minimal pairs, shadowing, and one clear pronunciation focus.
Task rules:
If task is "practice_feedback":
- Use target_text, recognized_text, scores, and weak_words.
- Mention the weakest word or sound if available.
- If overall_score is high, praise briefly and recommend a more natural rhythm.
- If weak_words are present, ask the learner to repeat the original target_text slowly.
- exercise_type must be "repeat_sentence".
- target_text and tts_text should usually equal the original target_text.
If task is "start_conversation":
- Start one simple A1 roleplay.
- Use one short teacher sentence as target_text and tts_text.
- Give a German next_recommendation that tells the learner how to answer.
- exercise_type must be "conversation".
If task is "continue_conversation":
- Read user_text and continue with one short teacher sentence.
- Correct only the most important issue, if any.
- Keep the roleplay moving with a simple A1 question or prompt.
- exercise_type must be "conversation".
If the task is missing or unknown:
- Return a safe repeat_sentence response for A1 practice.
If task is "next_exercise":
- Create one short pronunciation exercise for the learner.
- exercise_type must be "repeat_sentence" for the current MVP.
- reference_text and tts_text should be English and usually identical.
- display_prompt and reason_for_choice should be German.
- Use recent weak_words if present.
- Keep the sentence short, natural, and useful for A1.
- Do not include pronunciation scores.
```
## JSON contract
Use this contract mentally or as a structured-output schema if the UI supports it.
Exercise generation response for `next_exercise`:
```json
{
"type": "object",
@@ -139,31 +127,49 @@ Use this contract mentally or as a structured-output schema if the UI supports i
"properties": {
"exercise_type": {
"type": "string",
"enum": ["repeat_sentence", "conversation"]
"enum": ["repeat_sentence"]
},
"target_text": {
"target_language": {
"type": "string"
},
"teacher_feedback": {
"cefr_level": {
"type": "string"
},
"reference_text": {
"type": "string"
},
"display_prompt": {
"type": "string"
},
"tts_text": {
"type": "string"
},
"ssml": {
"type": "string"
"focus_points": {
"type": "array",
"items": {
"type": "string"
}
},
"next_recommendation": {
"success_criteria": {
"type": "array",
"items": {
"type": "string"
}
},
"reason_for_choice": {
"type": "string"
}
},
"required": [
"exercise_type",
"target_text",
"teacher_feedback",
"target_language",
"cefr_level",
"reference_text",
"display_prompt",
"tts_text",
"ssml",
"next_recommendation"
"focus_points",
"success_criteria",
"reason_for_choice"
]
}
```
@@ -172,34 +178,25 @@ Use this contract mentally or as a structured-output schema if the UI supports i
Paste these as chat input in Flowise.
Practice feedback:
Next exercise:
```json
{"task":"practice_feedback","target_text":"Can I have a coffee, please?","recognized_text":"Can I have coffee please","scores":{"recognized_text":"Can I have coffee please","overall_score":72,"accuracy_score":70,"fluency_score":78,"completeness_score":82,"prosody_score":69,"word_scores":[{"word":"Can","accuracy":91,"highlight":"good"},{"word":"I","accuracy":88,"highlight":"good"},{"word":"have","accuracy":74,"highlight":"ok"},{"word":"a","accuracy":55,"highlight":"weak"},{"word":"coffee","accuracy":79,"highlight":"ok"},{"word":"please","accuracy":68,"highlight":"weak"}],"provider":"mock"},"weak_words":[{"word":"a","accuracy":55,"highlight":"weak"},{"word":"please","accuracy":68,"highlight":"weak"}],"rules":"Answer in German. A1 learner. Max two corrections. Return valid JSON only."}
```
Start conversation:
```json
{"task":"start_conversation","target_language":"en-US","cefr_level":"A1","rules":"Create a simple A1 roleplay. Return valid JSON only."}
```
Continue conversation:
```json
{"task":"continue_conversation","target_language":"en-US","cefr_level":"A1","user_text":"My name Anna","rules":"Continue a simple A1 conversation. Return valid JSON only."}
{"task":"next_exercise","user_id":"usr_demo","target_language":"en-US","cefr_level":"A1","ui_language":"de-DE","profile":{"target_language":"en-US","cefr_level":"A1","daily_goal_minutes":10},"progress":{"total_attempts":2,"average_score":73,"weak_words":[{"word":"please","accuracy":61,"highlight":"weak"}],"recent_attempts":[]},"recent_weak_words":[{"word":"please","accuracy":61,"highlight":"weak"}],"preferred_type":"repeat_sentence","rules":"Generate one A1 English pronunciation exercise. Return valid JSON only with fields: exercise_type, target_language, cefr_level, reference_text, display_prompt, tts_text, focus_points, success_criteria, reason_for_choice."}
```
Expected output example:
```json
{
"exercise_type": "conversation",
"target_text": "Nice to meet you, Anna.",
"teacher_feedback": "Fast richtig: Sage \"My name is Anna.\"",
"tts_text": "Nice to meet you, Anna.",
"ssml": "",
"next_recommendation": "Antworte kurz: I am from ..."
"exercise_type": "repeat_sentence",
"target_language": "en-US",
"cefr_level": "A1",
"reference_text": "Can I have water, please?",
"display_prompt": "Sprich den Satz langsam und klar nach.",
"tts_text": "Can I have water, please?",
"focus_points": ["please", "water"],
"success_criteria": ["accuracy above 80", "clear final sounds"],
"reason_for_choice": "Die Uebung wiederholt ein schwaches Wort in einem kurzen Alltagssatz."
}
```
@@ -226,7 +223,7 @@ docker compose up --build -d speech-api
## Validation
1. In Flowise, run each test payload and confirm the reply is plain JSON text.
2. In the app, submit a pronunciation attempt and verify that the `teacher` response is not the local fallback.
2. In the app, open the dashboard or click the next lesson button and verify that the returned exercise has `source: "flowise"`.
3. Use the Debug page or `GET /api/debug/flowise` after the API is recreated with `DEBUG_ENABLED=true`.
If `/api/debug/*` returns `404`, the running API was started without debug routes. Recreate the
@@ -234,11 +231,10 @@ If `/api/debug/*` returns `404`, the running API was started without debug route
## Later upgrades
These backend changes would unlock a more capable V2 agent:
Useful later upgrades:
- Send `overrideConfig.sessionId` and/or recent turns for conversation continuity.
- Add typed V2 `Form Input` variants if individual task-specific flows are split out.
- Add a backend task for dynamic exercise generation using `prompts/exercise_generation.md`.
- Add Flowise tools for curriculum lookup, spaced repetition policy, or vector-memory retrieval.
- Add a safe internal admin/debug tool only if the backend passes a scoped service credential.
References: