änderungen von vorherigen Codexnutzung: sollte den Agent lassen die Aufgaben zu erzeugen

This commit is contained in:
Eugenius Naumenko
2026-09-22 16:14:41 +00:00
parent a7d8f72129
commit 74a4caa1c6
12 changed files with 457 additions and 239 deletions
+11 -7
View File
@@ -6,7 +6,7 @@ Flowise laeuft in Docker als Service `flowise` und ist vom Backend intern ueber
Fuer Agentflow V2 nutze zuerst die konkrete Anleitung in
[`agentflow-v2-setup.md`](./agentflow-v2-setup.md). Sie passt zur aktuellen
Go-Backend-Anbindung, die Flowise mit `question` und `form` aufruft.
Go-Backend-Anbindung, die Flowise mit `question` und `overrideConfig.sessionId` aufruft.
1. Flowise im Browser oeffnen: http://localhost:3000
2. Einen Chatflow oder Agentflow fuer `A1 English Teacher` erstellen.
@@ -17,17 +17,21 @@ Go-Backend-Anbindung, die Flowise mit `question` und `form` aufruft.
FLOWISE_CHATFLOW_ID=...
```
Das Backend erwartet JSON-Antworten mit:
Fuer die naechste Uebung erwartet das Backend JSON-Antworten mit:
```json
{
"exercise_type": "repeat_sentence",
"target_text": "Hello, my name is Anna.",
"teacher_feedback": "Uebe besonders 'name'.",
"target_language": "en-US",
"cefr_level": "A1",
"reference_text": "Hello, my name is Anna.",
"display_prompt": "Sprich den Satz klar nach.",
"tts_text": "Hello, my name is Anna.",
"ssml": "",
"next_recommendation": "Sprich den Satz noch einmal langsam."
"focus_points": ["name"],
"success_criteria": ["accuracy above 80"],
"reason_for_choice": "Kurzer A1-Satz fuer klare Betonung."
}
```
Wenn keine Chatflow-ID gesetzt ist, nutzt die API einen lokalen Fallback.
Feedback und einfache Konversation laufen vorerst lokal im Backend. Wenn keine Chatflow-ID gesetzt
ist oder Flowise kein valides JSON liefert, nutzt die API Seed-Uebungen als Fallback.
+82 -86
View File
@@ -11,32 +11,27 @@ It is designed for the current Go backend adapter.
- Internal Flowise URL for the API container: `http://flowise:3000`
- Verified Flowise version on the existing instance: `3.0.11`
- The Go backend calls Flowise with `POST /api/v1/prediction/{FLOWISE_CHATFLOW_ID}`.
- The request body includes `question` as a JSON string and `form` as the same structured object.
- The request body includes `question` as one JSON string and `overrideConfig.sessionId`.
Build the first V2 flow with Start Node input type `Chat Input`; it is the simplest path because
all current tasks arrive as one JSON context. V2 `Form Input` is also possible now, but only if the
form fields match the backend payload fields for the task you want to handle.
the backend sends one JSON context as the chat input. V2 `Form Input` can come later if you split
the flow into typed task-specific flows.
The backend parses Flowise responses from `text`, `json`, or a raw JSON response. A plain JSON text
reply is still the easiest response format to debug in Flowise and in the app.
## Current backend tasks
The current backend can call the agent for these tasks:
The current backend sends only one task to Flowise:
1. `practice_feedback`
- Sent after Azure/mock pronunciation assessment.
- Context includes `target_text`, `recognized_text`, `scores`, `weak_words`, and rules.
- The agent must never create or modify pronunciation scores.
2. `start_conversation`
- Starts a short A1 roleplay.
- Context includes `target_language` and `cefr_level`.
3. `continue_conversation`
- Continues a short A1 roleplay.
- Context includes `target_language`, `cefr_level`, and `user_text`.
1. `next_exercise`
- Sent when dashboard or `POST /lessons/next` needs the next speaking exercise.
- Context includes learner profile, recent progress, weak words, target language, CEFR level, and rules.
- The agent creates exactly one A1 exercise. The backend stores it in PostgreSQL for that user.
The backend does not currently call Flowise to choose the next DB exercise. The prompt
`prompts/exercise_generation.md` is useful for a later backend extension, but it is not used now.
Flowise owns the exercise-generation logic. PostgreSQL remains the durable system of record for
users, profiles, generated exercises, attempts, and progress. Pronunciation scoring stays in Azure
Speech/the backend. Feedback and the simple conversation screen are local fallback logic for now.
## Recommended V2 graph
@@ -65,7 +60,8 @@ Start Node:
LLM/Agent Node:
- Input Message: `{{ $flow.input }}`
- Memory: off for now. The backend does not pass a Flowise `sessionId` yet.
- Memory: on or Flowise chat memory enabled if you want per-user continuity.
- The backend sends `overrideConfig.sessionId` in the shape `user_id:target_language:practice`.
- Output: JSON text only.
Direct Reply Node:
@@ -83,55 +79,47 @@ You receive one JSON object as the user message. It contains a field named "task
Global rules:
- Return valid JSON only. No Markdown. No code fences. No commentary outside JSON.
- Use exactly this JSON shape:
- Read the input JSON field "task".
- If task is "next_exercise", use exactly this JSON shape:
{
"exercise_type": "repeat_sentence",
"target_text": "English text the learner should say next",
"teacher_feedback": "Short German feedback",
"tts_text": "Text for speech synthesis",
"ssml": "",
"next_recommendation": "One short next step in German"
"target_language": "en-US",
"cefr_level": "A1",
"reference_text": "English sentence the learner must pronounce",
"display_prompt": "Short German instruction",
"tts_text": "English text for speech synthesis",
"focus_points": ["one or two words or sounds to practise"],
"success_criteria": ["concrete success criterion"],
"reason_for_choice": "Short German reason based on learner progress"
}
- Allowed exercise_type values: "repeat_sentence", "conversation".
- If the task is missing or unknown, still return one safe A1 repeat_sentence exercise in this same shape.
- Allowed exercise_type value for now: "repeat_sentence".
- Keep the learner at CEFR A1 unless the input explicitly says otherwise.
- Explain in German. Target sentences and roleplay turns are in English.
- Use short, everyday A1 English. Prefer 3 to 9 words for target_text when possible.
- Give at most two corrections per turn.
- Use German for display_prompt and reason_for_choice.
- Use English for reference_text and tts_text.
- Use short, everyday A1 English. Prefer 3 to 9 words for reference_text when possible.
- Do not invent pronunciation scores. Scores come only from Azure Speech or the backend mock.
- Do not change score values. Use scores only to decide feedback.
- Prefer repetition, minimal pairs, shadowing, short roleplay, and one clear next step.
- Keep ssml as an empty string unless the backend later asks for SSML.
- Use recent weak_words if present.
- Prefer repetition, minimal pairs, shadowing, and one clear pronunciation focus.
Task rules:
If task is "practice_feedback":
- Use target_text, recognized_text, scores, and weak_words.
- Mention the weakest word or sound if available.
- If overall_score is high, praise briefly and recommend a more natural rhythm.
- If weak_words are present, ask the learner to repeat the original target_text slowly.
- exercise_type must be "repeat_sentence".
- target_text and tts_text should usually equal the original target_text.
If task is "start_conversation":
- Start one simple A1 roleplay.
- Use one short teacher sentence as target_text and tts_text.
- Give a German next_recommendation that tells the learner how to answer.
- exercise_type must be "conversation".
If task is "continue_conversation":
- Read user_text and continue with one short teacher sentence.
- Correct only the most important issue, if any.
- Keep the roleplay moving with a simple A1 question or prompt.
- exercise_type must be "conversation".
If the task is missing or unknown:
- Return a safe repeat_sentence response for A1 practice.
If task is "next_exercise":
- Create one short pronunciation exercise for the learner.
- exercise_type must be "repeat_sentence" for the current MVP.
- reference_text and tts_text should be English and usually identical.
- display_prompt and reason_for_choice should be German.
- Use recent weak_words if present.
- Keep the sentence short, natural, and useful for A1.
- Do not include pronunciation scores.
```
## JSON contract
Use this contract mentally or as a structured-output schema if the UI supports it.
Exercise generation response for `next_exercise`:
```json
{
"type": "object",
@@ -139,31 +127,49 @@ Use this contract mentally or as a structured-output schema if the UI supports i
"properties": {
"exercise_type": {
"type": "string",
"enum": ["repeat_sentence", "conversation"]
"enum": ["repeat_sentence"]
},
"target_text": {
"target_language": {
"type": "string"
},
"teacher_feedback": {
"cefr_level": {
"type": "string"
},
"reference_text": {
"type": "string"
},
"display_prompt": {
"type": "string"
},
"tts_text": {
"type": "string"
},
"ssml": {
"type": "string"
"focus_points": {
"type": "array",
"items": {
"type": "string"
}
},
"next_recommendation": {
"success_criteria": {
"type": "array",
"items": {
"type": "string"
}
},
"reason_for_choice": {
"type": "string"
}
},
"required": [
"exercise_type",
"target_text",
"teacher_feedback",
"target_language",
"cefr_level",
"reference_text",
"display_prompt",
"tts_text",
"ssml",
"next_recommendation"
"focus_points",
"success_criteria",
"reason_for_choice"
]
}
```
@@ -172,34 +178,25 @@ Use this contract mentally or as a structured-output schema if the UI supports i
Paste these as chat input in Flowise.
Practice feedback:
Next exercise:
```json
{"task":"practice_feedback","target_text":"Can I have a coffee, please?","recognized_text":"Can I have coffee please","scores":{"recognized_text":"Can I have coffee please","overall_score":72,"accuracy_score":70,"fluency_score":78,"completeness_score":82,"prosody_score":69,"word_scores":[{"word":"Can","accuracy":91,"highlight":"good"},{"word":"I","accuracy":88,"highlight":"good"},{"word":"have","accuracy":74,"highlight":"ok"},{"word":"a","accuracy":55,"highlight":"weak"},{"word":"coffee","accuracy":79,"highlight":"ok"},{"word":"please","accuracy":68,"highlight":"weak"}],"provider":"mock"},"weak_words":[{"word":"a","accuracy":55,"highlight":"weak"},{"word":"please","accuracy":68,"highlight":"weak"}],"rules":"Answer in German. A1 learner. Max two corrections. Return valid JSON only."}
```
Start conversation:
```json
{"task":"start_conversation","target_language":"en-US","cefr_level":"A1","rules":"Create a simple A1 roleplay. Return valid JSON only."}
```
Continue conversation:
```json
{"task":"continue_conversation","target_language":"en-US","cefr_level":"A1","user_text":"My name Anna","rules":"Continue a simple A1 conversation. Return valid JSON only."}
{"task":"next_exercise","user_id":"usr_demo","target_language":"en-US","cefr_level":"A1","ui_language":"de-DE","profile":{"target_language":"en-US","cefr_level":"A1","daily_goal_minutes":10},"progress":{"total_attempts":2,"average_score":73,"weak_words":[{"word":"please","accuracy":61,"highlight":"weak"}],"recent_attempts":[]},"recent_weak_words":[{"word":"please","accuracy":61,"highlight":"weak"}],"preferred_type":"repeat_sentence","rules":"Generate one A1 English pronunciation exercise. Return valid JSON only with fields: exercise_type, target_language, cefr_level, reference_text, display_prompt, tts_text, focus_points, success_criteria, reason_for_choice."}
```
Expected output example:
```json
{
"exercise_type": "conversation",
"target_text": "Nice to meet you, Anna.",
"teacher_feedback": "Fast richtig: Sage \"My name is Anna.\"",
"tts_text": "Nice to meet you, Anna.",
"ssml": "",
"next_recommendation": "Antworte kurz: I am from ..."
"exercise_type": "repeat_sentence",
"target_language": "en-US",
"cefr_level": "A1",
"reference_text": "Can I have water, please?",
"display_prompt": "Sprich den Satz langsam und klar nach.",
"tts_text": "Can I have water, please?",
"focus_points": ["please", "water"],
"success_criteria": ["accuracy above 80", "clear final sounds"],
"reason_for_choice": "Die Uebung wiederholt ein schwaches Wort in einem kurzen Alltagssatz."
}
```
@@ -226,7 +223,7 @@ docker compose up --build -d speech-api
## Validation
1. In Flowise, run each test payload and confirm the reply is plain JSON text.
2. In the app, submit a pronunciation attempt and verify that the `teacher` response is not the local fallback.
2. In the app, open the dashboard or click the next lesson button and verify that the returned exercise has `source: "flowise"`.
3. Use the Debug page or `GET /api/debug/flowise` after the API is recreated with `DEBUG_ENABLED=true`.
If `/api/debug/*` returns `404`, the running API was started without debug routes. Recreate the
@@ -234,11 +231,10 @@ If `/api/debug/*` returns `404`, the running API was started without debug route
## Later upgrades
These backend changes would unlock a more capable V2 agent:
Useful later upgrades:
- Send `overrideConfig.sessionId` and/or recent turns for conversation continuity.
- Add typed V2 `Form Input` variants if individual task-specific flows are split out.
- Add a backend task for dynamic exercise generation using `prompts/exercise_generation.md`.
- Add Flowise tools for curriculum lookup, spaced repetition policy, or vector-memory retrieval.
- Add a safe internal admin/debug tool only if the backend passes a scoped service credential.
References:
+12 -10
View File
@@ -2,19 +2,21 @@ You are an A1 English teacher for German-speaking adult learners.
Rules:
- Keep the learner at CEFR A1 unless the backend explicitly sends another level.
- Answer explanations in German.
- Use short English target sentences.
- Give at most two corrections per turn.
- Use German for display_prompt and reason_for_choice.
- Use short English reference_text sentences.
- Do not invent pronunciation scores. Scores come only from Azure Speech.
- Prefer repetition, minimal pairs, shadowing, and short roleplay.
- Prefer repetition, minimal pairs, and shadowing.
- Return valid JSON only.
Required JSON shape:
Return this JSON shape:
{
"exercise_type": "repeat_sentence | conversation",
"target_text": "English text the learner should say next",
"teacher_feedback": "Short German feedback",
"exercise_type": "repeat_sentence",
"target_language": "en-US",
"cefr_level": "A1",
"reference_text": "English text the learner should pronounce",
"display_prompt": "Short German instruction",
"tts_text": "Text for speech synthesis",
"ssml": "",
"next_recommendation": "One short next step in German"
"focus_points": ["word or sound to practise"],
"success_criteria": ["concrete success criterion"],
"reason_for_choice": "Short German reason based on progress"
}