änderungen von vorherigen Codexnutzung: sollte den Agent lassen die Aufgaben zu erzeugen
This commit is contained in:
+11
-7
@@ -6,7 +6,7 @@ Flowise laeuft in Docker als Service `flowise` und ist vom Backend intern ueber
|
||||
|
||||
Fuer Agentflow V2 nutze zuerst die konkrete Anleitung in
|
||||
[`agentflow-v2-setup.md`](./agentflow-v2-setup.md). Sie passt zur aktuellen
|
||||
Go-Backend-Anbindung, die Flowise mit `question` und `form` aufruft.
|
||||
Go-Backend-Anbindung, die Flowise mit `question` und `overrideConfig.sessionId` aufruft.
|
||||
|
||||
1. Flowise im Browser oeffnen: http://localhost:3000
|
||||
2. Einen Chatflow oder Agentflow fuer `A1 English Teacher` erstellen.
|
||||
@@ -17,17 +17,21 @@ Go-Backend-Anbindung, die Flowise mit `question` und `form` aufruft.
|
||||
FLOWISE_CHATFLOW_ID=...
|
||||
```
|
||||
|
||||
Das Backend erwartet JSON-Antworten mit:
|
||||
Fuer die naechste Uebung erwartet das Backend JSON-Antworten mit:
|
||||
|
||||
```json
|
||||
{
|
||||
"exercise_type": "repeat_sentence",
|
||||
"target_text": "Hello, my name is Anna.",
|
||||
"teacher_feedback": "Uebe besonders 'name'.",
|
||||
"target_language": "en-US",
|
||||
"cefr_level": "A1",
|
||||
"reference_text": "Hello, my name is Anna.",
|
||||
"display_prompt": "Sprich den Satz klar nach.",
|
||||
"tts_text": "Hello, my name is Anna.",
|
||||
"ssml": "",
|
||||
"next_recommendation": "Sprich den Satz noch einmal langsam."
|
||||
"focus_points": ["name"],
|
||||
"success_criteria": ["accuracy above 80"],
|
||||
"reason_for_choice": "Kurzer A1-Satz fuer klare Betonung."
|
||||
}
|
||||
```
|
||||
|
||||
Wenn keine Chatflow-ID gesetzt ist, nutzt die API einen lokalen Fallback.
|
||||
Feedback und einfache Konversation laufen vorerst lokal im Backend. Wenn keine Chatflow-ID gesetzt
|
||||
ist oder Flowise kein valides JSON liefert, nutzt die API Seed-Uebungen als Fallback.
|
||||
|
||||
@@ -11,32 +11,27 @@ It is designed for the current Go backend adapter.
|
||||
- Internal Flowise URL for the API container: `http://flowise:3000`
|
||||
- Verified Flowise version on the existing instance: `3.0.11`
|
||||
- The Go backend calls Flowise with `POST /api/v1/prediction/{FLOWISE_CHATFLOW_ID}`.
|
||||
- The request body includes `question` as a JSON string and `form` as the same structured object.
|
||||
- The request body includes `question` as one JSON string and `overrideConfig.sessionId`.
|
||||
|
||||
Build the first V2 flow with Start Node input type `Chat Input`; it is the simplest path because
|
||||
all current tasks arrive as one JSON context. V2 `Form Input` is also possible now, but only if the
|
||||
form fields match the backend payload fields for the task you want to handle.
|
||||
the backend sends one JSON context as the chat input. V2 `Form Input` can come later if you split
|
||||
the flow into typed task-specific flows.
|
||||
|
||||
The backend parses Flowise responses from `text`, `json`, or a raw JSON response. A plain JSON text
|
||||
reply is still the easiest response format to debug in Flowise and in the app.
|
||||
|
||||
## Current backend tasks
|
||||
|
||||
The current backend can call the agent for these tasks:
|
||||
The current backend sends only one task to Flowise:
|
||||
|
||||
1. `practice_feedback`
|
||||
- Sent after Azure/mock pronunciation assessment.
|
||||
- Context includes `target_text`, `recognized_text`, `scores`, `weak_words`, and rules.
|
||||
- The agent must never create or modify pronunciation scores.
|
||||
2. `start_conversation`
|
||||
- Starts a short A1 roleplay.
|
||||
- Context includes `target_language` and `cefr_level`.
|
||||
3. `continue_conversation`
|
||||
- Continues a short A1 roleplay.
|
||||
- Context includes `target_language`, `cefr_level`, and `user_text`.
|
||||
1. `next_exercise`
|
||||
- Sent when dashboard or `POST /lessons/next` needs the next speaking exercise.
|
||||
- Context includes learner profile, recent progress, weak words, target language, CEFR level, and rules.
|
||||
- The agent creates exactly one A1 exercise. The backend stores it in PostgreSQL for that user.
|
||||
|
||||
The backend does not currently call Flowise to choose the next DB exercise. The prompt
|
||||
`prompts/exercise_generation.md` is useful for a later backend extension, but it is not used now.
|
||||
Flowise owns the exercise-generation logic. PostgreSQL remains the durable system of record for
|
||||
users, profiles, generated exercises, attempts, and progress. Pronunciation scoring stays in Azure
|
||||
Speech/the backend. Feedback and the simple conversation screen are local fallback logic for now.
|
||||
|
||||
## Recommended V2 graph
|
||||
|
||||
@@ -65,7 +60,8 @@ Start Node:
|
||||
LLM/Agent Node:
|
||||
|
||||
- Input Message: `{{ $flow.input }}`
|
||||
- Memory: off for now. The backend does not pass a Flowise `sessionId` yet.
|
||||
- Memory: on or Flowise chat memory enabled if you want per-user continuity.
|
||||
- The backend sends `overrideConfig.sessionId` in the shape `user_id:target_language:practice`.
|
||||
- Output: JSON text only.
|
||||
|
||||
Direct Reply Node:
|
||||
@@ -83,55 +79,47 @@ You receive one JSON object as the user message. It contains a field named "task
|
||||
|
||||
Global rules:
|
||||
- Return valid JSON only. No Markdown. No code fences. No commentary outside JSON.
|
||||
- Use exactly this JSON shape:
|
||||
- Read the input JSON field "task".
|
||||
- If task is "next_exercise", use exactly this JSON shape:
|
||||
{
|
||||
"exercise_type": "repeat_sentence",
|
||||
"target_text": "English text the learner should say next",
|
||||
"teacher_feedback": "Short German feedback",
|
||||
"tts_text": "Text for speech synthesis",
|
||||
"ssml": "",
|
||||
"next_recommendation": "One short next step in German"
|
||||
"target_language": "en-US",
|
||||
"cefr_level": "A1",
|
||||
"reference_text": "English sentence the learner must pronounce",
|
||||
"display_prompt": "Short German instruction",
|
||||
"tts_text": "English text for speech synthesis",
|
||||
"focus_points": ["one or two words or sounds to practise"],
|
||||
"success_criteria": ["concrete success criterion"],
|
||||
"reason_for_choice": "Short German reason based on learner progress"
|
||||
}
|
||||
- Allowed exercise_type values: "repeat_sentence", "conversation".
|
||||
- If the task is missing or unknown, still return one safe A1 repeat_sentence exercise in this same shape.
|
||||
- Allowed exercise_type value for now: "repeat_sentence".
|
||||
- Keep the learner at CEFR A1 unless the input explicitly says otherwise.
|
||||
- Explain in German. Target sentences and roleplay turns are in English.
|
||||
- Use short, everyday A1 English. Prefer 3 to 9 words for target_text when possible.
|
||||
- Give at most two corrections per turn.
|
||||
- Use German for display_prompt and reason_for_choice.
|
||||
- Use English for reference_text and tts_text.
|
||||
- Use short, everyday A1 English. Prefer 3 to 9 words for reference_text when possible.
|
||||
- Do not invent pronunciation scores. Scores come only from Azure Speech or the backend mock.
|
||||
- Do not change score values. Use scores only to decide feedback.
|
||||
- Prefer repetition, minimal pairs, shadowing, short roleplay, and one clear next step.
|
||||
- Keep ssml as an empty string unless the backend later asks for SSML.
|
||||
- Use recent weak_words if present.
|
||||
- Prefer repetition, minimal pairs, shadowing, and one clear pronunciation focus.
|
||||
|
||||
Task rules:
|
||||
|
||||
If task is "practice_feedback":
|
||||
- Use target_text, recognized_text, scores, and weak_words.
|
||||
- Mention the weakest word or sound if available.
|
||||
- If overall_score is high, praise briefly and recommend a more natural rhythm.
|
||||
- If weak_words are present, ask the learner to repeat the original target_text slowly.
|
||||
- exercise_type must be "repeat_sentence".
|
||||
- target_text and tts_text should usually equal the original target_text.
|
||||
|
||||
If task is "start_conversation":
|
||||
- Start one simple A1 roleplay.
|
||||
- Use one short teacher sentence as target_text and tts_text.
|
||||
- Give a German next_recommendation that tells the learner how to answer.
|
||||
- exercise_type must be "conversation".
|
||||
|
||||
If task is "continue_conversation":
|
||||
- Read user_text and continue with one short teacher sentence.
|
||||
- Correct only the most important issue, if any.
|
||||
- Keep the roleplay moving with a simple A1 question or prompt.
|
||||
- exercise_type must be "conversation".
|
||||
|
||||
If the task is missing or unknown:
|
||||
- Return a safe repeat_sentence response for A1 practice.
|
||||
If task is "next_exercise":
|
||||
- Create one short pronunciation exercise for the learner.
|
||||
- exercise_type must be "repeat_sentence" for the current MVP.
|
||||
- reference_text and tts_text should be English and usually identical.
|
||||
- display_prompt and reason_for_choice should be German.
|
||||
- Use recent weak_words if present.
|
||||
- Keep the sentence short, natural, and useful for A1.
|
||||
- Do not include pronunciation scores.
|
||||
```
|
||||
|
||||
## JSON contract
|
||||
|
||||
Use this contract mentally or as a structured-output schema if the UI supports it.
|
||||
|
||||
Exercise generation response for `next_exercise`:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "object",
|
||||
@@ -139,31 +127,49 @@ Use this contract mentally or as a structured-output schema if the UI supports i
|
||||
"properties": {
|
||||
"exercise_type": {
|
||||
"type": "string",
|
||||
"enum": ["repeat_sentence", "conversation"]
|
||||
"enum": ["repeat_sentence"]
|
||||
},
|
||||
"target_text": {
|
||||
"target_language": {
|
||||
"type": "string"
|
||||
},
|
||||
"teacher_feedback": {
|
||||
"cefr_level": {
|
||||
"type": "string"
|
||||
},
|
||||
"reference_text": {
|
||||
"type": "string"
|
||||
},
|
||||
"display_prompt": {
|
||||
"type": "string"
|
||||
},
|
||||
"tts_text": {
|
||||
"type": "string"
|
||||
},
|
||||
"ssml": {
|
||||
"type": "string"
|
||||
"focus_points": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"next_recommendation": {
|
||||
"success_criteria": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"reason_for_choice": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"exercise_type",
|
||||
"target_text",
|
||||
"teacher_feedback",
|
||||
"target_language",
|
||||
"cefr_level",
|
||||
"reference_text",
|
||||
"display_prompt",
|
||||
"tts_text",
|
||||
"ssml",
|
||||
"next_recommendation"
|
||||
"focus_points",
|
||||
"success_criteria",
|
||||
"reason_for_choice"
|
||||
]
|
||||
}
|
||||
```
|
||||
@@ -172,34 +178,25 @@ Use this contract mentally or as a structured-output schema if the UI supports i
|
||||
|
||||
Paste these as chat input in Flowise.
|
||||
|
||||
Practice feedback:
|
||||
Next exercise:
|
||||
|
||||
```json
|
||||
{"task":"practice_feedback","target_text":"Can I have a coffee, please?","recognized_text":"Can I have coffee please","scores":{"recognized_text":"Can I have coffee please","overall_score":72,"accuracy_score":70,"fluency_score":78,"completeness_score":82,"prosody_score":69,"word_scores":[{"word":"Can","accuracy":91,"highlight":"good"},{"word":"I","accuracy":88,"highlight":"good"},{"word":"have","accuracy":74,"highlight":"ok"},{"word":"a","accuracy":55,"highlight":"weak"},{"word":"coffee","accuracy":79,"highlight":"ok"},{"word":"please","accuracy":68,"highlight":"weak"}],"provider":"mock"},"weak_words":[{"word":"a","accuracy":55,"highlight":"weak"},{"word":"please","accuracy":68,"highlight":"weak"}],"rules":"Answer in German. A1 learner. Max two corrections. Return valid JSON only."}
|
||||
```
|
||||
|
||||
Start conversation:
|
||||
|
||||
```json
|
||||
{"task":"start_conversation","target_language":"en-US","cefr_level":"A1","rules":"Create a simple A1 roleplay. Return valid JSON only."}
|
||||
```
|
||||
|
||||
Continue conversation:
|
||||
|
||||
```json
|
||||
{"task":"continue_conversation","target_language":"en-US","cefr_level":"A1","user_text":"My name Anna","rules":"Continue a simple A1 conversation. Return valid JSON only."}
|
||||
{"task":"next_exercise","user_id":"usr_demo","target_language":"en-US","cefr_level":"A1","ui_language":"de-DE","profile":{"target_language":"en-US","cefr_level":"A1","daily_goal_minutes":10},"progress":{"total_attempts":2,"average_score":73,"weak_words":[{"word":"please","accuracy":61,"highlight":"weak"}],"recent_attempts":[]},"recent_weak_words":[{"word":"please","accuracy":61,"highlight":"weak"}],"preferred_type":"repeat_sentence","rules":"Generate one A1 English pronunciation exercise. Return valid JSON only with fields: exercise_type, target_language, cefr_level, reference_text, display_prompt, tts_text, focus_points, success_criteria, reason_for_choice."}
|
||||
```
|
||||
|
||||
Expected output example:
|
||||
|
||||
```json
|
||||
{
|
||||
"exercise_type": "conversation",
|
||||
"target_text": "Nice to meet you, Anna.",
|
||||
"teacher_feedback": "Fast richtig: Sage \"My name is Anna.\"",
|
||||
"tts_text": "Nice to meet you, Anna.",
|
||||
"ssml": "",
|
||||
"next_recommendation": "Antworte kurz: I am from ..."
|
||||
"exercise_type": "repeat_sentence",
|
||||
"target_language": "en-US",
|
||||
"cefr_level": "A1",
|
||||
"reference_text": "Can I have water, please?",
|
||||
"display_prompt": "Sprich den Satz langsam und klar nach.",
|
||||
"tts_text": "Can I have water, please?",
|
||||
"focus_points": ["please", "water"],
|
||||
"success_criteria": ["accuracy above 80", "clear final sounds"],
|
||||
"reason_for_choice": "Die Uebung wiederholt ein schwaches Wort in einem kurzen Alltagssatz."
|
||||
}
|
||||
```
|
||||
|
||||
@@ -226,7 +223,7 @@ docker compose up --build -d speech-api
|
||||
## Validation
|
||||
|
||||
1. In Flowise, run each test payload and confirm the reply is plain JSON text.
|
||||
2. In the app, submit a pronunciation attempt and verify that the `teacher` response is not the local fallback.
|
||||
2. In the app, open the dashboard or click the next lesson button and verify that the returned exercise has `source: "flowise"`.
|
||||
3. Use the Debug page or `GET /api/debug/flowise` after the API is recreated with `DEBUG_ENABLED=true`.
|
||||
|
||||
If `/api/debug/*` returns `404`, the running API was started without debug routes. Recreate the
|
||||
@@ -234,11 +231,10 @@ If `/api/debug/*` returns `404`, the running API was started without debug route
|
||||
|
||||
## Later upgrades
|
||||
|
||||
These backend changes would unlock a more capable V2 agent:
|
||||
Useful later upgrades:
|
||||
|
||||
- Send `overrideConfig.sessionId` and/or recent turns for conversation continuity.
|
||||
- Add typed V2 `Form Input` variants if individual task-specific flows are split out.
|
||||
- Add a backend task for dynamic exercise generation using `prompts/exercise_generation.md`.
|
||||
- Add Flowise tools for curriculum lookup, spaced repetition policy, or vector-memory retrieval.
|
||||
- Add a safe internal admin/debug tool only if the backend passes a scoped service credential.
|
||||
|
||||
References:
|
||||
|
||||
@@ -2,19 +2,21 @@ You are an A1 English teacher for German-speaking adult learners.
|
||||
|
||||
Rules:
|
||||
- Keep the learner at CEFR A1 unless the backend explicitly sends another level.
|
||||
- Answer explanations in German.
|
||||
- Use short English target sentences.
|
||||
- Give at most two corrections per turn.
|
||||
- Use German for display_prompt and reason_for_choice.
|
||||
- Use short English reference_text sentences.
|
||||
- Do not invent pronunciation scores. Scores come only from Azure Speech.
|
||||
- Prefer repetition, minimal pairs, shadowing, and short roleplay.
|
||||
- Prefer repetition, minimal pairs, and shadowing.
|
||||
- Return valid JSON only.
|
||||
|
||||
Required JSON shape:
|
||||
Return this JSON shape:
|
||||
{
|
||||
"exercise_type": "repeat_sentence | conversation",
|
||||
"target_text": "English text the learner should say next",
|
||||
"teacher_feedback": "Short German feedback",
|
||||
"exercise_type": "repeat_sentence",
|
||||
"target_language": "en-US",
|
||||
"cefr_level": "A1",
|
||||
"reference_text": "English text the learner should pronounce",
|
||||
"display_prompt": "Short German instruction",
|
||||
"tts_text": "Text for speech synthesis",
|
||||
"ssml": "",
|
||||
"next_recommendation": "One short next step in German"
|
||||
"focus_points": ["word or sound to practise"],
|
||||
"success_criteria": ["concrete success criterion"],
|
||||
"reason_for_choice": "Short German reason based on progress"
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user