8.7 KiB
Agentflow V2 Setup
This is the recommended Flowise Agentflow V2 setup for the Speech Learning App. It is designed for the current Go backend adapter.
Runtime facts
- Production app:
https://speech.v2202510301049387828.goodsrv.de - Production API health:
https://speech.v2202510301049387828.goodsrv.de/api/health - Existing Flowise instance:
https://flowise.v2202510301049387828.goodsrv.de - Internal Flowise URL for the API container:
http://flowise:3000 - Verified Flowise version on the existing instance:
3.0.11 - The Go backend calls Flowise with
POST /api/v1/prediction/{FLOWISE_CHATFLOW_ID}. - The request body includes
questionas a JSON string andformas the same structured object.
Build the first V2 flow with Start Node input type Chat Input; it is the simplest path because
all current tasks arrive as one JSON context. V2 Form Input is also possible now, but only if the
form fields match the backend payload fields for the task you want to handle.
The backend parses Flowise responses from text, json, or a raw JSON response. A plain JSON text
reply is still the easiest response format to debug in Flowise and in the app.
Current backend tasks
The current backend can call the agent for these tasks:
practice_feedback- Sent after Azure/mock pronunciation assessment.
- Context includes
target_text,recognized_text,scores,weak_words, and rules. - The agent must never create or modify pronunciation scores.
start_conversation- Starts a short A1 roleplay.
- Context includes
target_languageandcefr_level.
continue_conversation- Continues a short A1 roleplay.
- Context includes
target_language,cefr_level, anduser_text.
The backend does not currently call Flowise to choose the next DB exercise. The prompt
prompts/exercise_generation.md is useful for a later backend extension, but it is not used now.
Recommended V2 graph
Create a new Agentflow V2 named Speech A1 English Teacher.
Recommended minimal graph:
Start Node (Chat Input)
-> LLM Node or Agent Node
-> Direct Reply Node
Use an LLM Node if you want the most stable JSON. Use an Agent Node if you want to add tools
later. For the current backend, the LLM Node is enough because the Go API already owns auth, DB,
Azure Speech, attempts, progress, and conversation persistence.
Start Node:
- Input Type:
Chat Input - Ephemeral Memory:
On - Flow State keys, if you use state:
teacher_response: empty stringvalidation_error: empty string
LLM/Agent Node:
- Input Message:
{{ $flow.input }} - Memory: off for now. The backend does not pass a Flowise
sessionIdyet. - Output: JSON text only.
Direct Reply Node:
- Message: the LLM/Agent output.
System prompt
Paste this into the LLM/Agent system message:
You are an A1 English teacher for German-speaking adult learners in a pronunciation and speaking app.
You receive one JSON object as the user message. It contains a field named "task".
Global rules:
- Return valid JSON only. No Markdown. No code fences. No commentary outside JSON.
- Use exactly this JSON shape:
{
"exercise_type": "repeat_sentence",
"target_text": "English text the learner should say next",
"teacher_feedback": "Short German feedback",
"tts_text": "Text for speech synthesis",
"ssml": "",
"next_recommendation": "One short next step in German"
}
- Allowed exercise_type values: "repeat_sentence", "conversation".
- Keep the learner at CEFR A1 unless the input explicitly says otherwise.
- Explain in German. Target sentences and roleplay turns are in English.
- Use short, everyday A1 English. Prefer 3 to 9 words for target_text when possible.
- Give at most two corrections per turn.
- Do not invent pronunciation scores. Scores come only from Azure Speech or the backend mock.
- Do not change score values. Use scores only to decide feedback.
- Prefer repetition, minimal pairs, shadowing, short roleplay, and one clear next step.
- Keep ssml as an empty string unless the backend later asks for SSML.
Task rules:
If task is "practice_feedback":
- Use target_text, recognized_text, scores, and weak_words.
- Mention the weakest word or sound if available.
- If overall_score is high, praise briefly and recommend a more natural rhythm.
- If weak_words are present, ask the learner to repeat the original target_text slowly.
- exercise_type must be "repeat_sentence".
- target_text and tts_text should usually equal the original target_text.
If task is "start_conversation":
- Start one simple A1 roleplay.
- Use one short teacher sentence as target_text and tts_text.
- Give a German next_recommendation that tells the learner how to answer.
- exercise_type must be "conversation".
If task is "continue_conversation":
- Read user_text and continue with one short teacher sentence.
- Correct only the most important issue, if any.
- Keep the roleplay moving with a simple A1 question or prompt.
- exercise_type must be "conversation".
If the task is missing or unknown:
- Return a safe repeat_sentence response for A1 practice.
JSON contract
Use this contract mentally or as a structured-output schema if the UI supports it.
{
"type": "object",
"additionalProperties": false,
"properties": {
"exercise_type": {
"type": "string",
"enum": ["repeat_sentence", "conversation"]
},
"target_text": {
"type": "string"
},
"teacher_feedback": {
"type": "string"
},
"tts_text": {
"type": "string"
},
"ssml": {
"type": "string"
},
"next_recommendation": {
"type": "string"
}
},
"required": [
"exercise_type",
"target_text",
"teacher_feedback",
"tts_text",
"ssml",
"next_recommendation"
]
}
Test payloads
Paste these as chat input in Flowise.
Practice feedback:
{"task":"practice_feedback","target_text":"Can I have a coffee, please?","recognized_text":"Can I have coffee please","scores":{"recognized_text":"Can I have coffee please","overall_score":72,"accuracy_score":70,"fluency_score":78,"completeness_score":82,"prosody_score":69,"word_scores":[{"word":"Can","accuracy":91,"highlight":"good"},{"word":"I","accuracy":88,"highlight":"good"},{"word":"have","accuracy":74,"highlight":"ok"},{"word":"a","accuracy":55,"highlight":"weak"},{"word":"coffee","accuracy":79,"highlight":"ok"},{"word":"please","accuracy":68,"highlight":"weak"}],"provider":"mock"},"weak_words":[{"word":"a","accuracy":55,"highlight":"weak"},{"word":"please","accuracy":68,"highlight":"weak"}],"rules":"Answer in German. A1 learner. Max two corrections. Return valid JSON only."}
Start conversation:
{"task":"start_conversation","target_language":"en-US","cefr_level":"A1","rules":"Create a simple A1 roleplay. Return valid JSON only."}
Continue conversation:
{"task":"continue_conversation","target_language":"en-US","cefr_level":"A1","user_text":"My name Anna","rules":"Continue a simple A1 conversation. Return valid JSON only."}
Expected output example:
{
"exercise_type": "conversation",
"target_text": "Nice to meet you, Anna.",
"teacher_feedback": "Fast richtig: Sage \"My name is Anna.\"",
"tts_text": "Nice to meet you, Anna.",
"ssml": "",
"next_recommendation": "Antworte kurz: I am from ..."
}
Activation
After saving the Agentflow V2, copy its ID into .env:
FLOWISE_CHATFLOW_ID=<flow-id>
If the flow has API-key protection enabled, also set:
FLOWISE_API_KEY=<flow-api-key>
Then recreate the API container on the VPS. Use --build when deploying backend code changes:
docker compose up --build -d speech-api
Validation
- In Flowise, run each test payload and confirm the reply is plain JSON text.
- In the app, submit a pronunciation attempt and verify that the
teacherresponse is not the local fallback. - Use the Debug page or
GET /api/debug/flowiseafter the API is recreated withDEBUG_ENABLED=true.
If /api/debug/* returns 404, the running API was started without debug routes. Recreate the
speech-api container with the current .env.
Later upgrades
These backend changes would unlock a more capable V2 agent:
- Send
overrideConfig.sessionIdand/or recent turns for conversation continuity. - Add typed V2
Form Inputvariants if individual task-specific flows are split out. - Add a backend task for dynamic exercise generation using
prompts/exercise_generation.md. - Add a safe internal admin/debug tool only if the backend passes a scoped service credential.
References:
- Flowise Agentflow V2: https://docs.flowiseai.com/using-flowise/agentflowv2
- Flowise Prediction API: https://docs.flowiseai.com/api-reference/prediction
- Flowise Tools and MCP: https://docs.flowiseai.com/tutorials/tools-and-mcp