Files
Speech/flowise/agentflow-v2-setup.md
T

245 lines
8.3 KiB
Markdown

# Agentflow V2 Setup
This is the recommended Flowise Agentflow V2 setup for the Speech Learning App.
It is designed for the current Go backend adapter.
## Runtime facts
- Production app: `https://speech.v2202510301049387828.goodsrv.de`
- Production API health: `https://speech.v2202510301049387828.goodsrv.de/api/health`
- Existing Flowise instance: `https://flowise.v2202510301049387828.goodsrv.de`
- Internal Flowise URL for the API container: `http://flowise:3000`
- Verified Flowise version on the existing instance: `3.0.11`
- The Go backend calls Flowise with `POST /api/v1/prediction/{FLOWISE_CHATFLOW_ID}`.
- The request body includes `question` as one JSON string and `overrideConfig.sessionId`.
Build the first V2 flow with Start Node input type `Chat Input`; it is the simplest path because
the backend sends one JSON context as the chat input. V2 `Form Input` can come later if you split
the flow into typed task-specific flows.
The backend parses Flowise responses from `text`, `json`, or a raw JSON response. A plain JSON text
reply is still the easiest response format to debug in Flowise and in the app.
## Current backend tasks
The current backend sends only one task to Flowise:
1. `next_exercise`
- Sent when dashboard or `POST /lessons/next` needs the next speaking exercise.
- Context includes learner profile, recent progress, weak words, target language, CEFR level, and rules.
- The agent creates exactly one A1 exercise. The backend stores it in PostgreSQL for that user.
Flowise owns the exercise-generation logic. PostgreSQL remains the durable system of record for
users, profiles, generated exercises, attempts, and progress. Pronunciation scoring stays in Azure
Speech/the backend. Feedback and the simple conversation screen are local fallback logic for now.
## Recommended V2 graph
Create a new `Agentflow V2` named `Speech A1 English Teacher`.
Recommended minimal graph:
```txt
Start Node (Chat Input)
-> LLM Node or Agent Node
-> Direct Reply Node
```
Use an `LLM Node` if you want the most stable JSON. Use an `Agent Node` if you want to add tools
later. For the current backend, the LLM Node is enough because the Go API already owns auth, DB,
Azure Speech, attempts, progress, and conversation persistence.
Start Node:
- Input Type: `Chat Input`
- Ephemeral Memory: `On`
- Flow State keys, if you use state:
- `teacher_response`: empty string
- `validation_error`: empty string
LLM/Agent Node:
- Input Message: `{{ $flow.input }}`
- Memory: on or Flowise chat memory enabled if you want per-user continuity.
- The backend sends `overrideConfig.sessionId` in the shape `user_id:target_language:practice`.
- Output: JSON text only.
Direct Reply Node:
- Message: the LLM/Agent output.
## System prompt
Paste this into the LLM/Agent system message:
```txt
You are an A1 English teacher for German-speaking adult learners in a pronunciation and speaking app.
You receive one JSON object as the user message. It contains a field named "task".
Global rules:
- Return valid JSON only. No Markdown. No code fences. No commentary outside JSON.
- Read the input JSON field "task".
- If task is "next_exercise", use exactly this JSON shape:
{
"exercise_type": "repeat_sentence",
"target_language": "en-US",
"cefr_level": "A1",
"reference_text": "English sentence the learner must pronounce",
"display_prompt": "Short German instruction",
"tts_text": "English text for speech synthesis",
"focus_points": ["one or two words or sounds to practise"],
"success_criteria": ["concrete success criterion"],
"reason_for_choice": "Short German reason based on learner progress"
}
- If the task is missing or unknown, still return one safe A1 repeat_sentence exercise in this same shape.
- Allowed exercise_type value for now: "repeat_sentence".
- Keep the learner at CEFR A1 unless the input explicitly says otherwise.
- Use German for display_prompt and reason_for_choice.
- Use English for reference_text and tts_text.
- Use short, everyday A1 English. Prefer 3 to 9 words for reference_text when possible.
- Do not invent pronunciation scores. Scores come only from Azure Speech or the backend mock.
- Use recent weak_words if present.
- Prefer repetition, minimal pairs, shadowing, and one clear pronunciation focus.
Task rules:
If task is "next_exercise":
- Create one short pronunciation exercise for the learner.
- exercise_type must be "repeat_sentence" for the current MVP.
- reference_text and tts_text should be English and usually identical.
- display_prompt and reason_for_choice should be German.
- Use recent weak_words if present.
- Keep the sentence short, natural, and useful for A1.
- Do not include pronunciation scores.
```
## JSON contract
Use this contract mentally or as a structured-output schema if the UI supports it.
Exercise generation response for `next_exercise`:
```json
{
"type": "object",
"additionalProperties": false,
"properties": {
"exercise_type": {
"type": "string",
"enum": ["repeat_sentence"]
},
"target_language": {
"type": "string"
},
"cefr_level": {
"type": "string"
},
"reference_text": {
"type": "string"
},
"display_prompt": {
"type": "string"
},
"tts_text": {
"type": "string"
},
"focus_points": {
"type": "array",
"items": {
"type": "string"
}
},
"success_criteria": {
"type": "array",
"items": {
"type": "string"
}
},
"reason_for_choice": {
"type": "string"
}
},
"required": [
"exercise_type",
"target_language",
"cefr_level",
"reference_text",
"display_prompt",
"tts_text",
"focus_points",
"success_criteria",
"reason_for_choice"
]
}
```
## Test payloads
Paste these as chat input in Flowise.
Next exercise:
```json
{"task":"next_exercise","user_id":"usr_demo","target_language":"en-US","cefr_level":"A1","ui_language":"de-DE","profile":{"target_language":"en-US","cefr_level":"A1","daily_goal_minutes":10},"progress":{"total_attempts":2,"average_score":73,"weak_words":[{"word":"please","accuracy":61,"highlight":"weak"}],"recent_attempts":[]},"recent_weak_words":[{"word":"please","accuracy":61,"highlight":"weak"}],"preferred_type":"repeat_sentence","rules":"Generate one A1 English pronunciation exercise. Return valid JSON only with fields: exercise_type, target_language, cefr_level, reference_text, display_prompt, tts_text, focus_points, success_criteria, reason_for_choice."}
```
Expected output example:
```json
{
"exercise_type": "repeat_sentence",
"target_language": "en-US",
"cefr_level": "A1",
"reference_text": "Can I have water, please?",
"display_prompt": "Sprich den Satz langsam und klar nach.",
"tts_text": "Can I have water, please?",
"focus_points": ["please", "water"],
"success_criteria": ["accuracy above 80", "clear final sounds"],
"reason_for_choice": "Die Uebung wiederholt ein schwaches Wort in einem kurzen Alltagssatz."
}
```
## Activation
After saving the Agentflow V2, copy its ID into `.env`:
```env
FLOWISE_CHATFLOW_ID=<flow-id>
```
If the flow has API-key protection enabled, also set:
```env
FLOWISE_API_KEY=<flow-api-key>
```
Then recreate the API container on the VPS. Use `--build` when deploying backend code changes:
```bash
docker compose up --build -d speech-api
```
## Validation
1. In Flowise, run each test payload and confirm the reply is plain JSON text.
2. In the app, open the dashboard or click the next lesson button and verify that the returned exercise has `source: "flowise"`.
3. Use the Debug page or `GET /api/debug/flowise` after the API is recreated with `DEBUG_ENABLED=true`.
If `/api/debug/*` returns `404`, the running API was started without debug routes. Recreate the
`speech-api` container with the current `.env`.
## Later upgrades
Useful later upgrades:
- Add typed V2 `Form Input` variants if individual task-specific flows are split out.
- Add Flowise tools for curriculum lookup, spaced repetition policy, or vector-memory retrieval.
- Add a safe internal admin/debug tool only if the backend passes a scoped service credential.
References:
- Flowise Agentflow V2: https://docs.flowiseai.com/using-flowise/agentflowv2
- Flowise Prediction API: https://docs.flowiseai.com/api-reference/prediction
- Flowise Tools and MCP: https://docs.flowiseai.com/tutorials/tools-and-mcp