My Rasa Mantle agent listed its branch opening hours, and when I asked a follow-up question, it sometimes replied only “Can I help you with anything else today?”. The question never got an answer, because the skill had already ended. Here’s why that happens on Rasa Pro 3.22, and the fix: one sentence in the skill’s instructions and a complete_when rule, so the skill can’t end before the question is answered.
What I Was Doing
I was running the agent from Rasa Tutorial — Part 3: Write a Skill’s Instructions, on Rasa Pro 3.22.0 with Claude Sonnet 5.5. Its Branch Hours skill tells customers when the branches are open, from the hours written in its instructions:
---
name: Branch Hours
description: Branch opening hours - what time branches open and close on weekdays, weekends and public holidays
---
Tell the customer when our branches are open. Every branch keeps the same
hours:
- Monday to Friday: 9 am to 5 pm
- Saturday: 9 am to 1 pm
- Sunday and public holidays: closed
Answer for the day the customer asks about. If they don't name a day, give
the hours for the whole week.
When the branches are closed, mention that the mobile app works around the
clock.
In the Rasa Inspector, I asked “When are your branches open?”, then the follow-up “Are you open on Sunday?”.
What I Expected
An answer to the follow-up: the branches are closed on Sundays.
What I Got
The first answer listed the hours for the whole week, Sunday included. The follow-up got no answer, only the question “Can I help you with anything else today?”:

It doesn’t happen every time. When I restarted the conversation and asked the same two questions, the agent answered the follow-up.
What I Checked
The Inspector shows what happened behind the reply. Open the Inspector mode menu at the top of the chat (the sliders button) and choose Inspect. Under my follow-up question, the Branch Hours skill completed, Rasa’s built-in Completed skill started, and it asked its wrap-up question:

Clicking the LLM call line just under the question shows what Claude was sent and what it returned. One of the tools Claude was offered was complete, and Claude’s whole response was a call to it, with no reply for the customer:

Rasa’s guide State when the skill is finished (opens in a new tab) explains both parts. complete finishes the active skill once its work is done. After a skill finishes and nothing else is waiting, the built-in Completed skill asks whether the customer wants more help.
Why It Happens
Claude decides when a skill is finished, by calling complete. My first answer had already listed Sunday as closed, so on the follow-up Claude judged the skill’s work done and called complete without replying. The Completed skill then asked its usual question, so the customer got a wrap-up instead of an answer.
Claude makes that choice again in every conversation, which is why the problem comes and goes. Nothing in my skill said that a follow-up still needs its own answer, and nothing stopped the skill from ending before it.
The Fix
The fix is two small changes to skills/branch_hours/skill.md. The first tells Claude the skill isn’t finished yet. The second makes Rasa refuse to end the skill too early.
1. Tell Claude a follow-up still needs an answer
Rasa’s guide says to write a sentence about finishing only where a step looks finished but isn’t. Here, giving the week’s hours looks finished, but a follow-up question still needs its answer. I added one sentence to the instructions:
Answer for the day the customer asks about. If they don't name a day, give
the hours for the whole week. Answer every follow-up question about the
hours too, even when your earlier answer already covered that day.
2. Make Rasa refuse an early complete
complete_when goes in the skill’s frontmatter, between the two --- lines. It lists what must be true before the skill can end. Rasa checks it each time Claude calls complete, and Claude never sees it. Each line under criteria is a plain sentence that Rasa checks against the conversation with a short extra LLM call. If a sentence isn’t true yet, Rasa refuses complete and keeps the skill open.
---
name: Branch Hours
description: Branch opening hours - what time branches open and close on weekdays, weekends and public holidays
complete_when:
criteria:
- "Every question the customer asked was answered after they asked it. A later thank-you or goodbye does not undo that."
---
The criterion’s second sentence follows Rasa’s advice: word each criterion so a later message can’t undo it, or a closing “thanks” could keep the skill open after the job is done.
Here’s the whole file with both changes:
---
name: Branch Hours
description: Branch opening hours - what time branches open and close on weekdays, weekends and public holidays
complete_when:
criteria:
- "Every question the customer asked was answered after they asked it. A later thank-you or goodbye does not undo that."
---
Tell the customer when our branches are open. Every branch keeps the same
hours:
- Monday to Friday: 9 am to 5 pm
- Saturday: 9 am to 1 pm
- Sunday and public holidays: closed
Answer for the day the customer asks about. If they don't name a day, give
the hours for the whole week. Answer every follow-up question about the
hours too, even when your earlier answer already covered that day.
When the branches are closed, mention that the mobile app works around the
clock.
Train the agent again, so the changed skill is packaged:
rasa train
Then open the Inspector with the new version:
rasa inspect
After the Fix
In a new conversation with the same two questions, the follow-up gets its answer:

In Inspect mode, the answer comes straight after the question, and there’s no “Branch Hours skill completed” line, so the skill stays open for more questions:

The skill still ends when the customer is done. After “Thanks, that’s all.”, Rasa’s check passes, Branch Hours completes, and the Completed skill asks whether the customer needs anything else, this time in the right place.
Common Mistakes
| Mistake | Do this instead |
|---|---|
| Writing when to finish into the instructions, such as “When the customer is satisfied, call complete” | Leave it out. Rasa already tells Claude when to finish, and its guide warns that such lines can end the skill early. Only point out a step that looks finished but isn’t. |
Writing complete_when as one sentence on the same line | Put criteria: under it, with a list of sentences. Rasa rejects a plain string when you train. |
| A criterion that a “thanks” can undo, such as “The agent answered the customer’s last message” | Say that a later thank-you or goodbye doesn’t undo it. |
Testing straight after editing skill.md | Run rasa train first. The Inspector runs the last trained version of your agent. |
Troubleshooting
| Problem | What to check |
|---|---|
| The agent still replies only “Can I help you with anything else today?” | Check that you ran rasa train after the change, and that complete_when sits between the two --- lines, indented as shown. |
| The agent replies “I’m sorry, I wasn’t able to do that. What else can I help you with?” | Rasa refused complete and Claude asked for it again. Check that the new sentence is in the instructions, not only complete_when. |
rasa train shows an error about the frontmatter | Check the indentation: criteria: two spaces in, and each sentence four spaces in, after - and in quotes. |
FAQ
Why does it happen only sometimes?
Claude decides when to call complete in every conversation, so the same two questions can end differently each time.
Is complete_when enough on its own?
Not always. Claude doesn’t see complete_when. When I tried it without the new sentence, Claude sometimes asked to complete again straight after Rasa refused, and Rasa then ended the turn with “I’m sorry, I wasn’t able to do that. What else can I help you with?”. The sentence stops Claude from trying too early, and complete_when catches it if it still does. Rasa’s guide pairs the two the same way.
Does complete_when cost anything?
A little. Each time Claude calls complete, Rasa makes one short extra LLM call to check the criteria, billed like the agent’s other calls.
Should I remove the “anything else” question instead?
No. After a finished task, it’s a useful question. The problem was the skill ending before it answered, and the fix removes that.
Can other skills end early like this?
Yes. Any skill that answers questions can end the same way on a follow-up. Use the same two changes in that skill’s skill.md, with a sentence that fits its own job.
Conclusion
The wrap-up question came from a skill that ended too early: Claude called complete instead of answering a follow-up. One sentence in the instructions tells Claude the job isn’t done, and complete_when makes Rasa refuse an early complete. With both, the follow-up gets its answer, and the skill still ends normally when the customer is done.
Additional Resources
- State when the skill is finished (opens in a new tab) (Rasa docs)
- skill.md reference: complete_when (opens in a new tab) (Rasa docs)
Related
- Rasa Tutorial — Part 3: Write a Skill’s Instructions
- Rasa Tutorial for Beginners: Mastering Rasa Pro in Python, with every part of the series
Rasa, Claude and other names mentioned are trademarks of their respective owners. JD Bots isn’t affiliated with or endorsed by Rasa Technologies or Anthropic.
0 comments