Skip to main content
0 reactions

Fixed: Rasa Mantle Asks the Customer to Confirm Twice Before a Tool Runs

Featured image for the fix Rasa Mantle Asks the Customer to Confirm Twice: a chat where the customer says “Friday, please.”, a dashed reply “Would you like me to book this…” marked “Then Rasa asks again”, and Rasa’s own question “Shall I book this appointment?” with a tick; notes say it was seen on Rasa Pro 3.22 and that the fix is to say it in Rasa’s question.
Fixed: Rasa Mantle asks the customer to confirm twice.

A Rasa Mantle agent asks “Would you like me to book this appointment?”, the customer says yes, and then Rasa asks “Shall I book this appointment?” all over again. Here’s why one sentence in a skill’s instructions caused it, and the fix that leaves a single question, Rasa’s own, with the reminder inside it.

Seen on Rasa Pro 3.22.0 with the Mantle engine and Claude Sonnet 5.5.

What I Was Doing

I was writing Part 9 of the Rasa Tutorial, on scoped instructions, with the banking agent from Part 8. Its booking skill already made Rasa ask the customer before booking (requires_confirmation, with an exact question). I added one paragraph per appointment topic, telling the agent what the customer should bring. This is the loan one:

skills/book_appointment/skill.md (excerpt)Plain text
if: session.book_appointment.topic == 'loan'
Before you book, tell the customer to bring their last three payslips and
their bank statements for the last three months.

What I Expected

Claude would pass the reminder on, call the booking tool, and Rasa would ask its confirmation question once.

What I Got

In the Rasa Inspector, I booked a loan appointment one detail at a time. After I gave the day, Claude listed the details, passed the reminder on and asked a question of its own. When I said yes, Claude called the booking tool, and Rasa asked its question as well:

Inspect mode in the Rasa Inspector: after the customer says “Friday, please.”, the agent lists the branch, topic and day, asks them to bring their last three payslips and bank statements, and asks “Would you like me to book this appointment?”, marked LLM generated; the customer replies “Yes, please.”, a tool call to book_appointment follows, and the agent asks “Shall I book this appointment? Branch: Riverton Station. Topic: loan. Day: friday.”, marked utter_ask_book_appointment_confirmation template.
Claude asked first, then Rasa asked again.

The customer had to say yes twice: once to Claude (“Would you like me to book this appointment?”, marked LLM generated) and once to Rasa (“Shall I book this appointment? Branch: Riverton Station. Topic: loan. Day: friday.”, marked as the template). The booking ran only after the second yes.

What I Checked

In Inspect mode, the skill panel shows which paragraph Claude was following at that moment: the loan one, outlined because the topic in memory was loan.

The Active skill panel of the Rasa Inspector for Book an Appointment: the instructions, then three scoped paragraphs that each start with “Before you book, tell the customer to bring”, for a new account, a loan and a card, with the loan paragraph outlined because the topic in memory is loan.
The loan paragraph Claude was following when it asked its own question.

The LLM call that answered Friday, please. had that paragraph in its prompt, and its Response was Claude’s message alone, with no call to book_appointment. The tool call came one message later, after the first yes.

Rasa’s Tool Constraints (opens in a new tab) guide describes this exactly: when requires_confirmation is on and the skill’s instructions ask as well, “the customer confirms twice”. Its advice is to let Rasa do the asking, and to tell the model to call the tool as soon as the details the tool requires are in memory.

Why It Happens

  • “Before you book, tell the customer…” gave Claude a job to finish before it called the tool.
  • Claude did that job in a message of its own. In my runs, the reply that passed the reminder on never called the tool as well, so the booking had to wait for the customer’s next message, and Claude ended its message with a question.
  • When the customer agreed, Claude called book_appointment. requires_confirmation then paused the call, as it always does, and Rasa asked its own question: the second yes.

The Fix

Take the reminder out of the instructions, and put it into Rasa’s own confirmation question. A response in responses.yml can have several variants, each with a condition written like an if: line, and Rasa sends the one that matches memory.

  1. Delete the three “Before you book” paragraphs from skills/book_appointment/skill.md, so this is the last thing Claude is told about booking:
    2. When you have all three, book it with @tool.book_appointment.
  2. In skills/book_appointment/responses.yml, give the confirmation question a variant for loans, above the one you had:
skills/book_appointment/responses.yml (excerpt)YAML
  utter_ask_book_appointment_confirmation:
    - text: >-
        Shall I book this appointment? Branch: {branch}. Topic: {topic}.
        Day: {day}. Please bring your last three payslips and your bank
        statements for the last three months.
      condition: "session.book_appointment.topic == 'loan'"
    - text: >-
        Shall I book this appointment? Branch: {branch}. Topic: {topic}.
        Day: {day}.
  • condition uses the same form as if: and requires.
  • Rasa checks the variants that have a condition, in order, and sends the first whose condition is true.
  • The variant without a condition is the fallback for every other topic. Once any variant has a condition, exactly one must have none.

Train again and restart the agent:

TerminalBash
rasa train
rasa inspect

If the Reminder Must Stay in the Instructions

Rewording it so it no longer waits for the booking also removes the second question:

skills/book_appointment/skill.md (excerpt)Plain text
if: session.book_appointment.topic == 'loan'
For a loan, tell the customer to bring their last three payslips and their
bank statements for the last three months. Say it once, in your next message.

In my runs, Claude then passed the reminder on together with its next question, and Rasa asked once. But when the customer gave the branch, topic and day in one message, Claude booked straight away and the reminder was never said. For a line that must always reach the customer, use the response variant.

After the Fix

The same conversation after training again:

A conversation in the Rasa Inspector after the fix: the customer says the appointment is about a loan, the agent asks which day suits them, the customer says Friday, the agent asks “Shall I book this appointment? Branch: Riverton Station. Topic: loan. Day: friday. Please bring your last three payslips and your bank statements for the last three months.”, the customer says “Yes, please.”, and the agent confirms booking APT-3072.
One question, from Rasa, now carrying the reminder.

One question, from Rasa, with the reminder in it, and one yes before the booking ran. With all three details in one message, the same single question came back; a card booking got the plain question without the loan reminder.

Common Mistakes

MistakeDo this instead
Asking for a yes in the instructions while requires_confirmation is onLet Rasa ask: tell Claude to book as soon as it has the details.
Telling Claude to say something “before you book”Put a line that must come before the booking into the confirmation question.
Giving every variant a conditionLeave exactly one variant without a condition, as the fallback.

Troubleshooting

ProblemFix
The customer is still asked twiceLook for any other sentence in skill.md that reads the details back or asks for a yes, and remove it, as in Part 8, Step 3.
The reminder doesn’t appear in the questionCheck the value in condition against the field’s enum_values, then train again and restart rasa inspect.
rasa train stops with an error about the responseEvery variant needs its text, and only one variant may go without a condition.

FAQ

Can I let Claude ask instead of Rasa?

Yes. Turn requires_confirmation off, let the instructions ask, and gate the tool with requires on a memory field that records the yes. Rasa’s Tool Constraints (opens in a new tab) guide calls that in-flow confirmation, and says to pick one mechanism per tool, never both.

Does the new variant cost a call to Claude?

No. Rasa fills the response in from memory itself, without calling Claude; the Inspector marks it as a template.

Conclusion

The second question came from an instruction that made Claude act before the booking. Letting Rasa ask, and putting the reminder into its question with a conditional variant, leaves one question and one yes.

Additional Resources

Rasa, Claude and other names mentioned are trademarks of their respective owners. JD Bots isn’t affiliated with or endorsed by Rasa Technologies or Anthropic.

0 comments