Docs

Get started / Test and roll out

Testing Methods

Try Risotto on real requests before you roll it out: forward requests to a test channel, pick threads with ๐ŸŽซ, or replay past requests in the Response Simulator

Before you open Risotto to employees, try it on real requests. There are three ways to do it, and you can combine them:

MethodDo employees see Risotto?What you test
Forward requests to a test channelNoThe full flow: tickets, runbooks, and tools. You are the requester.
Pick which threads Risotto answersOnly on the threads you pickThe full flow, with the employee as the requester.
Replay past requests in the Response SimulatorNoKnowledge answers and routing for a batch of messages, as a CSV file.

Forward requests to a test channel

Use this method when you already have a support channel and don't want employees to see Risotto yet. Risotto stays silent in the support channel. You forward real requests to a test channel and watch how Risotto handles them there.

Set up the channels

  1. Add Risotto to your support channel

    Add Risotto to your existing support channel, for example #it-help, the same way as in Initial Slack channel setup.

  2. Check that Silent mode is on

    In the dashboard, go to Settings > Request Sources and click Manage in Slack. On the Requests tab, click the support channel's row in the Slack channels card and check that Silent mode is on. A channel starts with Silent mode on when Risotto joins it. Leave Lock to department empty unless every request in the channel belongs to one department.

  3. Create a test channel

    In Slack, create a channel for testing, for example #risotto-requests-test, and add Risotto to it.

  4. Turn off Silent mode in the test channel

    Back on the Requests tab, click the test channel's row and turn Silent mode off. Leave Continue over DM off, so that Risotto answers in the test channel.

In Silent mode, Risotto ignores new messages in the channel unless a message mentions Risotto or someone reacts to it with ๐ŸŽซ. See Channel settings.

Forward a request

  1. Forward the employee's message

    In the support channel, hover over the employee's message and click Forward message. Choose the test channel and click Forward. See Slack's Forward messages in Slack.

  2. Watch Risotto's reply

    Risotto handles the forwarded post as a new request. A Thinking... line appears, then Risotto's thinking steps in a thread, starting with the new ticket, for example "Creating ticket IT-142: Can't connect to VPN". Then Risotto posts its answer in the thread.

Risotto reads the text and the files of a forwarded message, including screenshots. You, the person who forwards the message, become the ticket's requester, not the employee who wrote it. Risotto doesn't post in the support channel or message the employee.

Pick which threads Risotto answers

Use this method to let Risotto answer only the requests you choose. The support channel stays in Silent mode, and you react with ๐ŸŽซ to each request you want Risotto to take. Risotto answers the employee in the request's thread, so the employee sees the answer.

  1. Check that Silent mode is on

    In the dashboard, go to Settings > Request Sources and click Manage in Slack. On the Requests tab, click the request channel's row and check that Silent mode is on.

  2. React with ๐ŸŽซ to the request

    In Slack, add the ๐ŸŽซ (:ticket:) reaction to the employee's message: the message that starts the request. Risotto opens a ticket, shows its thinking steps in a thread on that message, starting with "Creating ticket IT-143: โ€ฆ", and then replies to the employee in the thread.

Anyone in the channel can use ๐ŸŽซ. Risotto ignores it on a thread that already has a ticket. Messages without the reaction get no reply, unless they mention Risotto.

Replay past requests in the Response Simulator

Use the Response Simulator to see how Risotto answers a batch of past requests, for example tickets that you export from your current help desk. There is no file upload: you post the requests in a Slack channel, and the Response Simulator reads the channel's most recent messages.

For each message, the Response Simulator writes the answer that Risotto finds in your indexed knowledge, the same answer as in the Knowledge Playground, and the department and category Risotto would file it under. It does not follow runbooks, run tools, start access requests, create tickets, or post in Slack. Only Org Admins and Org Managers can open it.

  1. Create a test channel

    In Slack, create a private channel, for example #risotto-simulator-test, and invite Risotto to it. Keep Silent mode on for the channel, as it is when Risotto joins, so that Risotto does not open tickets from your test messages.

  2. Post the requests

    Post each request as its own message in the channel, with the subject and the description in the same message. Type or paste the text, and remove personal data first. Don't mention Risotto: a message that mentions Risotto opens a ticket, even in Silent mode.

  3. Open the Response Simulator

    In the dashboard, open Analytics and click Other reports at the bottom of the page. The Reports page opens. The Response Simulator is below the Year in Review.

  4. Run the simulation

    In Slack Channel, search for the test channel by name, or paste its channel ID. In Number of Messages, pick how many of the channel's most recent messages to read: 10, 25, 50, 100, 250, or 500. Click Run Simulation. While it runs, the page shows how many messages it has processed.

  5. Download the results

    When Simulation complete shows, click Download CSV. The download link works for 1 hour. To start a new run, click Run Another.

The Response Simulator reads only the text of top-level messages. It skips thread replies and messages without text, so a forwarded message or a file with no text of its own is left out. Post text only: files and screenshots aren't read.

The CSV file has one row for each message, with these columns:

  • User: the person who posted the message.
  • Slack Message: the message text.
  • Playground Response: Risotto's answer from your indexed knowledge. Empty when Risotto finds no answer.
  • Department: the department Risotto routes the request to. When no department matches, your default department.
  • Category: the category Risotto picks from that department's categories.
  • Tool: the app or software the request is about, in lowercase, such as zoom. Empty when the request names none.
  • Slack Permalink: a link to the message in Slack.

A message is left out of the file when Risotto hits an error while answering it. If a run fails, the page shows why, for example "The Risotto bot is not in this channel. Please invite the bot to the channel and try again." or "No valid messages found in the channel.", and a Try Again button.

To test the full flow with runbooks and tools instead, use one of the methods above. Each message then becomes a real ticket, and the ticket syncs to your ticketing system if the department has ticket sync on.

Something wrong or missing? Report an issue ยท View as Markdown