Motivational Script Editor
In 2015, I co-founded the company Tinybots. As the lead designer I developed Tessa; a small social robot that supports people with cognitive disabilities and their families. Tessa is a small, talking robot that speaks friendly reminders, which family members can write and schedule via an easy-to-use app.
Initially, Tessa could only give reminders. Based on extensive user testing and feedback, Tessa needed to be interactive to provide better, and more context-aware support.
The key challenge was to design interactivity that was predictable. Routine and structure is important for people with dementia. Families wanted to make sure Tessa's voice was curated—what Tessa says should be up to them.
Initially, Tessa's could only give reminders (one-way communication)
Tessa's interactive scripts used speech recognition to detect yes and no answers. By using simple, closed questions, families could create branched conversations. The families were fully in control of what Tessa was saying, and when.
Several different low-fidelity concepts were made and tested with users to design a simple, but flexible way to create conversational scripts.
Some early concepts how to create and plan conversational scripts
To make sure all the different states, edge cases, and possible scenarios were covered, each individual flow was mapped. Not only what the app does, but also what the robot says and does, and what the robot communicates back to the app.
Flow for creating a new script: user actions (blue), system actions (green), UI screens (red), details obscured on purpose
The final result was a script editor that allows users to write different scenarios based on the answer Tessa recognized. If Tessa recognized "yes" after asking a question, Tessa would execute the yes-branch. If Tessa recognized a "no", then the no-branch is executed. The other-branch was only used when Tessa did not recognize a specific yes or no.
Tessa's script editor
The interface was designed to have simple design based on speech balloons. Each speech balloon in the script was one step in the script. Each step is a Say-action, an Ask-action, or a Wait-action, which the families could use to combine into their own conversational scripts.
Scripts can be reused, modified, and planned like regular reminders. After the user has set the time and date, Tessa will initiate that script at that moment. For example, you could write a script that asks you to check if you locked your front every night.
Tessa can also play music via the script feature
Collaborators:
Arno Nederlof
Erik Hoogeveen
Wang Long Li