🏆 CORE77 Design Awardee
Recognized by Core77 Design Awards as a Community Choice Prize Winner and Student Honoree in Consumer Technology.
CASE STUDY
Carnegie Mellon University
Chef - Conversational AI for culinary experiences with Blue Apron.
Team
Anna Boyle, Amrita Khoshoo, Deepika Dixit, Yiwei Huang
Role
Concept Development, Project Management, Design Research, Visual Design, UX/UI Prototyping
Tools
Adobe Illustrator, Adobe XD, After Effects
Time
2019
7 Weeks
Carnegie Mellon University
OVERVIEW
Our challenge was to design a virtual assistant for an existing brand that would benefit from emerging AI solutions.
Our team researched and tested what services would benefit from AI and emerging technology and designed a digital AI system across multi-platforms through form, motion, style, and conversation.
Cooking is a task-based communal activity that usually requires both hands, and oftentimes a piece of technology.
WHY DESIGN AI FOR BLUE APRON?
There was an opportunity for a virtual assistant to anticipate and aid with tasks while guiding multiple users through the cooking experience.
Chef - is accessible on Blue Apron’s mobile app and Google Nest Hub Max smart home display.
OUTCOME
Chef guides individuals and multiple users through the cooking experience, providing ingredient substitutions in real-time, keeping track of multiple timers, and integrating with third-party applications like MyFitnessPal, Spotify, and YouTube.
🍽
Feature Set
Chef is composed of 5 key features including, collaborative multiuser cooking, personalized dietary options, smart controls, food recognition, and tracking food intake.
1.0
Personalized Dietary Options
Chef makes personalized meal recommendations based on user preferences and third-party data.
2.0
Smart Controls
Chef connects to smart kitchen appliances for a streamlined cooking experience.
3.0
Collaborative Multiuser Cooking
Chef guides multiple users through Cooking and delegating tasks based on skill level.
4.0
Food Recognition
Chef uses food recognition to log meals and make recipe substitutions.
5.0
Tracking Food Intake
Chef customizes future meal recommendations based on personal taste.
1.0. | PROCESS
Exploratory Research
After choosing Blue Apron, we explored the brand in terms of visual brand language, the target audience, and the tone of the virtual assistant.
BRAND ANALYSIS
Visual Brand Language
The Blue Apron brand exhibited a strong, bold, and dynamic style. Its distinct illustration style was clean, friendly, and personalized.
Accessible, wholesome, and fun while being reassuring, and calm.
Target Audience
The largest proportion of Blue Apron’s user base is American women between the ages of 25–34.
We imagined our virtual assistant to be a male voice.
The Tone
We wanted the virtual assistant to provide helpful guidance throughout the cooking process and for it to embody a sense of calm to counter anxiety often related to cooking.
An upbeat and optimistic companion, reassuring the user along the way
CURRENT USER EXPERIENCE
We conducted informal research on the current customer experience and had informal conversations with friends who had used the service and researched reviews online and on YouTube.
“There was a lot of waste involved. And it doesn't eliminate the fact that cooking is still hard. There's still a lot of prep work, multitasking, patience...”
USER JOURNEY MAPPING
We outlined the user journey to understand the process of cooking using the Blue Apron meal kit through one of their recipes.
Tasks were identified; before, during, and after the process of meal preparation and then associated user emotions with each step based on the research we had conducted on Blue Apron users. We mapped the tone of the AI in response to the user to gain a better understanding of the interaction between the user and the technology.
VISUAL FORM
When defining the visual form, we abstracted some of the illustrations of Blue Apron’s cooking utensils and landed on the visual form of a bowl and ingredient.
Chef is composed of three elements: a fill bowl shape, an outline bowl shape, and a circular shape representing an abstract ingredient. We choose the colors based on Blue Apron’s primary and secondary color schemes.
MOTION AND EXPRESSION
Through our process of refining the motion we referred to the AI response states we outlined in our research and user flow map to help guide our iterations and design decisions.
As a team, we worked together to determine the transitions and motions for Chef while adhering to the brand language and tone of Blue Apron.
Base/ Ideal
Thinking/ No Results
Speaking
Timer On
Listening
Timer Done
Thinking
Meal Complete
2.0 | PROCESS
Evaluative Research
Following our exploratory research and the development of the visual form, we tested our conversational design to evaluate how Chef could assist user needs and adapt to different environments in the kitchen.
USER TEST KITCHEN
In this exercise, I was the solo cook, and Amrita verbalized Chef’s actions. During this research experiment, we identified three design considerations that helped influence updates to our mobile interface and Google Nest Hub Max.
These updates included updating conversational design, providing personalized appliance settings, adding visual indications to the cooking experience
1.0
Updating Conversational Design
Measuring ingredients became confusing with just numerical descriptions and Anna ended up pouring too much olive oil and pasta sauce into a pan.
This influenced how we approached our conversational design, by making conversation informative and relatable.
“Cut 1 inch cubes”
🔴 Difficult for the user’s cognitive conception
“Cut 1 inch cubes, which is the width of your thumb”
🟢 Helpful for the users to evaluate.
2.0
Personalized Appliance Settings
The oven preheated faster than expected which ended up cutting the cooking time of our pizza in half.
This led us to provide more personalized appliance settings. In our scenario, Chef is connected to a smart device and Chef knows the presets of the device.
3.0
Adding Visual Indicators
The oven preheated faster than expected which ended up cutting the cooking time of our pizza in half.
This led us to provide more personalized appliance settings. In our scenario, Chef is connected to a smart device and Chef knows the presets of the device.
UX/UI | Mobile
Real time meal substitutions and recommendations
Remembering food preferences for friends or family
Setting multiple timers
Our initial Mobile UI design walks a single user through the Blue Apron cooking experience. Our first iteration of the UI was visually influenced by the current Blue Apron App and included the following additional functionality.
Sampling of our initial UI below.
Modifying Mobile UI/UX
After conducting our user research in the our test kitchen scenario we identified opportunities to enhance the anatomy of the conversation between Chef and the user through the following updates to the UI below.
Google Nest Hub Max UI/UX
GOOGLE NEST HUB MAX DESIGN CONSIDERATIONS
Visual cooking timeline
Voice and face recognition
Gesture control for a hands free experience
Integration and references to third party apps like, Spotify, Pandora, Netflix, MyFitness Pal, FitBit and YouTube.
When designing the UI for the Google Nest Hub Max we worked at developing a design that matched the look and feel of our mobile experience but also considered the design of Google Nest Hub Max and the Google Assistant integration. To insure that the application of our design was accurate we purchased a Google Nest Hub Max to help us identify and implement existing macro and micro interactions.
UX/UI | Google Nest Hub Max
It enables gesture control so you don’t have to touch a screen while cooking.
It has a face and voice recognition feature that recognize individual users and personalizes their experience.
It has a subtle presence when prompting “Hey Google”, and the text shows up on the top of the screen. This AI styling is something we referenced when designing our own application of Chef in Google Nest Hub Max.
After designing the visual form and interactions of Chef for mobile we moved forward with Part 2 of this project where we were tasked with scaling our mobile design to another platform.
After researching a number of different platforms we decided to move forward with Google Nest Hub Max because of its following functionality.
It has Google Assistant built-in.
As a platform, it is useful in communal spaces and helps you undertake daily tasks by connecting with multiple devices, and third-party applications.
MULTI-USER JOURNEY
REFLECTION
Designing Conversations
How to create user-centered conversations both in terms of the user interface and user interactions.
With user interface design, less is always more. An intuitive and clean user interface is better.
Inspired by Google’s AI Design Guidelines we tried to use their principles to guide our work. This included using short, simple, concise words and prompting users with task-oriented responses.
Prototyping & Testing
Prototyping and testing are fundamental! The insights from our test kitchen experience played a larger role in our design.
Value Sensitive Design
Value-sensitive design, or thinking about how human values factor into what we’re doing, influenced our decision process while defining the autonomy of our virtual assistant.
Thinking through how a Chef takes action and communicates what he was doing was also considered.
We had good discussions and made some key decisions around user consent and control.
Next Steps
In the future, we would welcome the opportunity to further test our ecosystem and conversational AI.
