CASE STUDY
Carnegie Mellon University
Chef UX + AI
Conversational AI for culinary experiences with Blue Apron.
OVERVIEW
Our challenge was to design a virtual assistant for an existing brand that would benefit from emerging AI solutions.
As the use of AI continues to grow, it offers efficient solutions to problems facing people and businesses, as well as quick ways to access relevant information.
Our team researched an existing brand that would benefit from AI and emerging technology while developing a digital AI system across a multi-platform experience through form, motion, style, and conversation.
Team
Anna Boyle
Amrita Khoshoo
Deepika Dixit
Yiwei Huang
Time
2019
7 Weeks
Carnegie Mellon University
Role
Concept Development, Project Management, Design Research, Visual Design, UI/UX Prototyping
Tools
Adobe Illustrator, Adobe XD, After Effects
CORE77 DESIGN AWARDS
Recognized by the Core77 Design Awards as a notable Student Honoree and Community Choice Prize Winner in Consumer Technology.
WHY DESIGN AI FOR BLUE APRON?
Cooking is a task-based communal activity that usually requires both hands, and oftentimes a piece of technology.
Therefore there was an opportunity for a virtual assistant to anticipate and aid with tasks while guiding multiple users through the cooking experience.
OUTCOME
Chef - Blue Apron’s conversational AI for culinary experiences.
Accessible on Blue Apron’s mobile app and Google Nest Hub Max, smart home display.
Chef’s capabilities in the kitchen include: guiding individuals and multiple users through the Blue Apron cooking experience, providing ingredient substitutions in real-time, keeping track of multiple timers, and integrating with third-party applications like MyFitnessPal, Spotify, and YouTube.
1.0
Collaborative Multiuser Cooking
Chef guides multiple users through Cooking and delegating tasks based on skill level.
2.0
Personalized Dietary Options
Chef makes personalize meal recommendations based on user preferences and third-party data.
3.0
Smart Control
Chef connects to smart kitchen appliances for a streamlined cooking experience.
4.0
Food Recognition
Chef uses food recognition to log meals and make recipe substitutions.
5.0
Tracking Food Intake
Chef customizes future meal recommendations based on personal taste.
1.0 | PROCESS
Exploratory Research
First, we researched AI and virtual assistants in the market while also evaluating established brands that could benefit from having a virtual assistant as a part of their experience in the future.
After looking into a variety of existing brands and services, we landed on Blue Apron - a meal kit service that delivers personal meals to your door composed of locally sourced ingredients alongside recipe guidelines.
BRAND ANALYSIS
After choosing Blue Apron, we explored the brand in terms of visual brand language, the target audience, and the tone of the virtual assistant.
Visual Brand Language
The Blue Apron brand exhibited a strong, bold, and dynamic style. Its distinct illustration style was clean, friendly, and personalized.
Accessible, wholesome, and fun while being reassuring, and calm.
Target Audience
The largest proportion of Blue Apron’s user base is American women between the ages of 25–34. 82% of the account holders are female-sharing meals with partners.
To appeal to this specific demographic, we imagined our virtual assistant to be a male voice.
The Tone
Our team wanted a virtual assistant to provide helpful guidance throughout the cooking process and for it to embody a sense of calm to counter anxiety often related to cooking.
An upbeat and optimistic companion, reassuring the user along the way
THE CURRENT USER EXPERIENCE
In addition to the analysis of the Blue Apron brand we also conducted informal research on the current customer experience. We had informal conversations with friends who had used the service and also researched reviews online and on YouTube.
“There was a lot of waste involved. And it doesn't eliminate the fact that cooking is still hard. There's still a lot of prep work, multitasking, patience...”
USER JOURNEY MAPPING
After our analysis of the brand and current user experience, we outlined the user journey to understand the process of cooking using the Blue Apron meal kit through one of their recipes.
Through this exercise, we identified tasks and grouped them as before, during, and after the process of meal preparation and then associated user emotions to each step based on the research we had conducted on Blue Apron users.
After identifying the user’s emotions we mapped the tone of the AI in response to the user to gain a better understanding of the interaction between the user and this future device.
VISUAL FORM
When defining Chef’s visual form, we abstracted some of the illustrations of Blue Apron’s cooking utensils and landed on the visual form of a bowl and ingredient.
Our virtual assistant Chef is composed of three elements: a fill bowl shape, an outline bowl shape, and a circular shape representing an abstract ingredient. We choose the colors based on Blue Apron’s primary and secondary color schemes.
MOTION AND EXPRESSION
The form proved to be simple and flexible to modify across all states of motion. Through our process of refining the motion we referred to the AI response states we outlined in our research and user flow map to help guide our iterations and design decisions.
As a team we worked together to determine the transitions and motions for Chef while adhering to the brand language and tone of Blue Apron.
Google Assistant
Microsoft Cortana
Base/ Ideal
Thinking/ No Results
Speaking
Timer On
Listening
Timer Done
Thinking
Meal Complete
2.0 | PROCESS
Evaluative Research
Following our exploratory research and the development of Chef’s visual form, we tested our virtual assistant conversational design in analog to evaluate how Chef could assist user needs in the kitchen and adapt to different environments in the kitchen.
BLUE APRON TEST KITCHEN
In this exercise, I was the solo cook and Amrita verbalized Chef’s actions. We had three main takeaways from this experience.
1.0
Updating Conversational Design
Measuring ingredients became confusing with just numerical descriptions and Anna ended up pouring too much olive oil and pasta sauce into a pan.
This influenced how we approached our conversational design, by making conversation informative and relatable.
2.0
Personalized Appliance Settings
The oven preheated faster than expected which ended up cutting the cooking time of our pizza in half.
This led us to provide more personalized appliance settings. In our scenario, Chef is connected to a smart device and Chef knows the presets of the device.
3.0
Adding Visual Indicators
There was also confusion about what the pizza should look like and taste like when it’s cooked well.
There was also confusion about what the pizza should look like and taste like when it’s cooked well.
Design
Mobile UX/UI
Real time meal substitutions and recommendations
Remembering food preferences for friends or family
Setting multiple timers
Our initial Mobile UI design walks a single user through the Blue Apron cooking experience. Our first iteration of the UI was visually influenced by the current Blue Apron App and included the following additional functionality.
Sampling of our initial UI below.
Research | Part 2
It enables gesture control so you don’t have to touch a screen while cooking.
It has a face and voice recognition feature that recognize individual users and personalizes their experience.
It has a subtle presence when prompting “Hey Google”, and the text shows up on the top of the screen. This AI styling is something we referenced when designing our own application of Chef in Google Nest Hub Max.
After designing the visual form and interactions of Chef for mobile we moved forward with Part 2 of this project where we were tasked with scaling our mobile design to another platform.
After researching a number of different platforms we decided to move forward with Google Nest Hub Max because of its following functionality.
It has Google Assistant built-in.
As a platform, it is useful in communal spaces and helps you undertake daily tasks by connecting with multiple devices, and third-party applications.
User Testing
Updating conversational design
Providing personalized appliance settings
Adding visual indications to the cooking experience
Before we moved forward with the design for Google Nest Hub Max we decided to test our mobile experience in the kitchen. During this research experiment we identified three design considerations that helped influence updates to our mobile interface and Google Nest Hub Max.
Design
Modifying Mobile UI/UX
After conducting our user research in the our test kitchen scenario we identified opportunities to enhance the anatomy of the conversation between Chef and the user through the following updates to the UI below.
Personalized Meal Recommendations
Connection to Smart Kitchen Appliances
Google Nest Hub Max UI/UX
GOOGLE NEST HUB MAX DESIGN CONSIDERATIONS
Visual cooking timeline
Voice and face recognition
Gesture control for a hands free experience
Integration and references to third party apps like, Spotify, Pandora, Netflix, MyFitness Pal, FitBit and YouTube.
When designing the UI for the Google Nest Hub Max we worked at developing a design that matched the look and feel of our mobile experience but also considered the design of Google Nest Hub Max and the Google Assistant integration. To insure that the application of our design was accurate we purchased a Google Nest Hub Max to help us identify and implement existing macro and micro interactions.
Food Recognition
Customized Recommendations
Multi-User Journey
Reflection
Value Sensitive Design
Value sensitive design, or thinking about how human values factors into what we’re doing, influenced our decision process while defining the autonomy of our virtual assistant.
Thinking through how a Chef takes action and communicates what he was doing was also considered.
We had good discussions and made some key decisions around user consent and control.
Next Steps
In the future we would welcome the opportunity to further test our ecosystem and our virtual assistant Chef.
Designing Conversations
We learned about creating user-centered conversations both in terms of the user interface and the user interactions.
With user interface design, less is always more. An intuitive and clean user interface is better.
We were very inspired by Google’s AI Design Guidelines and tried to use their principles to guide our design. This included using short, simple, concise words and prompting users with task-oriented responses.
Prototyping & Testing
Prototyping and testing are fundamental! The insights from our test kitchen experience played a larger role in Part II of our design.
