Uncovered new product strategy for Searchable.ai with a diary study
Overview
Problem
The engineering and ML team frequently brought up how difficult it was to understand how well their machine learning models were working without qualitative data about users' experiences over an extended amount of time.
I proposed conducting a diary study to better understand how users' perceptions and attitudes changed about the product over time.
Outcome
Listening to user feedback, we identified an opportunity, a new feature we went on to develop called Collections.
Feedback was translated into feature requests and 15+ user stories were created.
Based on interviews, I created user personas to help better identify our target user.
Context
Info
Searchable.ai
Desktop/Web App
A productivity tool that allowed you to search across your cloud storage, web apps, and local drive in one place.
Role
UX Researcher
Team
Me, Evan Lin, Hung Nguyen, and Megan Tickle.
Timeline
2 weeks
How and why?
I listened closely to my team's challenges and heard a research opportunity.
Based on what I was hearing from team members' complaints, I knew that a diary study may provide some clarity they were looking for.
I proposed conducting a diary study to better understand how users' perceptions and attitudes changed about the product over time.
Objectives
I conducted a diary study to better understand:
How do users' currently go about searching for their stuff across different platforms?
How do users’ feelings and perceptions about Searchable.ai change over time?
What unexpected points of friction or delight do users experience while using Searchable.ai?
Methods & Process
Mapping out a research plan.
To set our study up for success, I worked with my team to determine the following:
-
Determining the aim of the study
-
Setting the budget
-
Deciding which tools to use
-
Defining the participants
Defining, Creating, Recruiting
Define Questions, Scope, and Ideal Participant
-
Users will submit diary entries about their daily app experience, using signal-contingent protocol.
-
Study will run for 5 workdays.
-
5 Ideal participants who worked in marketing, product, sales, and/or CS for a small to medium-sized startup.
Create Screener, Resources, and Prompts
-
I created a robust screener questions and recruited participants with UserInterviews.com.
-
I also created a "cheat sheet" for participants to use as reference throughout the study.
-
I then designed daily prompts for participants to fill out using Typeform.
Recruit and Onboard Participants
-
I vetted and selected participants who fit the target user profile.
-
I then scheduled an intro onboarding call with each participant. where they were walked through the app, shown how to log their journal entries, and provided additional information.
During the Study
Thinking ahead with scheduling.
For everyday of the study, participants received individual emails with a link to their Typeform, as well as a reminder email at the end of the day to fill out the survey, if they had not already submitted an entry.
If any participants reached out with questions or concerns, I made sure to respond in a timely manner.
Evaluating and Analyzing.
To stay on top data analysis, I analyzed data submissions as closely to real-time as possible.
I highlighted any texts with certain labels, such as "Problem", "Question", "Behavior", "Feature Request", and "Positive/Negative."
I was also sure to review entries multiple times to filter out for the most interesting and helpful info for our team.
After the Study
Following up with interviewing.
After the 5 days of entries were submitted, I scheduled follow-up interviews with each participant to:
-
ask follow-up questions regarding their entries.
-
learn any helpful additional context
-
receive participation feedback. regarding the actual study, for future reference.
Results
Sharing insights with the team.
After evaluating and analyzing, I synthesized the most important information for the Product, Engineering and Marketing team for an all-hands meeting, where we discussed next steps, which involved a plan to conduct further market research regarding target audience and market fit.
I also created, using data collected from the in-depth interviews and diary study, pictured below.
It captured valuable information about a super participant/user to help our team foster a deeper understanding and empathy for the people we were building the product for.
Uncovering new opportunities.
Listening to user feedback and by analyzing the biggest issues users ran into, we identified one of our greatest opportunities, a new feature we went on to develop called Collections.
I went on to translate that feedback into feature requests and 15+ user stories were created.
What I learned
Before conducting this study, we needed to better nail down, know, and understand who the target audience was. I realized during the study that selected participants may not have been testing an app that was built for their needs.
I would use a different app than Typeform to collect participants' diary entries. In an ideal world, I could use a video recording and transcription service to collect data.
We were only able to run the study for 5 days due to budgeting costs. But I would have liked to have ran the study for longer (at least a month) as 5 days was not long enough to gain significant insight into a user's sustained behavior.
Other Projects
