Reading time : 3 min
MyAI Builder empowers users to create customized AI solutions using uploaded data and configurable settings. Despite its innovative features, early users encountered significant usability challenges.
This research study aimed to identify key personas, uncover frustrations, refine critical workflows, and enhance user satisfaction across diverse user groups
Industry
Edtech, Artificial Intelligence
Team
2 UX Researchers
1 Product Manager
Timeline
1.5 months
Tools
Userlytics
Lookback
FigJam, Miro
ChatGPT
User interviews (participant recruiting, note-taking, interviewing)
Usability testing (scheduling, moderating)
Persona creation
Presenting findings (analyzing findings, audience appropriate presentations)
My research uncovered a key non-technical user persona, prompting a shift toward simplified workflows and intuitive design and it directly influenced the product roadmap—prioritizing features that aligned with actual user expectations and behaviors.
60%
Change in information architecture, reducing number of clicks required
20%
Unwanted features removed, reducing clutter
3
New features designed, reducing number of queries
Streamlining the goal
WHY?
Number of users having access to the application is greater than Number of projects created.
WHERE?
The two major areas critical for creating a new project is the
New project user flow
Training the bot to create a successful project, which happens in the settings panel on the right side of the interface
I highlighted the need of research and presented a research proposal to the leadership team, which was accepted.
Learning : Vague goals like "finding user pain points" aren't enough. Defining what to test—whether it’s a specific user flow, a section of the interface, or UX writing—makes research more targeted and actionable.
Methodology
Step 1:
Leveraging existing data
COLLABORATED WITH DEVELOPERS
I collaborated with developers to access backend data and dashboards highlighting user activity levels. This helped identify users to interview across three groups: frequent users, occasional users, and those with access who haven’t used the application yet.
DATA FROM PAST RESEARCH + CUSTOMER QUERIES
I reviewed past research data, gathered frequent queries from customer care, and monitored Slack channels for insights
Gathering this data guided strategic participant recruitment for usability testing and interviews, informed tailored research materials, revealed patterns, and refined research goals
My efforts in reached out to cross-functional departments was much appreciated by the manager.
Learning : Collaborating with other teams and exploring existing data can enrich research and save time. Research isn't always formal - observing users or noting offhand comments can uncover valuable insights.
WHY?
Step 2:
User Interviews - 9
INTERVIEW GOALS
Understanding overall user sentiment
Identifying persona
Existing onboarding process pain points
Use cases
Gains/Wins
RECRUITMENT AND RESEARCH MATERIAL
Participants were recruited based on their frequency of usage. The findings gathered through user interviews helped in preparing for the usability testing material.
PROCESS
I conducted 9 user interviews via Zoom, guided by clearly defined interview goals. Key observations and insights were documented in an Excel sheet during the sessions
Using these notes, I synthesized an empathy map to capture user thoughts, feelings, and behaviors, providing a foundation for deeper analysis
PARTICIPANT DATA
Users who have created 2+ projects - 3
Users who have created 1 or 2 projects - 3
Users who have 0 projects - 1
I was a note taker for 4 interviews and interviewer for other 5. I recruited participants and scheduled timings based on their availability.
Learning : Resources aren’t just about budgets or tools—they include the participants willing to share their time and insights. Maximizing the value of these interactions is crucial for meaningful outcomes.
WHY?
Interview findings
User sentiment
Users feel empowered and are excited to use MyAI Builder in their daily life.
“ I do see potential for My AI builder and the need"
Persona
Personas identified:
I am getting started
AI Builder
Advanced User
70% of user base is of AI Builder and I am getting started persona. (Non - technical)
Onboarding process
9/9 participants appreciated the help provided by Paul - a team member. This highlights the need of creating a robust help support in the application to improve efficiency.
Use cases
Creating course assets
Literature review
Report creation
Cost transparency
Users want to know how the 'paid' model will look like in the future as currently they are skeptical of investing their time in a tool that they might not be able to afford later on.
ChatGPT transfer
9/9 users were part of a challenge where they got ChatGPT license and now are moving towards MyAI Builder as their license is ending.
Step 2:
Usability testing - 8
TESTING GOALS
Pain points related to User Interface
User flows
What features do they use while training the bot
Identifying task specific pain points
ZOOM AND ONLINE USABILITY TESTING TOOLS
Two kinds of usability testing were conducted.
3 usability testing through Zoom
5 user interviews using online tools such as Userlytics and Lookback as I was also testing which online tool is better for future usability testing.
PROCESS
Predetermined tasks were formed catering to each testing goal
Usability testing was conducted, each session was recorded and notes were taken in an excel file.
Notes from the excel file was imported to a FigJam file, which was created to visualize and find patterns. I segregated the pain points/wins and observations based on the tasks
Task completion time and rate were noted
Frequency of each finding was also noted to prioritize recommendations and understand the impact (positive, negative, neutral) each finding is creating
PARTICIPANT DATA
The persona identified through Interviews and previous research were used to recruit participants.
2 - AI Builder persona
1 - Advanced user persona
5 - I am getting started persona
Interview findings
Task success and Completion rates
Task Success: 8/8 participants successfully completed the "Create a New Project" task.
Beginner Struggles: The "I am Getting Started" persona struggled with project configuration tasks.
Efficiency Issues: Advanced personas completed configuration tasks, but with longer-than-expected completion times.
Confused Terminologies
Technical terms: Participants found terms like "In GPT Interactions" and the distinction between "Project Description" and "System Prompt" unclear.
Ineffective Tooltips: Tooltips were unhelpful, particularly for 'AI Builder' and 'I am Getting Started' personas.
Cluttered space
Overwhelming Panels: Participants found the panels cluttered and visually overwhelming.
Missed Actions: 8/8 participants overlooked the 'Update Settings' button due to poor visibility.
Unnoticed Features: 40% of features were missed, with unnecessary UX copy contributing to visual noise.
Information architecture
Logical Grouping: 8/8 participants expected features like sharing settings to be grouped with related interface options.
Workflow Misalignment: Features were not organized around participants’ workflows, creating friction during onboarding and task completion.
Navigation Challenges: Participants struggled with unclear labels and an unintuitive dashboard structure.
Project Trouble shooting style
Conversational Troubleshooting: All 9/9 participants opted for a conversational approach to troubleshoot issues instead of using system settings like verbosity controls.
Natural Language Interaction: 'I am getting started' persona who were using this application for the first time asked the application itself to help them find a feature
Lack of system feedback
Feedback Deficiency: Participants expect progress bars during file uploads to estimate the time required.
Guided feedback: Participants noted a lack of help by the system, guiding them on their next steps. (For eg. the system didn't have a progress bar when uploading a file)
Step 2:
Usability testing - 8
TESTING GOALS
Pain points related to User Interface
User flows
What features do they use while training the bot
Identifying task specific pain points
ZOOM AND ONLINE USABILITY TESTING TOOLS
Two kinds of usability testing were conducted.
3 usability testing through Zoom
5 user interviews using online tools such as Userlytics and Lookback as I was also testing which online tool is better for future usability testing.
PROCESS
Predetermined tasks were formed catering to each testing goal
Usability testing was conducted, each session was recorded and notes were taken in an excel file.
Notes from the excel file was imported to a FigJam file, which was created to visualize and find patterns. I segregated the pain points/wins and observations based on the tasks
Task completion time and rate were noted
Frequency of each finding was also noted to prioritize recommendations and understand the impact (positive, negative, neutral) each finding is creating
PARTICIPANT DATA
The persona identified through Interviews and previous research were used to recruit participants.
2 - AI Builder persona
1 - Advanced user persona
5 - I am getting started persona
Recommendation
OVERALL RESEARCH STATEMENT
Users are excited about MyAI Builder's potential and eager to integrate it into their workflows, but they often feel overwhelmed by its extensive feature set, using only 40% of its capabilities. Simplifying the user experience and prioritizing the visibility of key features could drive higher adoption and better engagement.
COMMUNICATING FINDINGS
I created presentations of research findings related to business goals to the leadership team and detailed UX findings to the design and developer teams. Post the presentation I created a final recommendation report, which was sent to the design team.
As a designer on the team, I started a Design + Developer meetings where we discussed design team's plans addressing the research findings to understand each team's priorities and feasibility.
Learning : If findings aren't used, revisit communication - present clear insights aligned with business goals, mindful of each department's efforts.
Core design improvements suggested
HIGH LEVEL GOALS
Simplify Navigation and Interface Design
ACTIONS TAKEN
The design team is worked on the information architecture and changed 60% of it, cleaning the interface, catering to recently identified personas.
I conducted a card sorting activity with the design team, helping align team and stakeholders on category structures.
IMPACT
40%
reduction in steps needed to complete a task
37%
decrease in task completion time
Improve Terminologies and Use Natural language/ Conversational help
I developed a microcopy including tool tips, error messages, system feedback messages across the product
60%
reduction in user queries
An AI bot developed to help users that recorded 120 conversations in a day.
Improve Onboarding and Guidance
Slack channel is created and additional weekly customer help meetings are scheduled.
I highlighted the need to create a community of AI enthusiasts that would encourage word of mouth marketing and inspire others to experiment.
Weekly at least 5 new customers are added to the slack channel or weekly community meeting
Improve in-product onboarding
The design team created a re-design of the interface and designed 3 features for helping the non-technical users
Initial user feedback found the interface to be intuitive, clear, less cluttered and less technical.
Challenges
Technical terms needed to be understood before conducting research.
Collecting information across departments as there are individual contributors who spoke to customers directly.
Recruiting participants from the 'Advanced users' persona was difficult as there weren't many.
Budget constraints in using online testing tools so I was experimenting using the free tools available.
Conducting research in an agile environment where features change frequently and keeping research relevant and up-to-date with evolving products
Appreciation from Leadership
Recognized for driving impactful research insights that shaped design decisions. One screenshot captures a Zoom meeting where the design team presented ideas influenced by my research, and the other is a Slack message from my manager appreciating my presentation.
