CircleCI launched support for GitLab — its first decoupled experience — in Spring 2022, going to GA in July. Teams using GitLab SaaS could now build, test, and deploy on CircleCI. New users needed a simple path onto the platform to start delivering software quickly. Sign-up completions had plateaued well below target, and no qualitative research had been conducted on the new experience. I chose to take on this high-visibility research initiative as my UXR internship project.
CircleCI onboarding funnel
The problem
Data without answers
Funnel analytics showed users dropping out at specific steps, but the numbers couldn't explain why. The product team had competing hypotheses — was the friction technical, conceptual, or a copy problem? What nobody had done was actually observe users moving through the flow in real time.
My approach
A focused research sprint
I originally planned an unmoderated usability study using UserZoom — recording participants moving through the onboarding flow, capturing reactions, time on task, and task success rates. Due to the complexity of the tasks and technical constraints, that approach failed to yield the insights we needed.
Within a few days I pivoted to moderated usability sessions, using the team's Rolling Research interview guide as a template and tailoring it to the onboarding funnel specifically. I sourced six participants through User Interviews — ranging from new software developers at small companies to principal engineers at global enterprises. I also participated in the broader UX team's Rolling Research program, observing and assisting with interview operations alongside the usability work.
What I was trying to understand:
What users expected when onboarding a complex developer tool
Where they lost confidence or momentum in the flow
How they interpreted the terminology used in the UI — was it intuitive, too abstract, or misaligned with their mental models?
Where they got stuck, how they looked for help, and at what point they gave up
Where the moments of delight were — and what users said vs. what they actually did
6
Moderated usability sessions
99%
Could not complete GitLab authorization
100%
Needed documentation for basic setup
What I found
Three problems hiding in the data
The sessions were, as the Growth Team Staff PM put it, "treasures" — chock full of actionable insights.
The most striking finding was the authorization failure rate. 99% of participants could not complete authorization with GitLab through either OAuth or personal access token. This wasn't an edge case — it was nearly universal, and it was invisible in aggregate metrics.
The second major finding was conceptual. Only two participants — regardless of engineering experience level — were confident they understood what YAML was, a critical element of CircleCI setup. Users were clicking through configuration steps without a mental model for what the platform was actually doing. When something went wrong, they had no framework for troubleshooting.
Third, 100% of participants needed to consult product documentation to complete the most fundamental setup steps. The in-product copy assumed prior knowledge throughout. For someone encountering the platform for the first time, it was technically accurate but practically opaque.
One nuance worth noting: while 99% of participants rated the sign-up process itself as easy or neutral, observation told a different story. Non-verbal cues — sighs, facial expressions, visible frustration — revealed friction that participants weren't always voicing directly. The gap between what users said and what they did was itself a key finding.
PAT complexity — usability session findings
What changed
Research into product decisions
By combining analytics with direct observation, the team could now tie specific drop-off points to specific causes. PMs/Eng prioritized moving to OAuth only, dramatically simplifying the authorization experience. The team made continuous iterative changes to the funnel, tracking metrics closely enough to map fluctuations to individual changes. Based on the research findings and prior experiments, in my Product Design capacity, I was asked to take a POC design and develop an onboarding wizard to meet key user expectations.
The decision to eliminate personal access tokens in favor of OAuth-only authentication — in alignment with what my research surfaced — was significant enough that CircleCI's engineering team documented it publicly in their engineering blog.
Outcome
Measured impact
The research generated findings that continued to influence the product long after the sprint ended. Mari Federow, Associate Director of Customer Insights & Engagement, noted that the work was still being cited at CircleCI months later and credited with having a significant influence on improving early sign-up numbers.
12%
Improvement in early sign-up completion following OAuth-only implementation
What colleagues said
In their words
"Nan is an absolute delight to work with: she drives projects forward, keeping an eye on the big picture, while caring deeply about the details. As a content strategist, I've collaborated with Nan on both UX research and design projects, making a complex product simple and easy to use. Nan has an innate ability to ask questions that get to the heart of the matter and lead to improvements. Whether developing customer models, leading usability studies, or designing and testing new prototypes, she's always ready with constructive feedback. Nan is a star leader who will benefit any team devoted to delivering great experiences and excellent craftsmanship."
"Nan stood above the pack because she came in with a ton of cross-job experience and a true, clear passion for all things UX. She was relentless and motivated, met challenges head-on and worked through extremely tricky research scenarios. Work she did when she was here is still being cited and is credited with having a huge influence on drastically improving our numbers around early sign-ups. Nan quickly became an indispensable part of our team and I am sure she will do the same wherever she lands."
"Nan joined the team and didn't even miss a step. Her propensity for frequent communication, a genuine desire for feedback on her designs, and tendency to proactively seek answers meant that she was self sufficient from day 1. Her designs were clean and well thought out — she always provided a justification for her designs, giving us insights into her thought process and enabling us to share in the polishing of the UI/UX. She kept the user's needs and desires in her mind at all times."
