When I started experimenting with AI for analysis, I put the session transcripts into the tool and asked for themes. It gave me output quickly, but the output was flat. It gave me global themes across the whole sample with no sense of how participants diverged, and that divergence is often the interesting part. Findings could also land on the wrong task, and everything that happened outside the words was missing.

When I analyzed sessions by hand, I never worked from the transcript alone. My notes already had participant details that mattered for interpreting the data, since different people bring different factors that shape how they respond. My notes also had where each task started, what I saw people do, how they reacted emotionally, and what stood out to me as important. So I started giving AI the same context I give myself for analysis. It comes down to three changes to the analysis input, plus a verification step.

Context Layer 1: Participant Metadata

Before analysis, I add a block of participant information at the top of each transcript. This is the same information you might already have in a participant spreadsheet. What goes in the block depends on the study. Anything I have a hypothesis about belongs there: factors I think could change how people react to something, or why.

Then I tell the model to use it. I give it the study's research questions and ask for themes around each one, overall and by subgroup.

Before I added metadata to the transcripts, I had to define every subgroup comparison myself. I'd pull the participant details from my notes or the screener grid, then tell the AI something like: "P1, P3, P5, and P7 are experienced admins. P2, P4, P6, and P8 are new admins. What differences do you see across these two groups for this research question?" Then I'd do it again for the next comparison.

I already kept this metadata in my notes when I analyzed sessions by hand. Putting it in the transcript brings the same practice into AI-assisted analysis, and the model can make those cuts without me defining the groups every time.

Context layer 2: Task and stimulus labels

When AI pulls themes from a session, it has to infer where one task ended and the next began, and what each task was. The conversation doesn't always make that obvious. So I mark where each task starts, on its own line, right before that part of the session. Not inside a line of dialogue, or it may get read as something a participant said.

My labels also tell the model what each task is, so I can compare the same task across everyone. If a task includes a prototype, a concept image, or a specific page, I may label that too. This matters most when one person sees more than one version in a different order. Moderator instructions may be identical across versions, because that can be the whole point of a controlled comparison. Without labels, findings get flattened across versions, or even worse, attached to the wrong one.

I don't always do this. If a session was mostly conversation, it doesn't buy me much. And if the study uses a between-subjects design rather than a within-subjects design, the version each person saw will likely go in the metadata block instead. Even in a within-subjects design, I often add the order each person saw the versions to the metadata block.

Context layer 3: Notes on what the transcript can't hold

A transcript only holds one layer of a session, which is the words people say. It misses what people did, how they said it, and the context around the session. A participant might tell you a task was easy when it took three minutes and two wrong turns. The transcript only has the word easy. It can't tell sarcasm from sincerity either.

Transcription has gotten good enough that I stopped spending session time writing down most of what people say. Now my notes cover what's on screen, how someone interacts with it, emotional and nonverbal reactions, and moments where what someone said didn't match what they did. I also flag anything that stands out as strategically important to the team. Not writing everything down also freed up my attention. I'm more present in the conversation, and I have more room to follow something interesting.

My notes are a separate file from the transcript, and I give AI both files for each session. The notes file starts with the same participant metadata block and uses the same task labels, so the model can easily match what I observed to what was said at the same moment.

Verify before sharing

AI gets things wrong in predictable ways, and hallucination is only one of them. Context layers reduce these errors, but they don't remove them. 

Some ways I check for errors:

• Ask the model to check every quote against the source files

• Ask it to look for errors like these by name, and correct what it finds

• Word my prompts neutrally, since AI tends to agree with how a question is framed

• Sometimes rerun the check in a fresh session or a second AI tool, so the review doesn't share the original analysis's blind spots

Sharing the approach

I shared this approach with a research vendor that ran many of our team's studies, and wrote up guidance for them: what to put in the metadata, what AI catches and misses, and why session notes should focus on what AI misses instead of what people say word for word. I've also written about it in a LinkedIn series. The first post reached over 50,000 impressions.

Other ways I use AI

This changes constantly. Some of the ways I've used AI:

• Building a research database in NotebookLM so stakeholders can query past research themselves

• Running an agent that gathers information and sends me daily updates, including catching me up on meetings I couldn't attend and documents I haven't had time to read

• Getting up to speed quickly on a new area

• Checking my work against team and product strategy

• Using Claude skills others have built