Agentic Code Review Research
Studying how junior developers understand, trust, and verify AI-generated code review feedback.
Role
UX Researcher
Industry
HCI Research
Duration
3 months
Project Overview
AI code review tools can provide fast feedback, but junior developers still need to understand, evaluate, and verify the suggestions they receive. While tools like ChatGPT and Cursor can support debugging and learning, their feedback may also feel incomplete, overly confident, or difficult to trust.
I conducted mixed-methods UX research to understand how junior developers use AI-assisted code review tools, focusing on trust, learning support, feedback quality, and verification behavior. The goal was to identify how AI developer tools can better support human judgment instead of simply providing answers.
The Research Challenge
AI code review is not only a technical problem. It is also a user trust and learning problem.
The core challenge was:
How might we understand the way junior developers evaluate, trust, and learn from AI-generated code review feedback?
This meant looking beyond whether users found AI tools “helpful” and studying how they made decisions: when they accepted feedback, when they questioned it, and what support they needed to verify suggestions confidently.
Stage 1. Framing the Research Problem
I began by narrowing a broad interest in AI-assisted coding into a focused UX research problem around junior developers and AI code review. This user group was especially important because junior developers are still building technical judgment.
Through this framing, the project moved away from asking whether AI code review is simply “good” or “bad.” Instead, the study focused on how users experience AI feedback in real coding workflows, especially around helpfulness, clarity, trust, learning support, and verification behavior.
This helped position the research around the relationship between AI feedback and human decision-making.
Stage 2. Designing the Study
To capture both broad patterns and deeper user reasoning, I used a sequential explanatory mixed-methods approach.
The study began with a survey of 30 computer science students to understand how participants perceived AI code review tools across dimensions such as helpfulness, clarity, learning support, and trust. After identifying patterns from the survey, we conducted 11 semi-structured interviews to better understand the reasoning behind those responses.
This research strategy allowed us to connect quantitative trends with qualitative stories, making the findings more grounded and actionable.
Stage 3. Exploring User Workflows
During the interviews, I focused on how participants actually used AI tools during coding, debugging, and review. Rather than only asking for opinions, the interview questions explored specific moments of trust, confusion, hesitation, and verification.
Participants described using ChatGPT and Cursor in different ways. ChatGPT was often used for explanations, conceptual understanding, and step-by-step reasoning, while Cursor felt more directly integrated into the coding workflow through inline, context-based suggestions.
This stage helped reveal that AI code review is not a single experience. Users move between learning, debugging, checking, and deciding whether a suggestion is safe to apply.
Stage 4. Synthesizing Insights
After analyzing the survey and interview data, several patterns emerged.
Many junior developers did not treat AI as a final authority. Instead, they used it as a second opinion, explanation partner, or starting point for debugging. Beginners often valued detailed explanations because they wanted to understand why something was wrong, while more experienced users preferred concise and actionable feedback.
The research also revealed important trust gaps. Participants were concerned about hallucinated APIs, missed logic errors, limited cross-file understanding, and feedback that sounded confident even when it was not fully correct.
These findings showed that trust in AI depends not only on the quality of the answer, but also on whether users can understand and verify it.
Stage 5. Translating Findings into Product Opportunities
The final stage focused on turning research insights into product-facing design opportunities.
The findings pointed toward AI code review features that could better support junior developers, such as confidence indicators, “verify this suggestion” workflows, source-linked explanations, cross-file context summaries, and adjustable feedback depth for different experience levels.
This helped connect the research to practical design implications. Instead of designing AI tools that simply generate more feedback, the opportunity is to design tools that help users build judgment, understand risk, and learn from the review process.
Outcomes
The research showed that junior developers value AI feedback, but they do not automatically trust it. They often appreciate AI tools for explanation, debugging support, and alternative perspectives, while still needing stronger ways to verify correctness.
The final deliverables included survey findings, interview insights, qualitative themes, and product design recommendations for AI-assisted developer tools.
This project strengthened my ability to conduct UX research for AI systems and translate user behavior into actionable product opportunities.
Reflection
This project helped me understand that trust in AI is not only about accuracy. It is about whether users can understand, question, and verify what the system provides.
My biggest takeaway was that AI tools should support human judgment rather than replace it. For junior developers especially, good AI feedback should teach, explain, and encourage verification.
For future iterations, I would explore prototype concepts for verification-focused AI code review, such as confidence labels, source-backed explanations, and feedback modes tailored to different developer experience levels.
Other projects
Explainable Meeting Scheduler
Designing a transparent scheduling tool that helps graduate teams find fairer meeting times with less back-and-forth.
Interactive Video Segmentation Player
Designing an AI-powered video player that makes long videos easier to understand, navigate, and control
Responsive Event Search Web App
Designed and developed a responsive event search experience with structured filters, dynamic results, event detail views, and expandable venue information.
