I study how a teammate’s agreement or disagreement changes the subjective decisions people make with GenAI. Unlike an individual user study, this work introduces a social layer: each participant must interpret both an AI recommendation and another person’s judgment. That social layer complicates experimental control. Every partner shapes the interaction in ways the study cannot fully predict. With two independently recruited participants, I could not confidently attribute differences across sessions to the factor I intended to test.
My solution was to keep two people in the interaction while making one role controlled. One person is the participant. The other is a confederate, a trained research-team member who appears to be another participant. She knows the study protocol, behaves naturally, and follows neutral scripts prepared for every decision the participant might make. This approach is inspired by classic social-influence research, particularly Asch’s conformity experiments, which showed that people may shift their judgments to align with a group.
Designing that role became one of the most important and difficult parts of the study. This note focuses on why the confederate was necessary, how I designed her participation, and what this method made possible and complicated.
Controlling a subjective, live interaction
I intentionally designed the decision scenarios to be subjective. The task has no objectively correct answer, so participants cannot resolve the disagreement by finding a factual error.
This distinction matters. If a participant changes their mind after learning that their original answer was incorrect, that is informational influence. I am interested in something more difficult to isolate: whether another person’s judgment can affect how someone interprets the same GenAI recommendation and evaluates their own position.
Using a confederate required me to design two things together: the script and the person delivering it.
The script. Writing it was more difficult than simply producing a neutral response. Neutrality can mean different things across contexts and populations. A response that seems balanced in one setting may sound distant or persuasive in another. I combined a review of the literature with different script lengths and wording across ten pilot sessions. The goal was not to create a universally neutral script, but to develop neutral responses that fit this particular study and remained aligned with its purpose.
The confederate. She was not simply reciting a script. Because the interaction unfolded live, she had to listen carefully to each participant, identify their response, and select the appropriate scripted reply in real time. She needed to do this quickly and accurately without breaking the flow of the conversation or revealing her knowledge of the protocol. At the same time, she needed to remain natural enough to appear like someone encountering the study for the first time.
That ability also required practice and piloting. Because the study focuses on the effect of an expressed position, the confederate was trained to keep her tone, body language, facial expression, and other reactions as consistent as possible. The challenge was to make the interaction feel ordinary while executing the protocol precisely.
What the design makes measurable
This design lets me compare each participant’s individual decision, confidence, and need for additional information before and after the discussion. I can then examine whether a teammate’s agreement or disagreement is associated with change across any of these measures, rather than looking only at the final decision. I can also explore participants’ motivations when their opinions change.
What control leaves out
Using a confederate gives the study consistency, but that consistency comes with limitations.
The interaction cannot capture the full complexity of a naturally formed team. Real collaborators may have histories, roles, expertise, status differences, and existing relationships. They can introduce evidence, challenge assumptions, misunderstand one another, or shift the direction of a conversation unexpectedly.
This study deliberately reduces that complexity. It is not intended to reproduce every aspect of workplace collaboration. Instead, it creates a controlled starting point for examining how peer judgment enters a person’s relationship with GenAI.
Why the social layer matters
Designing this protocol has reinforced something that can be easy to overlook when studying human–AI interaction: another person’s judgment is not simply background context. It can shape how people interpret GenAI, assess their own reasoning, and move forward.
The confederate is a methodological tool, but designing her role has also become part of the research itself. Every decision about what she says, how she says it, and what she deliberately leaves out reflects a larger question: how can we study social influence without removing the social qualities that make an interaction believable?