Replication package for “No Two Developers Think Alike: How Problem-Solving Styles and Experience Shape Needs in Conversational Interaction with Copilot”

Replication package for “No Two Developers Think Alike: How Problem-Solving Styles and Experience Shape Needs in Conversational Interaction with Copilot”, accepted at ICSME 2026.

This replication package contains the following files:

• data
  • transcripts/*: transcripts composed of audio transcripts, chat logs, and telemetry
  • chats.csv: annotated chat logs
  • codebook.xlsx: codebook for the qualitative analysis of the transcripts
  • participants.csv: pre- and post-study survey data, including participants demographics, experience, and GenderMag/TAM/NASA-TLX data
• figures/*: figures included in the paper, most are generated in analyze.py
• session/*: English and Brazilian Portuguese versions of the study protocol, practice tasks, and full tasks
  • codebase/*: English and Brazilian Portuguese versions of the codebase and telemetry extension used in the code change tasks
• analyze.py: the code used to run the full quantitative analysis

To run the analysis, make sure you have conda installed. Then create an environment and install the dependencies:

conda env create -f environment.yaml -p ./.env
conda activate ./.env

You can now run analyze.py. We recommend to use in interactive window, e.g. in VSCode.