%201.png)
Comparing ChatGPT with Human and Voice Agent Interaction
How to use thematic analysis to compare the conversations of ChatGPT, humans, and Voice agents?
Overview
This project aims to investigate the current usage of ChatGPT and its role as a dialogue partner compared to humans. Through analyzing log data, specific tasks conducted with ChatGPT will be identified, along with the types of questions asked and the information generated by the system. Additionally, participants will provide their partner models of ChatGPT, allowing for a comparison with traditional voice agents and human partners.
This project is part of my ongoing MSc Human Computer Interaction thesis, and therefore, a detailed analysis of the results is currently in progress. For more up-to-date information, please feel free to contact zhouquincey@gmail.com.
-
Role: Sole UX Researcher
-
Method: Thematic Analysis+Questionnaire
-
Time: 3 months
Context
The aim of this study is to explore how people currently use ChatGPT. I aim to identify what specific tasks people conduct with ChatGPT, what types of questions people ask of the system and the types of information the system produces. Through the study I also look to assess how people see ChatGPT as a dialogue partner compared to a human. The project will therefore:
-
Identify through log data the types of tasks that people currently use ChatGPT for.
-
Measure people’s partner models of ChatGPT and compare it with traditional voice agent and human partners.
Methods
To address the research questions, participants will be asked to upload an anonymized version of their ChatGPT logs. We will also ask participants to complete self report questionnaire (Partner Modelling Questionnaire) when thinking about their interaction with a human partner and a voice assistant in a within participants study.
​
Participants will be recruited through Prolific, and will include adults aged 18 and above who have experience using ChatGPT. Prior to the experiment, participants will complete a pre-test survey to collect demographic information and to assess their prior experience with voice agents
​
My pre-test survey has been designed and is now ready. The questionnaire link is provided below: https://forms.gle/kLWCN6yxLUGTnvrS9. Everyone is welcome to participate and fill it out.