Using Customised LLM-Based Chatbots in Psychological Research: A Practical Tutorial
Abstract
Large language model (LLM)-based chatbots offer new methodological opportunities for psychological research by enabling scalable, standardised, and interactive research procedures. However, their usefulness depends on the researchers’ ability to customise, evaluate, and deploy them in ways that align with the methodological requirements of a given study. The current tutorial provides a practical, step-by-step framework for developing customised LLM-based chatbots for psychological research. We distinguish between no-code or low-code development approaches, including provider-hosted chatbot builders and visual or low-code programming platforms, and code-based approaches using proprietary model application programming interfaces (APIs) or locally hosted open-weight models. We compare these approaches in terms of technical expertise, behavioural and workflow control, study integration, cost, data governance, maintenance, and reproducibility. We also outline key components of the process of customising an LLM-based chatbot and illustrate deployment workflows across multiple platforms, including local desktop interfaces, messaging applications and survey platforms. Finally, we discuss key considerations for research ethics, privacy, data governance, security, transparency, and reproducibility, including recommendations for documenting and sharing chatbot configurations and study procedures. Through this tutorial, we aim to support researchers in integrating customised LLM-based chatbots into psychological research and in expanding the methodological possibilities available to them.
Related
- Piecewise modeling for longitudinal psychological research: a tutorial using R
- Bayesian Hierarchical Models for Longitudinal Meta-Analysis in Sports Science: A Practical Tutorial Using brms
- A Tutorial on Safe Anytime-Valid Inference: Practical Maximally Flexible Sampling Designs for Experiments Based on e-Values
- Interpreting P values in clinical research: A practical guide for clinicians
- What Pilot Studies Can (and Cannot) Do for Validity in Psychological Research
BibTeX
@misc{meilan2026using,
title = {{Using Customised LLM-Based Chatbots in Psychological Research: A Practical Tutorial}},
author = {Hu Meilan and Gabriel Rongyang Lau and Adalia Yin Hui Goh and Samuel Rhys Cox and Louis Tay and Andree Hartanto},
howpublished = {PsyArXiv},
year = {2026},
month = sep,
doi = {10.31234/osf.io/kf96p_v2},
url = {https://osf.io/preprints/psyarxiv/kf96p_v2/},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Using Customised LLM-Based Chatbots in Psychological Research: A Practical Tutorial
AU - Hu Meilan
AU - Gabriel Rongyang Lau
AU - Adalia Yin Hui Goh
AU - Samuel Rhys Cox
AU - Louis Tay
AU - Andree Hartanto
PY - 2026
DA - 2026/09/25
PB - PsyArXiv
DO - 10.31234/osf.io/kf96p_v2
UR - https://osf.io/preprints/psyarxiv/kf96p_v2/
AB - Large language model (LLM)-based chatbots offer new methodological opportunities for psychological research by enabling scalable, standardised, and interactive research procedures. However, their usefulness depends on the researchers’ ability to customise, evaluate, and deploy them in ways that align with the methodological requirements of a given study. The current tutorial provides a practical, step-by-step framework for developing customised LLM-based chatbots for psychological research. We distinguish between no-code or low-code development approaches, including provider-hosted chatbot builders and visual or low-code programming platforms, and code-based approaches using proprietary model application programming interfaces (APIs) or locally hosted open-weight models. We compare these approaches in terms of technical expertise, behavioural and workflow control, study integration, cost, data governance, maintenance, and reproducibility. We also outline key components of the process of customising an LLM-based chatbot and illustrate deployment workflows across multiple platforms, including local desktop interfaces, messaging applications and survey platforms. Finally, we discuss key considerations for research ethics, privacy, data governance, security, transparency, and reproducibility, including recommendations for documenting and sharing chatbot configurations and study procedures. Through this tutorial, we aim to support researchers in integrating customised LLM-based chatbots into psychological research and in expanding the methodological possibilities available to them.
ER -