A mixed-methods approach for complex intervention specification and evaluation (MACISE): illustrative example of Perintatal Psychiatry Access Programs
Abstract
AimsComplex interventions integrate several core components that interact in ways that are hypothesized to produce the intervention effect(s). Drawing on established practices in program evaluation, we developed the mixed-methods approach for complex intervention specification and evaluation (MACISE) to estimate programmatic reach of complex interventions.MethodsMACISE provides a sequential, mixed-methods approach to estimate programmatic reach of complex interventions. First, researchers conduct key informant interviews with intervention team members and review programmatic documents to: (1) characterize the key elements of the complex intervention, (2) elucidate how the core components generate the hypothesized intervention effects (conceptual model) and (3) describe the relevant activities and decision-points in a process map (operational model). Researchers then triangulate and visually depict how variables from the relevant intervention data align with the conceptual and operational model. Next, researchers validate the complex intervention, conceptual model, and process map with program staff through member-checking. Finally, statistical evaluation of programmatic reach of the complex intervention is conducted. Application of MACISE to Perinatal Psychiatry Access Programs (Access Programs) in Massachusetts and Washington State resulted in the development of conceptual and operational models and alignment to the measures for programmatic reach in the respective administrative datasets.ResultsIn both Access Programs, respondents reported the telephone consultation as the “engine” of the respective programs although programs operationally defined consultation differently. Following implementation of the Access Program in Massachusetts (July 2014 to January 2019), the mean number of telephone encounters (operationally defined as a physician-to-physician telephone consultation) for the Massachusetts program was 211.6 (range: 166–272) per month and the mean number of unique providers receiving this consultation per month was 117.0 (range: 88–151). For the 3 years following implementation of the Access Program in Washington (January 2016 to December 2018), the mean number of telephone encounters, (operationally defined as the number of staff-to-physician and physician-to-physician consultations) was 8.6 (range: 3–22) monthly and the mean number of unique providers contacted monthly was 7.6 unique providers (range: 2–21).ConclusionMACISE provides a practical approach to link intervention data to conceptual and operational models to facilitate systematic and accurate estimation of complex intervention reach.
Related
- A pilot evaluation of a preventive intervention to support mental health among dementia caregivers: a controlled longitudinal study
- A mixed methods feasibility evaluation of a shared decision-making intervention for community dwelling adults living with dementia, family carers and practitioners
- Mixed-methods evaluation of a rights-based intervention to reduce long-term segregation in mental health services: the HOPE(S) National Programme in England.
- Economic evaluation protocol of the Ntshembo (Hope) Trial (RCT) in rural and urban South Africa: an intervention to optimise adolescence body mass index (BMI) pre-conception to address the triple burden of malnutrition
- Effect of In-line Clinical Decision Support to Improve Accuracy of American Society of Anesthesiologists Physical Status Documentation: A Two Center Evaluation of a Quality Intervention
BibTeX
@article{bayne2026mixedmethods,
title = {{A mixed-methods approach for complex intervention specification and evaluation (MACISE): illustrative example of Perintatal Psychiatry Access Programs}},
author = {Mitchell Bayne and Ana J. Schaefer and Radley C. Sheldrick and Victor Puac-Polanco and Ekaanth Veerakumar and Amritha Bhat and Deborah Cowley and Ayse Akincigil and Grace Masters and Stephen Crystal and John Straus and Uruj Kamal Haider and Elizabeth McGinn and Chahdael B. Foreman and Naysha Shahid and Nancy Byatt and Thomas I. Mackie},
journal = {Frontiers in Public Health},
year = {2026},
month = nov,
doi = {10.3389/fpubh.2026.1905944},
url = {https://www.frontiersin.org/articles/10.3389/fpubh.2026.1905944},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - A mixed-methods approach for complex intervention specification and evaluation (MACISE): illustrative example of Perintatal Psychiatry Access Programs
AU - Mitchell Bayne
AU - Ana J. Schaefer
AU - Radley C. Sheldrick
AU - Victor Puac-Polanco
AU - Ekaanth Veerakumar
AU - Amritha Bhat
AU - Deborah Cowley
AU - Ayse Akincigil
AU - Grace Masters
AU - Stephen Crystal
AU - John Straus
AU - Uruj Kamal Haider
AU - Elizabeth McGinn
AU - Chahdael B. Foreman
AU - Naysha Shahid
AU - Nancy Byatt
AU - Thomas I. Mackie
PY - 2026
DA - 2026/11/04
JO - Frontiers in Public Health
DO - 10.3389/fpubh.2026.1905944
UR - https://www.frontiersin.org/articles/10.3389/fpubh.2026.1905944
AB - AimsComplex interventions integrate several core components that interact in ways that are hypothesized to produce the intervention effect(s). Drawing on established practices in program evaluation, we developed the mixed-methods approach for complex intervention specification and evaluation (MACISE) to estimate programmatic reach of complex interventions.MethodsMACISE provides a sequential, mixed-methods approach to estimate programmatic reach of complex interventions. First, researchers conduct key informant interviews with intervention team members and review programmatic documents to: (1) characterize the key elements of the complex intervention, (2) elucidate how the core components generate the hypothesized intervention effects (conceptual model) and (3) describe the relevant activities and decision-points in a process map (operational model). Researchers then triangulate and visually depict how variables from the relevant intervention data align with the conceptual and operational model. Next, researchers validate the complex intervention, conceptual model, and process map with program staff through member-checking. Finally, statistical evaluation of programmatic reach of the complex intervention is conducted. Application of MACISE to Perinatal Psychiatry Access Programs (Access Programs) in Massachusetts and Washington State resulted in the development of conceptual and operational models and alignment to the measures for programmatic reach in the respective administrative datasets.ResultsIn both Access Programs, respondents reported the telephone consultation as the “engine” of the respective programs although programs operationally defined consultation differently. Following implementation of the Access Program in Massachusetts (July 2014 to January 2019), the mean number of telephone encounters (operationally defined as a physician-to-physician telephone consultation) for the Massachusetts program was 211.6 (range: 166–272) per month and the mean number of unique providers receiving this consultation per month was 117.0 (range: 88–151). For the 3 years following implementation of the Access Program in Washington (January 2016 to December 2018), the mean number of telephone encounters, (operationally defined as the number of staff-to-physician and physician-to-physician consultations) was 8.6 (range: 3–22) monthly and the mean number of unique providers contacted monthly was 7.6 unique providers (range: 2–21).ConclusionMACISE provides a practical approach to link intervention data to conceptual and operational models to facilitate systematic and accurate estimation of complex intervention reach.
SN - 2296-2565
ER -