TY - JOUR
T1 - Does Being In-Person Matter? Demonstrating the Feasibility and Reliability of Fully Remote Observational Data Collection
AU - Basha, Sydni A.J.
AU - Cai, Qiyue
AU - Lee, Susanne
AU - Tran, Tiffany
AU - Majerle, Amy
AU - Tiede, Shauna
AU - Gewirtz, Abigail H.
N1 - Publisher Copyright:
© The Author(s) 2024.
PY - 2024
Y1 - 2024
N2 - Many conventional research methods employed in randomized controlled trials were not possible during the height of the COVID-19 pandemic. In particular, behavioral observations are nearly universally gathered in-person. Observational methods are valued for the rich, informative data they produce in comparison to non-observational methods and are a cornerstone of parenting and family research. COVID provided the opportunity to, and indeed necessitated, the transition to fully remote observation. However, little to no studies have investigated whether remotely collected observational data are methodologically sound. This paper assesses the feasibility of remote data collection by describing the transition between in-person and fully remote observational data collection during a Sequential, Multiple Assignment, Randomized Trial (SMART) of a parenting program that took place both before and during the pandemic. Using mixed-methods data from coders, the overall quality of video-recorded data collected both before and during COVID was examined. Coder reliability over time was assessed with intraclass correlation coefficients. Results suggest that the frequency of audio problems, the severity of visual problems, and the level of administration challenges decreased after transitioning to remote data collection. Additionally, coders showed good to excellent reliability coding remotely collected data, and reliability even improved on some measured tasks. Although challenges to remote data collection exist, this study demonstrated that observational data can be collected feasibly and reliably. As observational data collection is a key method to assess parenting practices, these findings should improve researcher confidence in utilizing remote observational methods in prevention science.
AB - Many conventional research methods employed in randomized controlled trials were not possible during the height of the COVID-19 pandemic. In particular, behavioral observations are nearly universally gathered in-person. Observational methods are valued for the rich, informative data they produce in comparison to non-observational methods and are a cornerstone of parenting and family research. COVID provided the opportunity to, and indeed necessitated, the transition to fully remote observation. However, little to no studies have investigated whether remotely collected observational data are methodologically sound. This paper assesses the feasibility of remote data collection by describing the transition between in-person and fully remote observational data collection during a Sequential, Multiple Assignment, Randomized Trial (SMART) of a parenting program that took place both before and during the pandemic. Using mixed-methods data from coders, the overall quality of video-recorded data collected both before and during COVID was examined. Coder reliability over time was assessed with intraclass correlation coefficients. Results suggest that the frequency of audio problems, the severity of visual problems, and the level of administration challenges decreased after transitioning to remote data collection. Additionally, coders showed good to excellent reliability coding remotely collected data, and reliability even improved on some measured tasks. Although challenges to remote data collection exist, this study demonstrated that observational data can be collected feasibly and reliably. As observational data collection is a key method to assess parenting practices, these findings should improve researcher confidence in utilizing remote observational methods in prevention science.
KW - COVID
KW - Military families
KW - Observational methodology
KW - Observational research
KW - Parenting
UR - http://www.scopus.com/inward/record.url?scp=85198346651&partnerID=8YFLogxK
U2 - 10.1007/s11121-024-01706-6
DO - 10.1007/s11121-024-01706-6
M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???
C2 - 38995340
AN - SCOPUS:85198346651
SN - 1389-4986
JO - Prevention Science
JF - Prevention Science
ER -