Mobile diaries – Benchmark against metered measurements: An empirical investigation

Researchers seeking to study the relationships between consumers' communications, attitudes, and behaviors could benefit from monitoring consumers over time, across multiple locations and channels, and in a way that reflects consumers' subjective perceptions. Diaries on smartphones (mobile diaries) can be used as a research tool for such purposes. A mobile diary is a self-report instrument whereby people use their mobile handset to repeatedly report experiences of interest. Mobile diaries are increasingly used in psychology, geography, medicine, and commercial marketing. Yet they have rarely been used for quantitative marketing research, and were not benchmarked against best-practice metrics in marketing. In this study, we aim to set the ground for using mobile diaries in quantitative marketing research. We first lay out the theoretical infrastructure for the usage of mobile diaries, and describe possible respondent reporting concerns, including concerns related to non-reporting, reporting over time, and concerns stemming from individual-level heterogeneity. We demonstrate the potential of mobile diaries, as well as the importance of the various concerns, using a benchmark test case in the context of primetime TV viewing. Our benchmark uses a sample of respondents with both mobile diary viewing reports and Nielsen People Meter (NPM) records. Our analysis reveals that averaging across all conditions, 47.4%–64.7% of the NPM records are reported by the diary. The major sources for mismatch are random time periods without alarms, short viewings, and periodic reporting inactivity (pulsing). Concerns such as a decrease in reporting rates over time (e.g., fatigue), smartphone ownership, and demographic variation across individuals have relatively small effects on reporting likelihood. Analyzing the cases in which diary reports do not have a matching NPM record, we find many of them can be attributed to out-of-home viewing and viewing on non-metered devices. This finding demonstrates how mobile diaries can complement metered measurements. Overall, aggregate diary-based ratings have a 0.90 correlation with NPM ratings. We discuss implications for designing and using mobile diary studies in marketing.