Is it the typeset or the type of statistics? Disfluent font and self-disclosure

Background. The security and privacy communities have become increasingly interested in results from behavioral economics and psychology to help frame decisions so that users can make better privacy and security choices. One such result in the literature suggests that cognitive disfluency (presenting questions in a hard-to-read font) reduces self-disclosure. Aim. To examine the replicability and reliability of the effect of disfluency on self-disclosure, in order to test whether such approaches might be used to promote safer security and privacy behaviors. Method. We conducted a series of survey studies on human subjects with two conditions - disfluent and fluent font. The surveys were completed online (390 participants throughout the United States), on tablets (93 students) and with pen and paper (three studies with 89, 61, and 59 students). The pen and paper studies replicated the original study exactly. We ran an independent samples t-test to check for significant differences between the averages of desirable responses across the two conditions. Results. In all but one case, participants did not show lower self-disclosure rates under disfluent conditions using an independent samples t-test. We re-analyzed the original data and our data using the same statistical test (paired t-test) as used in the original paper, and only the data from the original published studies supported the hypothesis. Conclusions.We argue that the effect of disfluency on disclosure originally reported in the literature might result from the choice of statistical analysis, and that disfluency does not reliably or consistently affect self-disclosure. Thus, disfluency may not be relied on for interface designers trying to improve security or privacy decision making.