Choosing between adaptive agents: Some unexpected implications of level of scrutiny

Even with ample time and opportunity to use extensive data, people often make do with small samples, which increases their risk of making the wrong decision. A theoretical analysis indicates, however, that when the decision involves continually selecting among competing, adaptive agents who are eager to be selected, an error-prone evaluation may be beneficial to the decision maker. In this case, the chance of an error can motivate competitors to exert greater effort, improving their level of performance - which is the prime concern of the decision maker. This theoretical argument was tested empirically by comparing the effects of two levels of scrutiny of performance. Results show that minimal scrutiny can indeed lead to better performance than full scrutiny, and that the effect is conditional on a bridgeable difference between the competitors. We conclude by pointing out that small-sample-based, error-prone decisions may also maintain competition and diversity in the environment.