Testing for Bias in Order Assignment

Abstract: Statistical methods are developed for assessing the likelihood of prejudicial bias in agent-assigned permutations, such as the ordering of candidates on an election ballot. The null hypothesis of an unbiased order assignment is represented by several forms of probabilistic exchangeability of the random orderings, while bias is represented either by compatibility with an assumed ranking of the items with respect to a hypothesized preference criterion (PC) or by linear concordance with assumed scores of the items on a PC scale. These methods are applied to five races in the 2016 Texas Republican primary election. Significant evidence of bias in at least one of the approximately 245 reporting counties is found in four of the five races; in two races significant evidence of bias in at least six and ten counties, respectively, is found.