We propose a theory of social norms (or conventions) that implement substantial levels of inequality between men and women, ethnic groups, and classes and that persist over long periods of time despite being inefficient and not supported by formal institutions. Consistent with historical cases, we extend the standard asymmetric stochastic evolutionary game model to allow subpopulation sizes to differ and idiosyncratic rejection of a status quo convention to be intentional to some degree (rather than purely random as in the standard evolutionary models). In this setting, if idiosyncratic play is sufficiently intentional and the subordinate class is sufficiently large relative to the elite, then risk-dominated conventions that are both more unequal and inefficient relative to alternative conventions will be stochastically stable and may persist for long periods. We show that the same is true in a general bipartite network of the population if most of the subordinate groups interactions are local, while the elite is more "cosmopolitan". We apply the model to the evolution of wage conventions on the bipartite network of workers and employers, and find that an unequal monopsonistic wage convention is robust to the idiosyncratic play of workers that otherwise might displace it.