\documentclass[smallheadings]{article}

\usepackage{tabularx,booktabs, placeins}
\newcolumntype{Y}{>{\centering\arraybackslash}X}

\usepackage{float}

\usepackage[]{graphicx}
\usepackage{mathrsfs}
\usepackage{mdwlist}
\usepackage{amssymb}
\usepackage{eucal}
\usepackage{units}
\usepackage{afterpage}


\usepackage{soul}

\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{epstopdf}
\usepackage{dsfont}
\usepackage{bm}
\usepackage{enumerate}
\usepackage[usenames, table]{xcolor}
\usepackage{hhline}
\definecolor{blue}{rgb}{0,0,1}
\definecolor{purple}{rgb}{0.6,0,1}
\usepackage{natbib}
\usepackage{multirow}
\usepackage[margin=1.2in]{geometry}
\usepackage{pdfpages}
\usepackage{float}
\usepackage{setspace}

\usepackage{pdflscape}
\usepackage{marvosym}


\usepackage{minitoc}
\usepackage[toc,page,header]{appendix}

\usepackage{multibib}
\newcites{appendix}{References}
\renewcommand \thepart{}
\renewcommand \partname{}


\usepackage[colorlinks = true, citecolor = blue, linkcolor = blue, urlcolor  = blue,]{hyperref}

\usepackage{rotating}
\usepackage[labelfont=bf]{caption}


\usepackage{array}
\newcolumntype{L}[1]{>{\raggedright\let\newline\\\arraybackslash\hspace{0pt}}m{#1}}
\newcolumntype{C}[1]{>{\centering\let\newline\\\arraybackslash\hspace{0pt}}m{#1}}
\newcolumntype{R}[1]{>{\raggedleft\let\newline\\\arraybackslash\hspace{0pt}}m{#1}}
\newcolumntype{Y}{>{\centering\arraybackslash}X}

\clubpenalty = 1000
\widowpenalty = 1000

\DeclareMathOperator{\plim}{plim}

\renewcommand{\theenumi}{(\roman{enumi})}
\renewcommand{\labelenumi}{\theenumi}
\newcommand{\bmath}{\begin{eqnarray}}
\newcommand{\emath}{\end{eqnarray}}
\newcommand{\bmathnn}{\begin{eqnarray*}}
\newcommand{\emathnn}{\end{eqnarray*}}
\newcommand{\bi}{\begin{itemize}}
\newcommand{\ei}{\end{itemize}}
\newcommand{\var}{\mathrm{var}}
\newcommand{\supp}{\mathrm{supp}}
\newcommand{\logit}{\mathrm{logit}}
\newcommand{\sech}{\mathrm{sech}}
\newcommand{\abs}{\mathrm{abs}}
\newcommand{\sgn}{\mathrm{sgn}}
\newcommand{\del}{\partial}
\newcommand{\w}{\omega}


\newcommand{\red}{\color{red}}
\newcommand{\gray}{\color{gray}}

\newcommand{\blue}{\color{blue}}
\newcommand{\green}{\color{green}}
\newcommand{\black}{\color{black}}
\newcommand{\purple}{\color{purple}}

\usepackage{ bbold }
\newcommand{\one}{\mathbb{1}}


\setcounter{MaxMatrixCols}{10}

\theoremstyle{plain}
\newtheorem{proposition}{Proposition}
\newtheorem{prediction}{Prediction}
\newtheorem*{example}{Example}
\setcounter{section}{0}
\newtheorem{theorem}[proposition]{Theorem}
\newtheorem{lemma}{Lemma}
\newtheorem{corollary}[proposition]{Corollary}
\newtheorem{result}{Result}
\newtheorem{hypothesis}{Hypothesis}
\theoremstyle{definition}
\newtheorem{definition}{Definition}
\newtheorem{assumption}{Assumption}
\newtheorem*{remark}{Remark}
\theoremstyle{plain}
\DeclareMathOperator*{\argmax}{argmax}
\DeclareMathOperator*{\argmin}{argmin}
\DeclareMathOperator{\core}{Core}
\DeclareMathOperator{\cov}{cov}
\usepackage{setspace}
\setstretch{1.4}


\usepackage{tikz}
\usetikzlibrary{fit,matrix}
\usepackage{collcell}
\definecolor{airforceblue}{rgb}{0.29, 0.44, 0.54}
\newcommand{\hhlline}[3]{\draw (#1-#2-2.north west) -- (#1-#2-#3.north east);}
\newcommand{\vllline}[3]{\draw (#1-5-#3.north west) -- (#1-#2-#3.north west);}
\newcommand{\vrlline}[3]{\draw (#1-5-#3.north east) -- (#1-#2-#3.north east);}

 \renewcommand{\thesection}{\Roman{section}.}
 \renewcommand{\thesubsection}{\Alph{subsection}.}


\defcitealias{SEP.voting}{Stanford Encyclopedia of Philosophy (2016)}


\begin{document}

\title{Social Preferences over Ordinal Outcomes}
\author{Sandro Ambuehl, B. Douglas Bernheim\footnote{Ambuehl: Department of Economics and UBS Center for Economics in Society, University of Zurich, Bl\"uemlisalpstrasse 10, 8006 Z\"urich, Switzerland, sandro.ambuehl@econ.uzh.ch. Bernheim: Department of Economics, Stanford University, 579 Jane Stanford Way, Stanford, CA, 94305, USA, bernheim@stanford.edu. We are grateful to Eva K\"upper for truly outstanding research assistance. This project was funded by the Department of Economics at Stanford University and by the Department of Economics at the University of Zurich. All experiments were approved by the Stanford IRB (protocol \#53339) and by the Ethics Review Board of the Department of Economics at the University of Zurich (OEC IRB \#2020-063).}}


\maketitle

\begin{abstract}
We study social preferences in settings where someone who chooses on behalf of others knows how those individuals rank the available options but may lack cardinal information concerning those comparisons. Contrary to majoritarian principles, most people place more weight on preventing least-preferred outcomes for others than on enabling most-preferred outcomes. Ranks matter both intrinsically and because they provide a basis for inferring cardinal utility. Ordinal aggregation principles are stable across domains and countries with divergent political traditions. Designing attractive social choice mechanisms is challenging in practice partly because aggregation principles that make manipulation difficult yield outcomes people consider normatively unappealing.


\textit{JEL codes: C91, D71}
\end{abstract}


\doparttoc
\faketableofcontents
\part{}


Envision a benevolent party (a ``Planner'') who must make a decision that impacts the members of some group (“Stakeholders”). The Planner possesses reliable information about the Stakeholders’ ordinal preferences over the available options (``within-menu rankings''), but not about their cardinal preferences (intensity). How would the typical individual in the role of Planner determine the ``best'' choice for the group? In other words, what is the nature of social preferences over ordinal outcomes?\footnote{As we use the term, an \emph{ordinal outcome} is the vector of ranks Stakeholders assign to a given social action for a fixed opportunity set.}

This issue is of broad importance for at least three reasons. First, cardinal information is often unreliable or unavailable in practice. For example, a supervisor charged with allocating a diverse set of assignments may learn which tasks individual team members prefer without being able to distinguish strong tastes from a tendency to complain. More fundamentally, in the spirit of \cite{Arrow1951}, people may intuitively question the validity of cardinal interpersonal utility comparisons. For example, they may distrust their ability to gauge differences between the pleasure different people derive from incremental income. The greater reliability of ordinal information may help explain why procedures for making group decisions typically focus on rankings; examples include matching markets, committee deliberations, and elections. Second, people may care about rankings intrinsically. For example, they may wish to avoid selecting an option that any group member ranks last, or they may feel ethically obliged to follow the will of the majority. For these reasons, inquiries into social preferences that impose the cardinal Bergson-Samuelson structure---the predominant approach in the pertinent literature \citep[e.g.][]{EngelmannStrobel, BoltonOckenfelsComment, JacksonYariv.Experiment}---may improperly identify underlying objectives. Third, it is arguable that, in representative democracies, policy makers ought to defer to citizens’ judgments about importance of ordinal outcomes and the appropriate criteria for ordinal preference aggregation. Under that view, our positive analysis of social choice has important normative implications.


We conduct experiments designed to address four main questions.  First, what aggregation principles best capture people's social preferences over ordinal outcomes? For instance, do they seek compromise solutions, or insist that the majority should prevail? Second are these principles stable, or do they vary from context to context? Third, do people intrinsically care about ordinal outcomes, or do they merely use them to draw inferences about cardinal outcomes and then aggregate using standard Bergson-Samuelson social welfare functions \citep[in the spirit of][]{Apesteguiaetal.Justice}? Fourth, do aggregation preferences vary across cultures with divergent political and social traditions?


In our experiments, subjects acting as Planners make potentially consequential decisions for groups of Stakeholders. These decisions are of two types: distributing work tasks among five Stakeholders (the \emph{work domain}) and assigning a contribution on behalf of five Stakeholders to a single political entity (the \emph{political domain}). Because we are interested in identifying structural aggregation preferences, we design the experiment to remove considerations arising from self-interest, paternalism  \citep[i.e., the tendency to ignore or discount Stakeholder judgments with which the Planner disagrees, as in][]{AmbuehlBernheimOckenfels}, and the potential incentive incompatibility of truthful preference revelation by Stakeholders.

Social preferences over ordinal rank profiles manifest themselves through desired \emph{social choice rules}---that is, functions mapping each ordinal preference profile over a given set of alternatives to a selection from that set. The choices Planners make on behalf of others reveal the social choice rules that embody the normative principles they favor. Distilling those principles is, nevertheless, conceptually challenging. Even in simple social choice problems (e.g., with five people and three options), the set of possible mappings from group members' preference profiles to choices is astronomically large, and merely cataloging the observed rules provides little insight. We therefore proceed in three steps. First, drawing on the theoretical literature, we identify a reasonably large set of plausible aggregation rules. Each rule implies a distinctive \emph{fingerprint} of implied best choices over the set of conceivable five-person three-option preference profiles. Second, we assign each subject to one of various pre-specified rules using a Bayes classifier. Third, we use a clustering algorithm to determine whether our pre-specified rules omit empirically important possibilities.


In answer to the first question, we find that the overwhelming majority of subjects behave as if they assign a cardinal utility level (a \emph{score}) to each rank and select the option that maximizes the sum of these utilities (\emph{scoring rules}). Many well-known social choice criteria fall within this category. For example, plurality rule assigns a score of one to each person's favorite alternative and zero to all others. The curvature of the mapping from ranks to scores determines whether improvements from low or high ranks are more important.
Concave scoring rules promote compromise by placing more weight on gains from low ranks. All else equal, they favor an option ranked $n$-th by two people over an alternative ranked $(n-1)$-th by one and $(n+1)$-th by another. Convex scoring rules do the opposite because they place more weight on gains from high ranks. All else equal, they favor an option ranked $(n-1)$-th by one person and $(n+1)$-th by another over an alternative ranked $n$-th by both. To illustrate, suppose there are three options and two Stakeholders. Consider a rule that assigns scores of 1, $s$, and 0 to each Stakeholders favorite, second-favorite, and least-favorite option, respectively. In that case, an option that both Stakeholders rank second yields a total score of $2s$, while an option that one ranks first and the other ranks third yields a total score of 1. If $s>0.5$, the rule is concave and it selects the first option (the compromise) over the second. If $s<0.5$, the rule is convex and it rejects the compromise option.
According to our findings, the two most prevalent as-if aggregation criteria are the linear scoring rule (\emph{Borda}), which places equal weight on gains from high and low ranks, and a rule with nearly maximal concavity resembling \emph{antiplurality}, which minimizes the number of last-place ranks. Most subjects ($> 60\%$) employ strictly concave scoring rules. Choices conform with plurality rule (which is the most convex scoring rule), the Condorcet rule (also known as pairwise majority rule), supermajority rule, and the unanimity criteria for relatively few subjects. The classification's fit is excellent and clustering analysis identifies no important omitted patterns.


The prevalence of strictly concave scoring rules calls to mind the familiar cardinal concept of an additive, strictly concave Bergson-Samuelson social welfare function, which aggregates by applying a concave transformation $w$ to each individual's cardinal utility $U_n$ before summing them; that is, $W=\sum_n w(U_n)$. While related, these concepts are distinct. Even if social preferences over ordinal outcomes derive from cardinal inferences, the curvature of the scoring function would depend, in part, on the curvature of the mapping from ranks to inferred cardinal values. Similarly, the prevalence of highly concave scoring rules calls to mind the maximin criterion \citep{rawls1971theory}, a familiar cardinal concept that also focuses on the less fortunate. Concave scoring rules, however, judge misfortune according to an individual's own rankings within a menu, whereas maximin judges misfortune according to interpersonal comparisons.

In answer to the second question, we find that social preferences over ordinal outcomes entail stable structural elements. Specifically, our classifications are highly predictive of choices out of sample, even across domains. However, there is also a meaningful degree of domain dependence: subjects prefer slightly more concave scoring rules when distributing work tasks than when aggregating political preferences.

In answer to the third question, we find that people use ordinal ranks to draw inferences about cardinal utilities, but also care about them intrinsically. Two findings support the importance of cardinal inferences. First, best-fit scoring parameters are correlated with direct measures of Planners’ beliefs about cardinal outcomes. Second, the cardinal inference hypothesis correctly predicts the ways in which Planner’s choices respond to information about the rankings of unavailable options. We establish the intrinsic importance of ordinal outcomes by studying treatments for which we disable the cardinal inference mechanism. In these treatments, distributions of monetary payoffs, which we reveal to the Planner, serve as the social alternatives. Planners’ choices are sensitive to changes in menus that alter the profile of ranks associated with each alternative without changing the profile of cardinal payoffs. Moreover, Planners are willing to choose alternatives that yield strictly inferior cardinal payoff distributions (in the sense of first-order dominance) to achieve more favorable profiles of Stakeholder ranks.


To answer to the fourth question, we run supplemental experiments with general population samples, wherein social choices determine the allocation of a contribution over well-known charities. We find that the distributions of aggregation preferences in the U.S. and Sweden, countries with divergent political and social traditions, are remarkably similar, and both resemble the distribution for the student sample used in our main experiment.\footnote{\citet{politiciansProject} provide further evidence or robustness by comparing distributions of best-fit social choice rules for elected representatives in German federal and state parliaments and the German general population.} Moreover, we find suggestive evidence that the use of more concave scoring rules in experimental decisions correlates with a preference for electing compromise candidates.


Our paper contributes primarily to the literature on social preferences \citep[reviewed in][]{fehrCharness} by focusing on judgments involving the aggregation of ordinal outcomes. The prior literature contains hints about the nature of those judgments, for example regarding \emph{last place aversion} \citep{Kuziemkoetal.LastPlaceAversion, Camereretal.Replicability, MartinangeliWindsteiger}, but little more. One exception is \citet{KaraSertel},which uses hypothetical choices over abstract options for four preference profiles to determine the best fit among three ordinal aggregation rules. Unlike the current paper, they did not examine the relation between ordinal and cardinal aggregation. Furthermore, our analysis uses far more exhaustive lists of rules and preference profiles, studies real choices, checks for the presence of omitted rules, tests out-of-sample predictive accuracy, and examines stability across domains and cultures. Another exception is \citet{HerreinerPuppe}, who study bargaining in settings where parties only learn the ordinal rankings associated with each alternative. The study finds that some people forego Pareto improvements to reduce differences between the ranks parties assign to the selected outcome, possibly because they use ranks as proxies for monetary value. These results are related to our finding concerning the popularity of concave scoring rules. However, \citet{HerreinerPuppe}'s bargaining setting implicates selfishness and strategic concerns; unlike our study, they do not focus on disinterested judgments concerning aggregation. Indeed, even in the cardinal domain, most of the literature studies self-other trade-offs rather than other-other trade-offs; see, for example, \citet{JacksonYariv.Experiment, EngelmannStrobel, BoltonOckenfelsComment}.

Our paper also contributes to a large literature on social choice. Instead of asking which criteria people \emph{ought} to use when aggregating ordinal preferences \citep[in the tradition of][]{Arrow1950}, we investigate the criteria they \emph{actually} use. Our results imply that most people violate Arrow’s Independence of Irrelevant Alternatives when making choices for groups.\footnote{\citet{DaviesShah} also document violations of IIA when subjects aggregate others' preferences.}  By interpreting a social choice rule as an expression of social preferences rather than as a procedure for group decision making, we depart from an important branch of this literature, beginning with \cite{Gibbard} and \cite{Satterthwaite}, that studies implementability. While issues of manipulability inevitably arise in practice, they pertain to the constraints that appear in mechanism design problems, rather than to the objective functions decision makers seek to maximize subject to those constraints. It is the latter we seek to illuminate. In contrast, studies that elicit subjects' preferences over voting procedures (such as \citet{EngelmannGruener}, \citet{HoffmanRenes}, and \citet{Engelmannetal.MinorityProtection}) seek to illuminate judgments that account for all practical considerations impacting a procedure's performance. Our findings are nevertheless relevant for this branch of social choice theory because the rules that best describe our subjects’ ordinal aggregation preferences are known to be easily manipulable. Thus, the pursuit of procedures that are more difficult to manipulate may render outcomes less normatively appealing to most of the affected parties.\footnote{Knowing that people favor the normative values underlying highly manipulable social choice rules helps to explain why Planners sometimes defect from announced incentive-compatible decision criteria after group members have submitted their preferences \citep[see][for evidence]{Featherstone.RankEfficiency}. Many of the rules that mirror our subjects' social preferences are used in practice despite their susceptibility to manipulation. The Econometric Society uses the Borda rule to elect Officers and Council \citep{econometricSociety}. The Borda rule also determines Slovenian parliamentary elections \citep{Toplak} and elections within the Irish Green Party \citep{Baker}. Swiss parliamentary elections incorporate disapproval votes which follow the spirit of the antiplurality rule \citep{PortmannStojanovic}. Several nations use multistage methods for presidential elections \citep[see, e.g., ][]{Richie}. The Debian Project uses a Condorcet extension for all decision-making by committees and plebiscites \citep{Debian}.}


Finally, our paper contributes to the literature on \emph{positive welfare economics}, which uses empirical methods to determine how people evaluate the well-being of other individuals and groups \citep[see, e.g.,][]{Konow.reviewFairness, Gaertner.HandbookChapter, GaertnerSchokkaert.Book, Andreonietal.WhenFairIsntFair, Almaasetal., AmbuehlBernheimOckenfels, ArrietaBolte}.

The remainder of the paper is organized as follows. Section \ref{sec:conceptualFramework} lays out our main strategy for understanding the criteria people implicitly use to make decisions on behalf of groups when they only have ordinal information about others' preferences. Section \ref{sec:design} details our experimental design. Section \ref{sec:analysis} provides our classification results and conducts robustness analyses. Section \ref{contextDependence} documents the structural stability and context dependence of social preferences over ordinal outcomes. Section \ref{sec:ordinalCardinal} examines whether subjects draw cardinal inferences from ordinal information, and whether they care about ordinal outcomes intrinsically. Section \ref{genPop} examines the robustness of our results using Swedish and US general population samples. Section \ref{sec:conclusion} concludes.


\section{Conceptual framework}\label{sec:conceptualFramework}


We are concerned with settings in which a decision maker, the \emph{Planner}, must make a selection from a set of $K$ social options, $\mathcal{A}$, on behalf of $N$ \emph{Stakeholders}. Each option has direct consequences for the Stakeholders but not for the Planner. We assume throughout that each Stakeholder $i$ has a strict linear preference ordering $\succ_i$ over $\mathcal{A}$. Before making a decision, the Planner learns the group's ordinal \emph{preference profile}, $P=(\succ_1, \ldots, \succ_N)$. We interpret \emph{social choice rules} as expressions of Planners' social preferences. Any such rule provides a complete account of the Planner's choices for every preference profile; in other words, it is a mapping $R$ from preference profiles into nonempty subsets of $\mathcal{A}$ (\emph{best choices}). A \emph{resolute} rule admits no ties, in the sense that it maps every profile to a single option. An \emph{irresolute} rule maps at least one preference profile to a non-singleton set.

It bears emphasis that, in this article, we do \emph{not} interpret social choice rules as \emph{procedures} for group decision making. While any given social choice rule, such as a voting rule, may have a procedural flavor, we do not contend that, in making decisions for their assigned groups, our subjects consciously apply such procedures. Nor does our experiment speak to questions concerning attributes of these procedures, such as procedural fairness, other than the outcomes they produce. Our approach parallels consumer theory, which uses utility functions to summarize preference relations, but does not assert that consumers choose by computing the maxima of those functions.

The task of identifying the rule that best describes a Planner's aggregation preferences is challenging because the set of possibilities is astronomically large.\footnote{To illustrate, consider a simple setting with 5 Stakeholders and 3 alternatives. In that case, there are $7^{42}=3.1\times 10^{35}$ possible social choice rules that satisfy \emph{anonymity} (all Stakeholders treated symmetrically) and \emph{neutrality} (all social options treated symmetrically), of which $3.5\times 10^{28}$ exhibit no Pareto violations.}
Therefore, our analysis proceeds in three steps. First, based on the literature, we identify a reasonably large collection of social choice rules encompassing the most plausible alternatives. Second, we classify Planners according to the pre-specified rules that most closely match their actual choices. Third, we deploy clustering analysis to determine whether our pre-specified rules omit empirically important alternatives.


The literature sorts social choice rules into three broad classes. Members of the first class, \emph{Scoring rules}, assign points to ranks and select all point-maximizing alternatives. The \emph{Borda rule}, for instance, assigns $f(k_i) = N - k_i$ points to an alternative if stakeholder $i$ ranks it $k_i$-th out of $N$ alternatives, and selects the alternative for which the sum of points, $\Sigma_{i=1}^N (N- k_i)$, is highest. We call a scoring rule \emph{convex} or \emph{concave} depending on the curvature of the mapping $f$ from ranks to scores. Convex rules place emphasis on ensuring stakeholders obtain highly ranked alternatives, which gives them a ``winner-take-all'' flavor. In contrast, concave rules place emphasis on preventing stakeholders from receiving alternatives with low ranks, which means they encode a preference for compromise. The most convex rule assigns points only if stakeholders obtain their most preferred alternatives, which means it maximizes the number of first-place ranks (\emph{plurality rule}). The most concave rule assigns a point to all alternatives except for the least preferred one, which means it minimizes the number of last-place ranks (\emph{antiplurality rule}). The Borda rule is linear.


The second class consists of rules that generalize the well-known \emph{Condorcet} criterion. Each such rule involves a binary relation that deems alternative $A$ to be better than alternative $B$ if the fraction of stakeholders who prefer $A$ over $B$ is at least $q$ for some $q\geq \frac{1}{2}$. Setting $q=\frac{1}{2}$ yields the Condorcet relation; setting $q=1$ yields the unanimity relation. The rule then selects maximal elements according to that relation (for example, a Condorcet winner in the case of $q=\frac{1}{2}$).  A well-known limitation of rules in this class is that maximal elements may not exist (the Condorcet paradox). \emph{Condorcet extensions} select the maximal elements when they exist, and differ according to the selections they make when there is no such element \citep[see, e.g.,][]{ComSocHandbook}.

The third class consists of \emph{multistage rules}, which winnow down the set of alternatives in a series of steps. They introduce almost limitless possibilities, inasmuch as they can, in principle, apply different criteria in each step. We will focus on \emph{scoring runoffs}, which repeatedly eliminate the alternative with the lowest score until only one option remains \citep[see][]{Freemanetal.AxiomaticRunoff}.


When classifying Planners according to the pre-specified rules that most closely match their actual choices, we focus primarily on choice problems involving 5 Stakeholders and 3 options. This focus offers three advantages.


First, 5-Stakeholder 3-option problems are the simplest and hence most cognitively manageable settings that provide adequate scope for differentiation among a broad collection of rules.

Second, with 3 options, we can associate each scoring rule with a single parameter, $s\in [0,1]$, representing the score assigned to the middle alternative (after normalizing so that highest and lowest ranked alternatives receives scores of one and zero, respectively). This property facilitates the interpretation of our results. Specifically, the function relating ranks to weights is concave when $s>1/2$ and convex when $s<1/2$. In this context, concavity (convexity) implies that replacing a third-place rank with a second-place rank is more (less) valuable than replacing a second-place rank with a first-place rank. In other words, concave scoring rules codify an aversion to giving Stakeholders their least favorite of the three alternatives, while convex scoring rules codify an attraction to giving Stakeholders their favorite alternatives.


\begin{table}[htbp!]
\caption{Three-alternative profiles and choices of selected aggregation rules  \label{scoreIdentifyingProfiles} \label{threeOptionProfiles}}
\begin{center}
\begin{footnotesize}
\setlength{\tabcolsep}{2pt}
\vspace{-0.4cm}
\begin{tabular}{ccccccccccccccccc}
(1) & \multicolumn{5}{c}{(2)} & (3) & (4) & (5) & (6) & (7) & (8) & (9) & (10) & (11) & (12) & (13) \\
\toprule
Index & \multicolumn{5}{c}{Profile}   & \multicolumn{10}{c}{Rule predictions} \\
\cmidrule(r){7-17}
& & & & & & Scoring cutoff & \multicolumn{5}{c}{Scoring rules, $s\in$} & Runoff cutoff & \multicolumn{3}{c}{Runoff rules, $s\in$} & Condorcet \\
\cmidrule(r){8-12} \cmidrule(r){14-16}
& & & & & & $\bar{s}$ & $\{0\}$ & $(0,\bar{s})$ & $\{\bar{s}\}$ & $(\bar{s}, 1)$ & $\{1\}$ & $\hat{s}$ & $[0,\hat{s})$ & $\{\hat{s}\}$ & $(\hat{s}, 1]$ & \\
\\
{1}$^*$ & B & B & A & A & A & 1/3 & A & A & \{A,B\} & B & B & - & A & A & A & A\\
 & C & C & B & B & B & & & & & & & & & & & \\
 & A & A & C & C & C & & & & & & & & & & & \medskip \\
2 & B & C & C & A & A & 1/3 & \{A,C\} & A & \{A,B\} & B & B & 1/4 & A & A & B & B\\
 & A & B & B & B & B & & & & & & & & & & & \\
 & C & A & A & C & C & & & & & & & & & & & \medskip \\
{3}$^*$ & A & A & A & B & B & 1/2 & A & A & \{A,B\} & B & B & 1 & A & B & - & A\\
 & C & B & B & C & C & & & & & & & & & & & \\
 & B & C & C & A & A & & & & & & & & & & & \medskip \\
{4}$^*$ & B & B & A & A & A & 1/2 & A & A & \{A,B\} & B & B & - & A & A & A & A\\
 & A & C & B & B & B & & & & & & & & & & & \\
 & C & A & C & C & C & & & & & & & & & & & \medskip \\
5 & C & A & A & A & B & 1/2 & A & A & \{A,B\} & B & B & - & A & A & A & A\\
 & B & B & B & B & C & & & & & & & & & & & \\
 & A & C & C & C & A & & & & & & & & & & & \medskip \\
{6}$^*$ & C & C & A & A & A & 3/5 & A & A & \{A,B\} & B & B & - & A & A & A & A\\
 & B & B & B & B & B & & & & & & & & & & & \\
 & A & A & C & C & C & & & & & & & & & & & \medskip \\
7 & C & A & A & A & B & 2/3 & A & A & \{A,B\} & B & B & 1 & A & B & - & A\\
 & B & C & B & B & C & & & & & & & & & & & \\
 & A & B & C & C & A & & & & & & & & & & & \medskip \\
{8}$^*$ & C & A & A & A & B & 2/3 & A & A & \{A,B\} & B & B & - & A & A & A & A\\
 & B & B & B & B & A & & & & & & & & & & & \\
 & A & C & C & C & C & & & & & & & & & & & \medskip \\
{9}$^*$ & A & A & A & C & C & 3/4 & A & A & \{A,B\} & B & B & 1 & A & B & - & A\\
 & C & B & B & B & B & & & & & & & & & & & \\
 & B & C & C & A & A & & & & & & & & & & & \medskip \\
10 & B & A & A & A & A & 3/4 & A & A & \{A,B\} & B & B & - & A & A & A & A\\
 & C & B & B & B & B & & & & & & & & & & & \\
 & A & C & C & C & C & & & & & & & & & & & \medskip \\
{11}$^*$ & C & A & A & A & A & 4/5 & A & A & \{A,B\} & B & B & - & A & A & A & A\\
 & B & B & B & B & B & & & & & & & & & & & \\
 & A & C & C & C & C & & & & & & & & & & & \medskip \\
12 & C & B & B & A & A & 0 & - & - & \{A,B\} & B & B & 1 & A & B & - & \{A,B,C\}\\
 & A & C & C & B & B & & & & & & & & & & & \\
 & B & A & A & C & C & & & & & & & & & & & \medskip \\
13 & B & C & C & A & A & 1/2 & \{A,C\} & \{A,C\} & \{A,B,C\} & B & B & 1/2 & A & \{A,B,C\} & B & \{A,B,C\}\\
 & A & B & B & C & B & & & & & & & & & & & \\
 & C & A & A & B & C & & & & & & & & & & & \medskip \\
14 & C & B & B & A & A & - & \{A,B\} & \{A,B\} & \{A,B\} & \{A,B\} & \{A,B\} & - & A & A & A & A\\
 & A & C & A & B & B & & & & & & & & & & & \\
 & B & A & C & C & C & & & & & & & & & & & \medskip \\
15 & B & C & C & A & A & 0 & - & - & \{A,C\} & A & \{A,B\} & - & A & A & A & A\\
 & A & B & A & B & B & & & & & & & & & & & \\
 & C & A & B & C & C & & & & & & & & & & & \medskip \\
16 & C & B & B & A & A & 0 & - & - & \{A,B\} & A & A & - & A & A & A & A\\
 & A & C & A & B & C & & & & & & & & & & & \\
 & B & A & C & C & B & & & & & & & & & & & \medskip \\
17 & B & C & C & A & A & 0 & - & - & \{A,C\} & A & A & - & A & A & A & A\\
 & A & A & A & B & B & & & & & & & & & & & \\
 & C & B & B & C & C & & & & & & & & & & & \medskip \\

\bottomrule
\end{tabular}
\end{footnotesize}
\end{center}
\vspace{-0.3cm}
\scriptsize
\textbf{\textit{Notes:}} Each profile is displayed as a 3$\times$5-matrix. Columns correspond to Stakeholders and rows to preference ranks. A Stakeholder's first, second, and third-ranked alternatives are listed in the first, second, and third rows, respectively.  For Condorcet-cyclical profiles, we indicate the set of options in the top-cycle.  For decisions in the political domain, we only use the profiles indicated with an asterisk.
\end{table}


Third, with 5 Stakeholders and 3 options, we can in principle investigate choices exhaustively on the entire preference domain (42 distinct strict preference profiles) by eliciting a choice for every conceivable profile. In practice, many of the 42 preference profiles provide little or no discernment among rules, so we omit them from the experiment. For example, if all Stakeholders have the same ranking, the best social choice is obvious.


Our analysis is based on the 17 discerning profiles listed in Table \ref{threeOptionProfiles},\footnote{Appendix \ref{excludedProfiles} lists the omitted profiles and exhibits their limited ability to distinguish among rules.} which also shows how these profiles differentiate among familiar social choice rules. For example, if a subject uses a scoring rule, we can determine the parameter $s$ from the choices they make for profiles 1 through 11. To understand this point, first consider profile 11. Because all Stakeholders rank option $B$ second, its score is $S_s(B) = 5s$. Four Stakeholders rank option $A$ first and one ranks it third, so its score is $S_s(A) = 4$. Option $C$ is rank-dominated by $A$, so no scoring rule will select it. Therefore, the subject will choose option $B$ if $S_s(B) > S_s(A)$, or equivalently, $s > 0.8$, and will choose option $A$ if $s < 0.8$. If $s=0.8$, the subject is indifferent between options $A$ and $B$. Now consider profile 6, which differs from profile 11 only in that Stakeholder 2's preferences between $A$ and $C$ are reversed. Reasoning as before, we see that scoring rules select $B$ over $A$ when $s > 0.6$.
Indeed, for each of these eleven profiles, there is a threshold $\bar{s}$ at which the optimal choice for a scoring rule with parameter $s$ switches from the alternative listed in column 7 to the alternative listed in column 5. These thresholds divide the interval $[0,1]$ into a sequence of subintervals with boundaries in the set $\mathcal{C} = \{0, \frac{1}{3}, \frac{1}{2}, \frac{3}{5}, \frac{2}{3}, \frac{3}{4}, \frac{4}{5}, 1\}$. The subject's decisions place their scoring parameter in one of these intervals. In some cases, we can also distinguish choices corresponding to scoring parameters at the boundaries of these intervals.\footnote{When a scoring parameter coincides with an interval boundary $s\in\mathcal{C}$, the set of options selected for a given preference profile is the union of those selected with scoring parameter $s-\epsilon$ and those selected with scoring parameter $s+\epsilon$ for $\epsilon >0$ sufficiently small. Accordingly, scoring rules with parameters $s\in\mathcal{C}$ are less resolute (produce more ties) than scoring rules with parameter $s\notin\mathcal{C}$.}

The remaining profiles provide further scope for differentiating among social choice rules. For example, because profiles 12 and 13 exhibit Condorcet cycles, all three options are best choices according to the  top-cycle Condorcet extension, but not according to many other rules.

Each social choice rule generates a ``fingerprint" of selections across the 17 preference profiles; see Figure \ref{fingerprints}. Each column in the figure represents a 5-Stakeholder 3-option preference profile, which we identify in the first panel. For example, the first column corresponds to the profile wherein three Stakeholders prefer $A$ to $B$ to $C$, while two prefer $B$ to $C$ to $A$. Each subsequent panel corresponds to a specified social choice rule; it shows the options that rule selects for each preference profile. Each subject's sequence of choices also generates a fingerprint. We will assign each subject to the rule with the fingerprint that matches her own most closely.


\begin{figure}[htb!]
\caption{Fingerprints of an example selection of ordinal aggregation rules.  \label{fingerprints}}
\begin{center}
\begin{tikzpicture}\matrix (matrix) [matrix of nodes, row sep=-1\pgflinewidth, column sep=-1\pgflinewidth, nodes={align=center, minimum size=0.7cm, minimum height=0.45cm, inner sep=0, anchor=center, text=black, font=\normalfont}, column 1/.style={nodes={anchor=west,align=left}}] {
 & |[fill=none, draw=none]| & |[fill=none, draw=none]| & |[fill=none, draw=none]| & |[fill=none, draw=none]| & |[fill=none, draw=none]| & |[fill=none, draw=none]| & |[fill=none, draw=none]| & |[fill=none, draw=none]| & |[fill=none, draw=none]| & |[fill=none, draw=none]| & |[fill=none, draw=none]| & |[fill=none, draw=none]| & |[fill=none, draw=none]| & |[fill=none, draw=none]| & |[fill=none, draw=none]| & |[fill=none, draw=none]| & |[fill=none, draw=none]| & |[fill=none, draw=none]|\\
 & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]|\\
|[fill=none, draw=none, text=black, text width = 2cm, anchor=west, align=left]| Index & |[fill=none, draw=none]| 1 & |[fill=none, draw=none]| 2 & |[fill=none, draw=none]| 3 & |[fill=none, draw=none]| 4 & |[fill=none, draw=none]| 5 & |[fill=none, draw=none]| 6 & |[fill=none, draw=none]| 7 & |[fill=none, draw=none]| 8 & |[fill=none, draw=none]| 9 & |[fill=none, draw=none]| 10 & |[fill=none, draw=none]| 11 & |[fill=none, draw=none]| 12 & |[fill=none, draw=none]| 13 & |[fill=none, draw=none]| 14 & |[fill=none, draw=none]| 15 & |[fill=none, draw=none]| 16 & |[fill=none, draw=none]| 17\\
 & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]| & |[fill=none, draw=none, minimum height = 0.25cm]|\\
|[fill=none, draw=none, text=black, text width = 2cm, anchor=west, align=left]| $A>B>C$& |[fill=none, draw=none]| 3& |[fill=none, draw=none]| 2& |[fill=none, draw=none]| 2& |[fill=none, draw=none]| 3& |[fill=none, draw=none]| 3& |[fill=none, draw=none]| 3& |[fill=none, draw=none]| 2& |[fill=none, draw=none]| 3& |[fill=none, draw=none]| 2& |[fill=none, draw=none]| 4& |[fill=none, draw=none]| 4& |[fill=none, draw=none]| 2& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 2& |[fill=none, draw=none]| 2& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 2\\
|[fill=none, draw=none, text=black, text width = 2cm, anchor=west, align=left]| $A>C>B$& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 0\\
|[fill=none, draw=none, text=black, text width = 2cm, anchor=west, align=left]| $B>A>C$& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 1\\
|[fill=none, draw=none, text=black, text width = 2cm, anchor=west, align=left]| $B>C>A$& |[fill=none, draw=none]| 2& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 2& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 2& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 0\\
|[fill=none, draw=none, text=black, text width = 2cm, anchor=west, align=left]| $C>A>B$& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 2\\
|[fill=none, draw=none, text=black, text width = 2cm, anchor=west, align=left]| $C>B>A$& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 2& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 2& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 2& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 2& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 1& |[fill=none, draw=none]| 0& |[fill=none, draw=none]| 0\\
|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|\\
|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|\\
|[fill=none, draw=none, text=black, text width = 2cm, align=center]| \qquad A & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!33,draw=black]| & |[fill=airforceblue!33,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]|\\
|[fill=none, draw=none, text=black, text width = 2cm, align=center]| \qquad B & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!33,draw=black]| & |[fill=airforceblue!33,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]|\\
|[fill=none, draw=none, text=black, text width = 2cm, align=center]| \qquad C & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!33,draw=black]| & |[fill=airforceblue!33,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]|\\
|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|\\
|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|\\
|[fill=none, draw=none, text=black, text width = 2cm, align=center]| \qquad A & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!50,draw=black]|\\
|[fill=none, draw=none, text=black, text width = 2cm, align=center]| \qquad B & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!0,draw=black]|\\
|[fill=none, draw=none, text=black, text width = 2cm, align=center]| \qquad C & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!50,draw=black]|\\
|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|\\
|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|\\
|[fill=none, draw=none, text=black, text width = 2cm, align=center]| \qquad A & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!33,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]|\\
|[fill=none, draw=none, text=black, text width = 2cm, align=center]| \qquad B & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!33,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]|\\
|[fill=none, draw=none, text=black, text width = 2cm, align=center]| \qquad C & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!33,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]|\\
|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|\\
|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|\\
|[fill=none, draw=none, text=black, text width = 2cm, align=center]| \qquad A & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]|\\
|[fill=none, draw=none, text=black, text width = 2cm, align=center]| \qquad B & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!100,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!50,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]|\\
|[fill=none, draw=none, text=black, text width = 2cm, align=center]| \qquad C & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]| & |[fill=airforceblue!0,draw=black]|\\
};
\hhlline{matrix}{5}{18};
\hhlline{matrix}{11}{18};
\vllline{matrix}{11}{2};
\vllline{matrix}{11}{3};
\vllline{matrix}{11}{4};
\vllline{matrix}{11}{5};
\vllline{matrix}{11}{6};
\vllline{matrix}{11}{7};
\vllline{matrix}{11}{8};
\vllline{matrix}{11}{9};
\vllline{matrix}{11}{10};
\vllline{matrix}{11}{11};
\vllline{matrix}{11}{12};
\vllline{matrix}{11}{13};
\vllline{matrix}{11}{14};
\vllline{matrix}{11}{15};
\vllline{matrix}{11}{16};
\vllline{matrix}{11}{17};
\vllline{matrix}{11}{18};
\vrlline{matrix}{11}{18};
\path (matrix-1-2.north west) rectangle (matrix-1-18.south east) node[midway] {Preference profile};
\path (matrix-12-2.north west) rectangle (matrix-12-18.south east) node[midway] {Condorcet};
\path (matrix-17-2.north west) rectangle (matrix-17-18.south east) node[midway] {Scoring, $s=0$};
\path (matrix-22-2.north west) rectangle (matrix-22-18.south east) node[midway] {Scoring, $s=1/2$};
\path (matrix-27-2.north west) rectangle (matrix-27-18.south east) node[midway] {Scoring, $s=1$};
\end{tikzpicture}\\
\end{center}
\footnotesize
\textbf{\textit{Notes}}: Each column corresponds to a three-alternative preference profile. The profiles appear in the top panel: each cell shows the number of Stakeholders with the preference ranking indicated to the left of the row. In the second through fifth panels, we use dark shading if the rule chooses the corresponding option, and no shading if the rule does not choose the option. Non-resolute rules choose more than one option for some profiles. We use intermediate shades to indicate the number of tied options.
\end{figure}


Our main classification results encompass 23 $\emph{benevolent}$ social choice rules (meaning that they satisfy the Pareto criterion). We start with the 15 distinguishable rules that emerge from the scoring method. We can differentiate between $s = \frac{1}{2}$ (linearity), 5 ranges of scoring parameters in the strictly convex region and 10 in the strictly concave range. Our data can also differentiate among 5 scoring runoff rules corresponding, respectively, to values of $s$ in the following ranges: $[0, \frac{1}{4}]$, $(\frac{1}{4},\frac{1}{2})$, $s=\frac{1}{2}$, $(\frac{1}{2},1)$, and $s=1$.\footnote{We implement scoring runoff rules as follows. In the first stage, we calculate the score associated with each option and drop the option with the lowest score. We then choose the majority-preferred option from the remaining alternatives. (Because this step involves at most two options, majority rule coincides with all scoring rules.) If the lowest two options are tied, we drop both of them. If all three options are tied, the runoff rule selects all of them. Three-way ties occur only for profile 13, and only for the scoring runoff rule with $s=\frac{1}{2}$. We identify the set of distinguishable scoring runoff rules using a brute-force computer script.}
Finally, we can distinguish three social choice rules that emerge from the $p$-supermajority top-cycle method. These rules correspond to values of $p$ in the following ranges: $[\frac{1}{2}, \frac{3}{5}]$, $(\frac{3}{5}, \frac{4}{5}]$, and $(\frac{4}{5}, 1]$. Henceforth we adopt a slight abuse of terminology and call these rules Condorcet, Supermajority, and Unanimity, respectively. While our basic classification does not include other Condorect extensions, we conduct robustness analyses to determine whether this omission is material.

Because we distinguish between social choice rules based on their fingerprints, larger differences between fingerprints facilitate more reliable classifications. Appendix \ref{app:distanceBetweenRules} shows, for each pair of social choice rules, the number of preference profiles (out of the 17 we use for identification) for which their implications differ. On average, pairs of rules differ on 8.58 of the 17 profiles, and nearly 90\% of pairs differ on three or more profiles.

It is worth emphasizing that we selected the set of 17 profiles based on their contributions to identification rather than the frequencies with which they arise in the wild. Our approach assumes that people apply stable aggregation principles regardless of whether a profile is relatively common or uncommon. We corroborate this assumption by examining in-sample fit, out-of-sample predictive power, and out-of-context stability.


\section{Experimental design}\label{sec:design}


\paragraph{Overview}
We assign subjects to one of two roles: each \emph{Planner} (`she') chooses alternatives that potentially affect groups of five \emph{Stakeholders} (`he'). The only purpose of including Stakeholders in the experiment is to ensure that the Planners' decisions are consequential. Choice problems fall into two domains, task assignment (\emph{work}) problems and budget allocation (\emph{political}) problems. For each domain, the Planner views a sequence of preference profiles and, in each instance, selects one of several alternatives. We match one out of every four Planners, selected at random, with a real group of Stakeholders. The actual preference profile for that group is among the ones that Planner considers, but is not identified as such. Although we only implement decisions that pertain to actual Stakeholder groups, from each Planner's perspective any one of her decisions could turn out to be a real choice. Consequently, as long as she cares about the Stakeholders to some degree, she has an incentive to reveal her aggregation preferences truthfully for every preference profile she encounters. We focus primarily on the work domain, for which we employ the 17 profiles shown in Table \ref{threeOptionProfiles}. We examine the political domain, for which we employ a smaller set of profiles, to evaluate context-dependence.

\paragraph{Tasks for the work domain}

For the work domain, Stakeholders are \emph{workers} recruited from Amazon Mechanical Turk. Each worker receives $\$15$ for correctly completing a single assigned task. The compensation is sufficiently high to ensure that any attrition is non-systematic.

There are five work tasks that resemble familiar activities for online workers, and that different workers might plausibly rank in different orders: image labeling, hate speech filtering, audio transcription, movie reviews classification, and a logic-based task. Appendix \ref{app:designDetails} provides details.

Workers reveal their preferences over tasks in a preliminary session. We ensure incentive compatibility by informing workers that their rankings determine the task they perform with 5\% probability,\footnote{We pre-select two tasks at random without telling the worker which they are, and assign the worker to the one he ranks higher.} and that some other process will determine their assigned tasks with 95\% probability. Because we leave that process unspecified, workers cannot report preferences strategically. After the Planners make their choices, workers complete their assigned tasks in a second session.


\paragraph{Social choice problems for the work domain}

A social alternative is a \emph{task assignment}: it assigns each of the five tasks to one of the five workers in a group.
Planners choose from menus of task assignments. For three-option problems, menus consist of three randomly selected task assignments.

For the main portion of our experiment, Planners are students at the University of Zurich and the Swiss Federal Institute of Technology. (We examine US and Swedish general population samples in Section \ref{genPop}). At the beginning of the experiment, they acquaint themselves with the work tasks by reading descriptions and performing abbreviated versions (except for the matching task). To make the allocation tasks less abstract, we advise Planners that Stakeholders are mTurkers and provide information about their hourly compensation in the experiment.

A Planner proceeds through several rounds of decision making for her group. In each round, she observes an ordinal preference profile for the task assignments. For Planners who are matched with actual workers, one of these corresponds to her workers' preferences over task assignments, which we infer from their elicited preferences over tasks. The Planner then chooses an assignment she considers best for the group. We also ask her to identify any alternative she considers just as good as the one she selects.

If the Planner knew everything about the task assignments, her decision might depend on whether she agreed or disagreed with Stakeholders' rankings. For example, she might place little weight on an expressed desire to complete the hate-speech filtering task. From our perspective, those paternalistic judgments would be confounds. To ensure that Planners cannot second-guess a worker's preferences, we limit the Planner's knowledge. Specifically, we show the Planner a menu of geometric symbols, explain that each represents a task assignment, and describe each worker's ranking of those assignments. However, we do not explain which worker performs which task in any given assignment.


Each Planner makes all decisions using one of several interfaces. The interface shown in Figure \ref{socialArchitectInterface0} represents each worker as a vertical bar. Within each bar, the geometric symbols representing the assignments are ordered according to the worker's preference, with the most preferred assignment on top. By clicking buttons, Planners can highlight or hide options. Highlighting an option makes the distribution of its ranks (the data that is crucial for scoring rules) readily apparent. When an option is hidden, the display re-positions the remaining assignments onto two lines. This feature makes pairwise preference counts (the  data that is crucial for Condorcet extensions) readily apparent. Additionally, Planners can hide or rearrange the workers, either by dragging and dropping them, or by clicking a button to shuffle them randomly. A possible concern is that the presentation of preference profiles may influence Planners' choices by highlighting particular information. To ensure robustness, we therefore vary the interface across Planners, using \emph{all} structurally distinct possibilities for displaying preference profiles in a two-dimensional table. Appendix \ref{additionalDisplayFormats} provides details.


\begin{figure}[htb!]
\caption{Planners' decision interface. \label{socialArchitectInterface0}}
\begin{center}
\begin{tabular}{|c|}
\hline \\ \medskip
\includegraphics[height = 0.7\textheight]{graphs/figure2.png}\\
\hline
\end{tabular}\\
\end{center}
\footnotesize \textbf{\textit{Notes:}} Subjects can drag and drop columns, click a button to shuffle columns, highlight choice options, hide choice options, and hide workers. If a subject hides an option, the remaining options are arranged on two rows regardless of their initial position. Symbols for options and colors for the bars representing each worker are randomly drawn each round. The order of workers is randomly drawn each round.
\end{figure}


We randomize symbols (representing options) and colors (representing workers) to ensure Planners do not conflate decisions across preference profiles. We also randomize the positions of all alternatives and of all workers in each round.

In addition to the 17 preference profiles shown in Table \ref{threeOptionProfiles}, each Planner who is matched to a real group of five workers views that group's actual preference profile and makes a choice, while other Planners view a randomly generated preference profile. Planners also make 10 additional decisions, randomly intermingled with the others; see Sections \ref{fourOptionMain} and \ref{unavailableAlternatives} for details. Altogether, Planners make choices for 28 preference profiles in the work domain.


\paragraph{The political domain}

To examine the consistency of Planners' aggregation preferences across domains, we also present them with decisions involving political contributions. In this domain, Stakeholders (\emph{citizens}) are voting-age members of the general Swiss population.\footnote{We recruited Stakeholders mostly through the survey company pollfish.com. We supplemented this sample by placing ads on Facebook and in the laboratory at the University of Zurich. Age and citizenship are self-reported.}  Planners, who are also voting-age Swiss nationals, direct a contribution of Fr. 30 (roughly \$33.90 at the time of the experiment) to one of the five largest Swiss political parties, as measured by the number of members in the Swiss National Council.

The procedures we use for political contributions are generally the same as for the work domain. Here it is especially important to emphasize that, for any given preference profile, Planners do not know which geometric symbol corresponds to which party, so they cannot impose their own political preferences.


To keep the length of the experiment manageable for our subjects, we use a smaller set of preference profiles for the political domain. Over 12 rounds, Planners view, in random order, the starred preference profiles in column 2 of Table \ref{threeOptionProfiles}, one additional profile (either the actual profile for her assigned citizen group or a randomly generated profile, depending on whether we assign her to a group), and a selection of four four-option profiles (see Section \ref{analysisClassSeparating}).


\paragraph{Additional elicitations} \label{additionalElicitations}


For the work domain, we ask Planners to predict the average reservation wages for the tasks workers rank first, second, and third, knowing only that rankings involve three tasks randomly selected from the five possibilities.
For the political domain, we ask Planners to predict the average Swiss citizen's willingness to pay to trigger or prevent the donation the citizen ranks first, second, and third.
As we discuss subsequently, we use these responses to assess the hypothesis that Planners aggregate ordinal preferences are based on implicit inferences about cardinal utility.

We elicit risk preferences using the method developed in \citet{HoltLaury}. We also elicit altruism toward workers through modified dictator games. In addition, subjects complete the four-item version of the Cognitive Reflection Test by \citet{CRTalternative}. For the last 143 subjects, we added four questions probing their knowledge of social choice theory.
Subjects also report a variety of personal characteristics. See Appendix \ref{additionalElicitations} for further details.


\paragraph{Implementation}

Subjects make all decisions within a domain (work vs. political) before proceeding to the next domain. We randomize the order of the two domains, as well as the order of preference profiles within each domain, at the individual level. To avoid nudging subjects to think about cardinal utility, we elicit risk aversion and beliefs about WTA/WTP after the preference aggregation tasks are complete.

A randomly selected decision (pertaining to WTP/WTA, risk preferences, or social preferences) from part C determines the Planner's own payment. As we have explained, we incentivize social choices in the sense that each one may be consequential for other participants, and we also incentivize the elicitation of workers' preferences.

The full instructions for Planners, which we present on-screen in English, appear in Appendix \ref{instructionsMain}. They require subjects to try out each option in the preference display (e.g., hide and highlight). Subjects must pass two multi-part comprehension checks to continue with the study.


\paragraph{Data}
Our analysis focuses on 405 subjects in the role of Planner. Subjects participated in 11
online sessions, supervised via video-conferencing software (Zoom), in January 2021. We
restricted the sample to Swiss citizens by checking each participants' government-issued
ID. All data and code are available in \citet{AmbuehlBernheimData}.

The median subject completed the session in 82 minutes and received Fr. 50 (worth approximately \$56.5 at the time of the experiment). Twelve potentially eligible subjects started the study but did not complete it. Only 5 of them did so after presenting a valid ID.
The median age is 23. Among the subjects who were asked, 7\% reported having taken a class that covered social choice theory, but only 1\%, 3\%, and 2\% could correctly name  Arrow's theorem, the Condorcet paradox, and the Borda rule, respectively. While our subject pool includes a higher proportion of women (61\%) than men and skews towards the political left (70\%,  15\%, and 14\% of subjects rate themselves as left, center, and right, respectively), we obtain similar results in general population samples; see Section \ref{genPop}.


\section{Classification results}\label{classification}\label{bayesClassifierIntroduction}\label{sec:analysis}

In this section, we present our main results concerning the prevalence of the various as-if social choice rules. We consider aggregate choice patterns followed by subject-level classifications.


\paragraph{Aggregate choice patterns}\label{aggregate}


Figure \ref{empiricalFingerprintAll} shows choice frequencies for each of the 17 three-option profiles in the work domain. Structure is readily apparent, such as the near-unanimous decisions for profiles 2, 16, and 17. Because nearly all subjects choose the option that is both rank-dominant and a Condorcet winner in profiles 16 and 17, we can infer that they act benevolently toward their groups. Notably, disagreements are common for the score-identifying profiles 3 to 11, indicating heterogeneity in aggregation preferences.


\begin{figure}[htbp!]
\caption{Distribution of choices in three-option problems \label{empiricalFingerprintAll}}
\begin{center}
\begin{tikzpicture}\matrix (matrix) [matrix of nodes, row sep=-1\pgflinewidth, column sep=-1\pgflinewidth, nodes={align=center, minimum size=0.8cm, minimum height = 0.5cm, inner sep=0, anchor=mid, text=black, font=\normalfont}, column 1/.style={nodes={anchor=mid}}, nodes in empty cells] {
|[fill=none, draw=none, text=black, text width = 2cm, anchor=mid, align=left]| Profile & |[fill=none,draw=none]| 1 & |[fill=none,draw=none]| 2 & |[fill=none,draw=none]| 3 & |[fill=none,draw=none]| 4 & |[fill=none,draw=none]| 5 & |[fill=none,draw=none]| 6 & |[fill=none,draw=none]| 7 & |[fill=none,draw=none]| 8 & |[fill=none,draw=none]| 9 & |[fill=none,draw=none]| 10 & |[fill=none,draw=none]| 11 & |[fill=none,draw=none]| 12 & |[fill=none,draw=none]| 13 & |[fill=none,draw=none]| 14 & |[fill=none,draw=none]| 15 & |[fill=none,draw=none]| 16 & |[fill=none,draw=none]| 17\\
|[fill=none, draw=none, text=black, text width = 2cm, anchor=mid, align=left]| Condorcet& |[fill=none,draw=none]| A& |[fill=none,draw=none]| B& |[fill=none,draw=none]| A& |[fill=none,draw=none]| A& |[fill=none,draw=none]| A& |[fill=none,draw=none]| A& |[fill=none,draw=none]| A& |[fill=none,draw=none]| A& |[fill=none,draw=none]| A& |[fill=none,draw=none]| A& |[fill=none,draw=none]| A& |[fill=none,draw=none]| ABC& |[fill=none,draw=none]| ABC& |[fill=none,draw=none]| A& |[fill=none,draw=none]| A& |[fill=none,draw=none]| A& |[fill=none,draw=none]| A\\
|[fill=none, draw=none, text=black, text width = 2cm, anchor=mid, align=left]| Scoring& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|\\
|[fill=none, draw=none, text=black, text width = 2cm, anchor=mid, align=left]| \quad $\bar{s}$ & |[fill=none, draw=none, text=black, anchor=mid, align=center]| $\nicefrac{1}{3}$& |[fill=none, draw=none, text=black, anchor=mid, align=center]| $\nicefrac{1}{3}$& |[fill=none, draw=none, text=black, anchor=mid, align=center]| $\nicefrac{1}{2}$& |[fill=none, draw=none, text=black, anchor=mid, align=center]| $\nicefrac{1}{2}$& |[fill=none, draw=none, text=black, anchor=mid, align=center]| $\nicefrac{1}{2}$& |[fill=none, draw=none, text=black, anchor=mid, align=center]| $\nicefrac{3}{5}$& |[fill=none, draw=none, text=black, anchor=mid, align=center]| $\nicefrac{2}{3}$& |[fill=none, draw=none, text=black, anchor=mid, align=center]| $\nicefrac{2}{3}$& |[fill=none, draw=none, text=black, anchor=mid, align=center]| $\nicefrac{3}{4}$& |[fill=none, draw=none, text=black, anchor=mid, align=center]| $\nicefrac{3}{4}$& |[fill=none, draw=none, text=black, anchor=mid, align=center]| $\nicefrac{4}{5}$& |[fill=none, draw=none, text=black, anchor=mid, align=center]| $0$& |[fill=none, draw=none, text=black, anchor=mid, align=center]| $\nicefrac{1}{2}$& |[fill=none, draw=none, text=black, anchor=mid, align=center]| -& |[fill=none, draw=none, text=black, anchor=mid, align=center]| $0$& |[fill=none, draw=none, text=black, anchor=mid, align=center]| $0$& |[fill=none, draw=none, text=black, anchor=mid, align=center]| $0$\\
|[fill=none, draw=none, text=black, text width = 2cm, anchor=mid, align=left]| \quad $s < \bar{s}$ & |[fill=none, draw=none, text=black, anchor=mid, align=center]| A & |[fill=none, draw=none, text=black, anchor=mid, align=center]| A$^{a)}$ & |[fill=none, draw=none, text=black, anchor=mid, align=center]| A & |[fill=none, draw=none, text=black, anchor=mid, align=center]| A & |[fill=none, draw=none, text=black, anchor=mid, align=center]| A & |[fill=none, draw=none, text=black, anchor=mid, align=center]| A & |[fill=none, draw=none, text=black, anchor=mid, align=center]| A & |[fill=none, draw=none, text=black, anchor=mid, align=center]| A & |[fill=none, draw=none, text=black, anchor=mid, align=center]| A & |[fill=none, draw=none, text=black, anchor=mid, align=center]| A & |[fill=none, draw=none, text=black, anchor=mid, align=center]| A & |[fill=none, draw=none, text=black, anchor=mid, align=center]| -$^{b)}$ & |[fill=none, draw=none, text=black, anchor=mid, align=center]| AC & |[fill=none, draw=none, text=black, anchor=mid, align=center]| AB & |[fill=none, draw=none, text=black, anchor=mid, align=center]| -$^{a)}$ & |[fill=none, draw=none, text=black, anchor=mid, align=center]| -$^{b)}$ & |[fill=none, draw=none, text=black, anchor=mid, align=center]| -$^{a)}$\\
|[fill=none, draw=none, text=black, text width = 2cm, anchor=mid, align=left]| \quad $s > \bar{s}$& |[fill=none, draw=none, text=black, anchor=mid, align=center]| B& |[fill=none, draw=none, text=black, anchor=mid, align=center]| B& |[fill=none, draw=none, text=black, anchor=mid, align=center]| B& |[fill=none, draw=none, text=black, anchor=mid, align=center]| B& |[fill=none, draw=none, text=black, anchor=mid, align=center]| B& |[fill=none, draw=none, text=black, anchor=mid, align=center]| B& |[fill=none, draw=none, text=black, anchor=mid, align=center]| B& |[fill=none, draw=none, text=black, anchor=mid, align=center]| B& |[fill=none, draw=none, text=black, anchor=mid, align=center]| B& |[fill=none, draw=none, text=black, anchor=mid, align=center]| B& |[fill=none, draw=none, text=black, anchor=mid, align=center]| B& |[fill=none, draw=none, text=black, anchor=mid, align=center]| B& |[fill=none, draw=none, text=black, anchor=mid, align=center]| B& |[fill=none, draw=none, text=black, anchor=mid, align=center]| AB& |[fill=none, draw=none, text=black, anchor=mid, align=center]| A$^{c)}$& |[fill=none, draw=none, text=black, anchor=mid, align=center]| A& |[fill=none, draw=none, text=black, anchor=mid, align=center]| A\\
|[fill=none, draw=none, text=black, text width = 2cm, anchor=mid, align=left]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|\\
|[fill=none, draw=none, text=black, text width = 2cm, anchor=mid, align=left]| Choice freq.& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|\\
|[fill=none, draw=none, text=black, text width = 2cm, anchor=mid, align=left]| \quad A& |[fill=airforceblue!13, draw=black, text=black]| 13& |[fill=airforceblue!2, draw=black, text=black]| 2& |[fill=airforceblue!36, draw=black, text=black]| 36& |[fill=airforceblue!37, draw=black, text=black]| 37& |[fill=airforceblue!24, draw=black, text=black]| 24& |[fill=airforceblue!38, draw=black, text=black]| 38& |[fill=airforceblue!56, draw=black, text=white]| 56& |[fill=airforceblue!60, draw=black, text=white]| 60& |[fill=airforceblue!72, draw=black, text=white]| 72& |[fill=airforceblue!70, draw=black, text=white]| 70& |[fill=airforceblue!70, draw=black, text=white]| 70& |[fill=airforceblue!8, draw=black, text=black]| 8& |[fill=airforceblue!15, draw=black, text=black]| 15& |[fill=airforceblue!62, draw=black, text=white]| 62& |[fill=airforceblue!93, draw=black, text=white]| 93& |[fill=airforceblue!98, draw=black, text=white]| 98& |[fill=airforceblue!99, draw=black, text=white]| 99\\
|[fill=none, draw=none, text=black, text width = 2cm, anchor=mid, align=left]| \quad B& |[fill=airforceblue!86, draw=black, text=white]| 86& |[fill=airforceblue!96, draw=black, text=white]| 96& |[fill=airforceblue!64, draw=black, text=white]| 64& |[fill=airforceblue!63, draw=black, text=white]| 63& |[fill=airforceblue!75, draw=black, text=white]| 75& |[fill=airforceblue!61, draw=black, text=white]| 61& |[fill=airforceblue!43, draw=black, text=black]| 43& |[fill=airforceblue!39, draw=black, text=black]| 39& |[fill=airforceblue!25, draw=black, text=black]| 25& |[fill=airforceblue!29, draw=black, text=black]| 29& |[fill=airforceblue!29, draw=black, text=black]| 29& |[fill=airforceblue!91, draw=black, text=white]| 91& |[fill=airforceblue!73, draw=black, text=white]| 73& |[fill=airforceblue!37, draw=black, text=black]| 37& |[fill=airforceblue!6, draw=black, text=black]| 6& |[fill=airforceblue!1, draw=black, text=black]| 1& |[fill=airforceblue!1, draw=black, text=black]| 1\\
|[fill=none, draw=none, text=black, text width = 2cm, anchor=mid, align=left]| \quad C& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!1, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!1, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!1, draw=black, text=black]| 1& |[fill=airforceblue!1, draw=black, text=black]| 1& |[fill=airforceblue!3, draw=black, text=black]| 3& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!1, draw=black, text=black]| 1& |[fill=airforceblue!1, draw=black, text=black]| 1& |[fill=airforceblue!13, draw=black, text=black]| 13& |[fill=airforceblue!1, draw=black, text=black]| 1& |[fill=airforceblue!1, draw=black, text=black]| 1& |[fill=airforceblue!1, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0\\
|[fill=none, draw=none, text=black, text width = 2cm]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|& |[fill=none, draw=none]|\\
};
\draw[thick]  (matrix-1-1.north west) -- (matrix-1-18.north east);
\draw  (matrix-2-1.north west) -- (matrix-2-18.north east);
\draw  (matrix-6-1.south west) -- (matrix-6-18.south east);
\draw[thick]  (matrix-12-1.south west) -- (matrix-12-18.south east);
\end{tikzpicture}
\end{center}
\footnotesize
$^{a)}$ For $s = 0$, \{A,C\} is selected. \\
$^{b)}$ For $s = 0$, \{A,B\} is selected. \\
$^{c)}$ For $s = 1$, \{A,B\} is selected. \medskip\\
\footnotesize
\textbf{\textit{Notes:}} Darker shades of blue indicate higher frequencies. The numerical percentage appears within each shaded cell. The profile numbering is the same as for Table \ref{threeOptionProfiles}.
\end{figure}


Patterns for profiles 1 and 2 anticipate our overall conclusions. If, just for the moment, we confine attention to scoring rules and the Condorcet top-cycle extension, we would infer that 86\% of subjects use scoring rules with $s > \frac{1}{3}$ (because they choose B in profile 1), that only 2-3\% of subjects use scoring rules with $s < \frac{1}{3}$ (because they choose A or C in profile 2), and that 10\% of subjects use the Condorcet rule (because the fraction of B increases from 86\% to 96\% between profiles 1 and 2). The third-option comparisons inherent in scoring rules also point to widespread violations of Arrow's Independence of Irrelevant Alternatives axiom.

Choices for profiles 12 and 13 corroborate the low prevalence of the Condorcet rule with the top-cycle extension, in that we see little indication of indeterminacy despite the presence of cycles. For profile 12, 91\% of subjects agree on option B, which is rank-dominant, and for profile 13, 73\% of subjects select option B, which minimizes the number of last-place ranks.

Choices for profiles 3, 4, and 5 corroborate the prevalence of concave scoring rules, as 64\%, 63\%, and 75\% of subjects select option B, respectively. In contrast, all convex scoring rules and the Condorcet rule select option A, which is chosen by 36\%, 37\%, and 24\% of subjects, respectively. Either pattern is consistent with the Borda rule, which is only partially resolute for these profiles.


\label{HowClassify}\label{bayesClassifier}

\paragraph{Classification methods}
Next we assign each subject to the social choice rule that best matches their selections. We employ a Bayes classifier \citep[see, e.g.,][]{Webb.NaiveBayes}, which assigns each subject the social choice rule with the greatest posterior probability conditional on the subject's observed choices.
Critically, this method gives irresolute rules less ``credit" (in terms of the increment to the posterior probability) than resolute rules when both turn out to be consistent with a given choice. Without this feature, a classification method would favor irresolute rules artificially. To compute posterior probabilities, we make four assumptions \citep[analogously to][]{CostaGomesetal.ECMA}: (i) The prior distribution over pre-specified rules is uniform.
(ii) For each of the 17 profiles, the subject follows her assigned rule with probability $1-\epsilon$, and uniformly randomizes over the three options with probability $\epsilon$. The prior distribution of $\epsilon$ is uniform over $[0,1]$. (iii) Decision errors are independent across preference profiles. (iv) When a rule is irresolute, the subject randomizes uniformly and independently over the prescribed choices.\footnote{Because our displays present alternatives and workers in random order, redrawn in each round, even positional criteria such as always choosing the option on the left will yield uniform distributions.} We use the popular Maximum A Posteriori (MAP) criterion, which assigns each subject the rule $R^*$ and noise level $\epsilon^*$ that maximize the posterior probability, $P(R, \epsilon | c)$, where $c$ is their choice vector.
When more than one rule maximizes the Bayesian posterior, we assign the subject to a maximizing rule at random. Appendix \ref{BayesMonteCarlo} derives the classifier formally and shows that it performs well in simulations.


We assign each subject to one of 39 rules. To the set of 23 benevolent social choice rules discussed in Section \ref{sec:conceptualFramework}, we add malevolent versions that apply the benevolent versions after inverting the Stakeholder's preference rankings. For technical reasons, the Bayes classifier will never select a benevolent rule with $s \in \{\frac{3}{5}, \frac{4}{5}\}$ or a malevolent rule with $s \in \{\frac{1}{3}, \frac{3}{5}, \frac{2}{3}, \frac{3}{4}, \frac{4}{5}\}$.\footnote{Each of these rule differs from nearby scoring rules on exactly one profile. Moreover, for the single differentiating profile, each prescribes the union of the options selected when the scoring parameter is slightly larger, and when it is slightly smaller. Because our classifier always favors resoluteness over irresoluteness when both possibilities are consistent with the same observation, it will never select such a rule over all nearby scoring rules.
} Any subject who employs one of these seven rules is classified as using either a slightly higher or slightly lower value of $s$.
For similar reasons, any subject with $s=1$ tends to be classified as using $\frac{4}{5}<s<1$, so our exposition merges the estimates of these two categories.
Differentiating among all 46 rules requires us to use the subjects' expressions of indifference in addition to their choices. While those expressions do not have consequences, each concerns a consequential choice the subject has just made, and there is no reason to answer incorrectly. Appendix \ref{app:indifferenceData} shows that we obtain similar results.


\paragraph{Results}\label{bayesClassifierResults}

Figure \ref{barChart} displays our main classification results. Four features merit emphasis.

First, choices for a clear majority of subjects are consistent with \emph{strictly concave scoring rules} ($\frac{1}{2} < s \leq 1$; 62.5\%). In contrast, choice patterns consistent with strictly \emph{convex} scoring rules ($s<\frac{1}{2}$) are uncommon. The most convex rule (plurality) describes the choices of surprisingly few subjects (1\%).

Second, we assign the largest number of subjects to the Borda rule and to near-antiplurality (with $\nicefrac{4}{5}<s\leq 1$). Together, these two cases account for over 40 percent of all subjects.

Third, our classification procedure assigns fewer than 5\% of subjects to the Condorcet rule, and almost none to other $p$-supermajority rules (supermajority and unanimity). A small minority follow scoring runoff rules (4.9\%). Because our pre-specified rules contain only one Condorcet extension (top cycle), these classifications could in principle understate the prevalence of choices consistent with the Condorcet class. We therefore enlarge the set of pre-specified rules to include \emph{all conceivable} Condorcet extensions, and reclassify subjects using our first procedure.\footnote{Because all but two of the 17 profiles admit a Condorcet Winner, there are only nine resolute Condorcet extensions. As long as we include all of them, there is no need to add any irresolute Condorcet extensions, because our classifier never assigns a subject to a less resolute rule that is equally consistent with the subject's choices.} The classifier assigns 8.9\% of subjects to a Condorcet extension, a modest increase, some of which would occur by chance. It is unlikely that the low prevalence of Condorcet reasoning is a statistical or experimental artifact, because the Condorcet rule coincides on a large number of profiles with convex scoring rules, which are also unpopular. Appendix \ref{analysisClassSeparating} provides further corroboration of this finding based on four-option problems.

Fourth, we see little  evidence that subjects are malevolent or lazy. The fraction of subjects assigned to malevolent rules is de minimis. Lazy subjects would be inclined to select the most easily implementable rule. Plurality rule is arguably the least cognitively demanding alternative, followed by plurality runoff, yet the vast majority of subjects evidently find the distributional implications of both unappealing. With the exception of antiplurality, the concave scoring rules, which describe the majority of our subjects, embody more nuanced and complex judgments. An analysis of decision times corroborates these assessments: decision making is relatively rapid for subjects assigned to plurality rule and relatively slow for those assigned to scoring rules with $0.5 \leq s \leq 1$ (see Appendix \ref{appendixResponseTimes}).

Appendix \ref{analysisPreferenceDisplay} shows that these results are robust across our three formats for displaying preference profiles to subjects.


\begin{figure}[htb]
\caption{Best fitting pre-specified rules\label{barChart}}
\begin{center}
\includegraphics[width = 0.9\textwidth]{graphs/figure4}
\end{center}
\footnotesize
\textit{\textbf{Notes:}} The random-choice benchmark for malevolent rules exceeds the range of the graph (44.8\%).
\end{figure}


\paragraph{Random benchmark}
We draw statistical inferences by comparing our classification to a random-choice benchmark. We generate 4000 artificial subjects whose simulated selections for each profile are uniformly distributed across options. We use the Bayes classifier to assign each simulated subject to a pre-specified social choice rule. Finally, we draw 1,000 bootstrap samples of 405 simulated subjects and compute the 1st and 99th percentiles for the percentages of simulated subjects assigned to each rule.


Figure \ref{barChart} plots, for each rule, the mean fraction of simulated subjects assigned to that rule, as well as the 1st and 99th percentiles of the corresponding distribution. The average fraction of simulated subjects assigned to a malevolent rule is 44.8\%, which exceeds the range of the figure.\footnote{This benchmark deviates from 50\% because our selection of profiles is not random.} Turning to benevolent rules, we see that the fraction of subjects assigned to every weakly concave scoring rule except $\frac{3}{4}<s<\frac{4}{5}$ exceeds the 99th percentile of the random benchmark. In contrast, the fractions of subjects assigned to the two most convex scoring rules ($s<\frac{1}{3}$) fall short of the 1st percentile of the random benchmark. While the fraction of subjects assigned to the Condorcet rule is small, it is larger than we would observe by chance, a conclusion for which we find corroboration in Appendix \ref{analysisClassSeparating}. Finally, scoring runoff rules, supermajority, and unanimity are no more prevalent (and in some cases significantly less prevalent) than for the random-choice benchmark.


\paragraph{Goodness of fit}

The average value of the estimated noise parameter $\epsilon^*$ is 0.10, which signifies that the average subject chose randomly in just 1.7 of the 17 preference profiles. A whopping 42.0\% of our subjects fit their assigned rule perfectly. For the remainder, the mean noise parameter is 0.16, equivalent to choosing randomly for 2.7 of the 17 preference profiles.
In contrast, the average estimated noise parameter for our simulated random-choice data is 0.47, and 0.58 when excluding simulated subjects classified as supermajority or unanimity (10 of 17 profiles).\footnote{The mean estimated noise parameter in the random-choice data never equals its actual value (unity) because some random-choice sequences will always match some choice rules relatively well.}


\begin{figure}[ht!]
\caption{Empirical fingerprints of classified subjects\label{empiricalFingerprints}}
\begin{center}
\begin{tikzpicture}\matrix (matrix) [matrix of nodes, row sep=-1\pgflinewidth, column sep=-1\pgflinewidth, nodes={align=center, minimum size=0.8cm, minimum height = 0.7cm, inner sep=0, anchor=center, text=black, font=\normalfont}, column 1/.style={nodes={anchor=west,align=left}}] {
|[fill=none, draw=none, text=black, text width = 1cm, anchor=west, align=left]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]|\\
|[fill=none, draw=none, text=black, text width = 1cm, anchor=west, align=left]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]|\\
|[fill=none, draw=none, text=black, text width = 1cm, anchor=west, align=left]| \quad A& |[fill=airforceblue!8, draw=black, text=black]| 8& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!66, draw=black, text=white]| 66& |[fill=airforceblue!55, draw=black, text=white]| 55& |[fill=airforceblue!40, draw=black, text=black]| 40& |[fill=airforceblue!84, draw=black, text=white]| 84& |[fill=airforceblue!94, draw=black, text=white]| 94& |[fill=airforceblue!95, draw=black, text=white]| 95& |[fill=airforceblue!95, draw=black, text=white]| 95& |[fill=airforceblue!95, draw=black, text=white]| 95& |[fill=airforceblue!97, draw=black, text=white]| 97& |[fill=airforceblue!1, draw=black, text=black]| 1& |[fill=airforceblue!28, draw=black, text=black]| 28& |[fill=airforceblue!66, draw=black, text=white]| 66& |[fill=airforceblue!99, draw=black, text=white]| 99& |[fill=airforceblue!98, draw=black, text=white]| 98& |[fill=airforceblue!100, draw=black, text=white]| 100\\
|[fill=none, draw=none, text=black, text width = 1cm, anchor=west, align=left]| \quad B& |[fill=airforceblue!92, draw=black, text=white]| 92& |[fill=airforceblue!99, draw=black, text=white]| 99& |[fill=airforceblue!34, draw=black, text=black]| 34& |[fill=airforceblue!45, draw=black, text=black]| 45& |[fill=airforceblue!60, draw=black, text=white]| 60& |[fill=airforceblue!16, draw=black, text=black]| 16& |[fill=airforceblue!5, draw=black, text=black]| 5& |[fill=airforceblue!5, draw=black, text=black]| 5& |[fill=airforceblue!5, draw=black, text=black]| 5& |[fill=airforceblue!5, draw=black, text=black]| 5& |[fill=airforceblue!3, draw=black, text=black]| 3& |[fill=airforceblue!99, draw=black, text=white]| 99& |[fill=airforceblue!48, draw=black, text=black]| 48& |[fill=airforceblue!34, draw=black, text=black]| 34& |[fill=airforceblue!1, draw=black, text=black]| 1& |[fill=airforceblue!2, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0\\
|[fill=none, draw=none, text=black, text width = 1cm, anchor=west, align=left]| \quad C& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!1, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!1, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!24, draw=black, text=black]| 24& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0\\
|[fill=none, draw=none, text=black, text width = 1cm, anchor=west, align=left]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]|\\
|[fill=none, draw=none, text=black, text width = 1cm, anchor=west, align=left]| \quad A& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!50, draw=black, text=white]| 50& |[fill=airforceblue!50, draw=black, text=white]| 50& |[fill=airforceblue!50, draw=black, text=white]| 50& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!33, draw=black, text=black]| 33& |[fill=airforceblue!50, draw=black, text=white]| 50& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100\\
|[fill=none, draw=none, text=black, text width = 1cm, anchor=west, align=left]| \quad B& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!50, draw=black, text=white]| 50& |[fill=airforceblue!50, draw=black, text=white]| 50& |[fill=airforceblue!50, draw=black, text=white]| 50& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!33, draw=black, text=black]| 33& |[fill=airforceblue!50, draw=black, text=white]| 50& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0\\
|[fill=none, draw=none, text=black, text width = 1cm, anchor=west, align=left]| \quad C& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!33, draw=black, text=black]| 33& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0\\
|[fill=none, draw=none, text=black, text width = 1cm, anchor=west, align=left]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]|\\
|[fill=none, draw=none, text=black, text width = 1cm, anchor=west, align=left]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]|\\
|[fill=none, draw=none, text=black, text width = 1cm, anchor=west, align=left]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]|\\
|[fill=none, draw=none, text=black, text width = 1cm, anchor=west, align=left]| \quad A& |[fill=airforceblue!5, draw=black, text=black]| 5& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!10, draw=black, text=black]| 10& |[fill=airforceblue!14, draw=black, text=black]| 14& |[fill=airforceblue!1, draw=black, text=black]| 1& |[fill=airforceblue!1, draw=black, text=black]| 1& |[fill=airforceblue!15, draw=black, text=black]| 15& |[fill=airforceblue!11, draw=black, text=black]| 11& |[fill=airforceblue!33, draw=black, text=black]| 33& |[fill=airforceblue!14, draw=black, text=black]| 14& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!4, draw=black, text=black]| 4& |[fill=airforceblue!3, draw=black, text=black]| 3& |[fill=airforceblue!63, draw=black, text=white]| 63& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100\\
|[fill=none, draw=none, text=black, text width = 1cm, anchor=west, align=left]| \quad B& |[fill=airforceblue!95, draw=black, text=white]| 95& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!90, draw=black, text=white]| 90& |[fill=airforceblue!86, draw=black, text=white]| 86& |[fill=airforceblue!96, draw=black, text=white]| 96& |[fill=airforceblue!99, draw=black, text=white]| 99& |[fill=airforceblue!85, draw=black, text=white]| 85& |[fill=airforceblue!89, draw=black, text=white]| 89& |[fill=airforceblue!62, draw=black, text=white]| 62& |[fill=airforceblue!86, draw=black, text=white]| 86& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!96, draw=black, text=white]| 96& |[fill=airforceblue!96, draw=black, text=white]| 96& |[fill=airforceblue!37, draw=black, text=black]| 37& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0\\
|[fill=none, draw=none, text=black, text width = 1cm, anchor=west, align=left]| \quad C& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!3, draw=black, text=black]| 3& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!5, draw=black, text=black]| 5& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!1, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0\\
|[fill=none, draw=none, text=black, text width = 1cm, anchor=west, align=left]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]| & |[fill=none,draw=none]|\\
|[fill=none, draw=none, text=black, text width = 1cm, anchor=west, align=left]| \quad A& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!50, draw=black, text=white]| 50& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100\\
|[fill=none, draw=none, text=black, text width = 1cm, anchor=west, align=left]| \quad B& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!100, draw=black, text=white]| 100& |[fill=airforceblue!50, draw=black, text=white]| 50& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0\\
|[fill=none, draw=none, text=black, text width = 1cm, anchor=west, align=left]| \quad C& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0\\
};
\path (matrix-1-2.north west) rectangle (matrix-1-18.south east) node[midway] {\textbf{A. Subjects classified as Borda ($s=0.5$)}};
\path (matrix-2-2.north west) rectangle (matrix-2-18.south east) node[midway] {Empirical choices};
\path (matrix-6-2.north west) rectangle (matrix-6-18.south east) node[midway] {Rule prediction};
\path (matrix-11-2.north west) rectangle (matrix-11-18.south east) node[midway] {\textbf{B. Subjects classified as near-antiplurality  ($\frac{4}{5}<s<1$)}};
\path (matrix-12-2.north west) rectangle (matrix-12-18.south east) node[midway] {Empirical choices};
\path (matrix-16-2.north west) rectangle (matrix-16-18.south east) node[midway] {Rule prediction};
\end{tikzpicture}
\end{center}
\footnotesize
\textbf{\textit{Notes:}} In both panels, the top half displays the choices by subjects assigned to a certain rule. The bottom half displays the choices the rule predicts. Darker shades of grey indicate higher frequencies. The numerical percentage appears within each shaded cell. For the theoretical rule, we take the frequency of each best option to be 50\% for a two-way tie and 33\% for a three-way tie.
\end{figure}

A more detailed picture of goodness-of-fit emerges from comparisons between the theoretical fingerprints associated with particular rules and the empirical fingerprints of the subjects assigned to those rules. The top half of Figure \ref{empiricalFingerprints}, panel A, plots the distribution of choices, by profile, for subjects assigned to the Borda rule. The bottom half of the same panel shows the Borda prescriptions. The fit is plainly tight. While these subjects rarely depart from the Borda rule, they do deviate somewhat from uniform resolution of ties. Panel B of Figure  \ref{empiricalFingerprints} provides analogous information for the near-antiplurality rule, which differs from Borda on 10 of the 17 profiles. The fit is also good, if slightly less tight.


Overall, the 21 pre-specified benevolent social choice rules fit the data remarkably well given the astronomical number of possible rules in our setting.
To further allay possible concerns about overfitting, we show in Section \ref{contextDependence} that our classification predicts well out of sample.

\paragraph{Completeness of the classification} \label{clustering}

To determine whether our classification analysis excludes empirically important rules, we look for clusters of subjects whose choices more closely resemble each others' than any of the pre-specified possibilities. Our approach is similar to that of \citet{CostaGomesCrawford}. To identify clusters, we use the $k$-modes clustering algorithm, which is an adaptation of the well-known $k$-means method to categorical data \citep{Huang.kModes}.
We use our pre-specified rules to create additional potential cluster modes that are always present as the algorithm iterates. Because we do not use the indifference data for this exercise, we include every resolute version of each pre-specified rule. Following \citet{CostaGomesCrawford}, if a subject is equidistant from a pre-specified rule and an endogenous cluster, we assign them to the pre-specified rule. We run the algorithm multiple times, gradually increasing the fixed number of endogenous clusters $k$ until no further non-trivial clusters emerge.

We find only two non-trivial endogenous clusters. The larger includes just under 5\% (19 of 405) of subjects; the smaller includes fewer than 2\% (7 of 405) of subjects. The rule for the larger one is a one-profile deviation from near-antiplurality: on profile 9, it selects option A rather than B. This profile is the only one for which near-antiplurality selects an option ranked last by some Stakeholder, and not ranked first by any other Stakeholder. Appendix \ref{app:clusteringDetails} presents additional details.

\paragraph{Corroboration based on class-separating preference profiles}\label{fourOptionMain}

We corroborate our findings on the importance of scoring rules relative to Condorcet extensions using two additional preference profiles, each of which cleanly separates these classes. Both of these profiles involve four options rather than three. Critically, each has a unique Condorcet winner that is rank-dominated. \emph{Proper} scoring rules (which do not assign the same score to any two ranks) never select rank-dominated options. Only 7.4\% of choices are consistent with a preference for the Condorcet winner, which is in line with the finding in Figure \ref{barChart}. A related analysis corroborates the importance of scoring rules relative to the plurality runoff rule. Appendix \ref{app:fourOptionProfiles} provides details.


\section{Structural stability versus context dependence}\label{contextDependence}

Does our classification capture subjects' structural aggregation principles, contextual judgments, or both? To answer this question, we first compare choices across our two domains. Because subjects made decisions for fewer preference profiles in the political domain, we re-estimate the rule distribution for the work domain using only the 7 profiles common to both domains. Doing so reduces the number of rules we can distinguish. Specifically, we can no longer identify scoring rules involving any value of $s$ that lies at the boundary between two intervals.\footnote{Without using subjects' expressions of indifference, we cannot distinguish any such scoring rule unless the rules for the two adjacent intervals differ on at least two preference profiles. Appendix \ref{rulesDistPolit} shows the distance between all pairs of pre-specified rules in the political domain.} Moreover, the fingerprints of scoring rules with $s < \frac{1}{3}$ (including plurality rule), the Condorcet rule, and all scoring runoff rules with $s<1$ coincide for these seven profiles. For our current purposes, we will call this group the ``plurality-equivalent rules."


\begin{figure}[htbp!]
\caption{Comparison between work and political domains \label{workVpolitical}}
\vspace{-0.3cm}
\begin{center}
\includegraphics[width = 0.8\textwidth]{graphs/figure6} \\
\end{center}
\vspace{-0.8cm}
\footnotesize
\textbf{\textit{Notes:}} Classifications are based on the same set of seven preference profiles for both domains. The category `plurality equivalent' includes scoring rules with $s < \nicefrac{1}{3}$, the Condorcet rule, and all scoring runoff rules with $s < 1$. Whiskers denote 95\% confidence intervals.
\end{figure}

Figure \ref{workVpolitical} shows the differences in the resulting distributions of classifications for the work and political domains. Systematic differences are immediately apparent. Within the political domain, subjects' choices are less frequently consistent with strictly concave scoring rules, especially $\frac{4}{5}<s\leq 1$, and more frequently consistent with the Borda rule or a plurality-equivalent rule. The difference is highly statistically significant ($p<0.01$, Wilcoxon signed-rank test applied to the 400 subjects classified as following a benevolent rule in both domains).

Crucially, these differences are of limited magnitude. The increase in the frequency of plurality-equivalent rules, for instance, is on the order of 5.7\%, while the decrease in the frequency of all strictly concave rules is 16.5\%. The absence of larger differences suggests a meaningful degree of structural stability.


To evaluate the stability of aggregation preferences more comprehensively, we examine the out-of-sample and cross-domain predictive power of our classifications, using all profiles available in each domain. For each evaluation, we designate a training set of preference profiles which we use to classify Planners, as well as a test set which we use to evaluate predictive success. We score predictions as follows. If the actual choice lies outside the predicted best-choice set, the score is zero. If it lies within the predicted best-choice set, the score is 1, $\frac{1}{2}$, or $\frac{1}{3}$ depending on whether there are 1, 2, or 3 best options. With this scoring system, if underlying choices are in fact uniformly random, the average score will be $\frac{1}{3}$ irrespective of the sizes of predicted best-choice sets. If all of our individual-level assignments were correct, we would obtain a score of 0.924 in the work domain and 0.955 in the political domain, given the estimated distributions of pre-specified rules for each domain. These numbers are less than 1 because some rules are irresolute, and we do not seek to predict how subjects resolve ties. We conduct both within- and cross-domain predictions. For the within-domain predictions, we use the leave-one-out approach. For cross-domain predictions, we designate all profiles in one domain as the training set and all profiles in the other domain as the test set.\footnote{The training set does not always contain sufficient information to distinguish among all rules that have different predictions in the test set. In these cases, we treat all rules that maximize the Bayesian posterior in the training set for a given subject as equally likely and select one at random.}


\begin{table}[htb!]
\caption{Out-of-sample predictive power of the Bayesian classification \label{predictivePower}}
\begin{center}
\begin{tabular}{l>{\centering\arraybackslash}p{2.2cm}>{\centering\arraybackslash}p{2.2cm}}
\toprule
& (1) & (2)  \\
Dependent variable & \multicolumn{2}{c}{Fraction of correct predictions} \\
 & \multicolumn{2}{c}{(weighted by resoluteness)} \\
 \cmidrule(r){2-3}
Test domain & {Work} & Politics  \\
\midrule
\\ \textbf{Predictions} \\ Training domain	 & 		 & 		\\
\quad Work	 & 	0.853	 & 	0.758	\\
\quad Politics	 & 	0.805	 & 	0.755	\\
\\ \textbf{Benchmarks}	 & 		 & 		\\
1. Theoretical maximum	&	0.924	&	0.955	\\
2. Random choice	 & 	0.333	 & 	0.333	\\
3.a Estimated rule frequencies, work domain	 & 		 & 		\\
\quad Mean	 & 	0.736	 & 	0.617	\\
\quad 99th percentile	 & 	0.754	 & 	0.645	\\
3.b Estimated rule frequencies, political domain	 & 		 & 		\\
\quad Mean	 & 	0.716	 & 	0.642	\\
\quad 99th percentile	 & 	0.736	 & 	0.669	\\
\bottomrule
\end{tabular}
\end{center}
\footnotesize
\textbf{\textit{Notes:}}  Within-domain predictions are based on a leave-out-one approach.
\end{table}


As shown in Table \ref{predictivePower}, the average predictive accuracy scores for our four evaluations range from 0.755 (politics to politics) to 0.853 (work to work). In all four cases, predictive performance is far superior to the random-choice benchmark of $\frac{1}{3}$. However, that benchmark is a low bar. Our predictions would outperform it even if they merely captured generally reasonable tendencies such as avoiding dominated alternatives. We construct a more demanding benchmark by randomly assigning subjects to aggregation rules using the estimated distribution of rules for each domain. We obtain the distributions for the benchmarks by drawing 1000 bootstrap samples of these random assignments. In each case, we find that the average score for our predictions exceeds the 99th percentile of the benchmarks' distribution by a wide margin. Considering that the theoretical maximum for the work domain is 0.924, the within-domain score of 0.853 implies that our classification achieves 62\% of the potential gain in predictive accuracy over the work-domain benchmark (0.736). Similarly, considering that the theoretical maximum for the political domain is 0.955, the within-domain score of 0.755 achieves 36\% of the potential gain in predictive accuracy over the political-domain benchmark (0.642). The superior performance within the work domain is driven by the inclusion of profiles for which the choices of many popular rules coincide, which makes them easier to predict.


We conclude that the assigned rules capture important aspects of subjects' general aggregation preferences. While we find some degree of context-specificity, the relative accuracy of the cross-domain predictions reassures us that ordinal aggregation entails stable structural elements.


\section{Why does ordinal information matter?}\label{sec:ordinalCardinal}

What is the role of ordinal information in Planners' social preferences? Do Planners seek to aggregate cardinal utilities by inferring them from preference ranks when cardinal information is unavailable \citep[in the spirit of][]{Apesteguiaetal.Justice}, or do they have intrinsic preferences over ordinal outcomes such as a desire to avoid alternatives that some Stakeholders rank last? Subsection \ref{sec:cardinalInferences} demonstrates that the cardinal inference mechanism plays an important role; Subsection \ref{cardinalExperiment} shows that Planners also care about ordinal outcomes intrinsically.

\subsection{Do Planners draw cardinal inferences?}\label{sec:cardinalInferences}


We use two strategies to investigate the role of cardinal inferences. First, we ask whether Planners' beliefs about Stakeholders' cardinal valuations, which we elicited, predict their ordinal aggregation principles. Second, we test an implication of the cardinal inference hypothesis concerning the effects of an informational intervention.

\paragraph{Relation between as-if scoring parameters and beliefs about cardinal valuations}


A money-metric utilitarian should choose the alternative $x$ that minimizes $\sum_n u_{R(n,x)}$, where $R(n,x)$ is the rank Stakeholder $n$ assigns to option $x$ and $u_r$ is the average dollar-equivalent reservation wage the Planner associates with a Stakeholder's $r$-th ranked task. This objective is equivalent to applying the scoring rule that assigns, respectively, one point, $\tilde{s}$ points, and 0 points for Stakeholders who receive their top-ranked, second-ranked, and third-ranked options, where
$\tilde{s} = (u_3 - u_2) / (u_3 - u_1)$.


Because we elicited Planners' beliefs about Stakeholders' reservation wages, $u_r$, we can examine the relation between the synthetic utilitarian scoring parameter $\tilde{s}$ and the best-fitting scoring parameter $s$ across subjects.
To avoid selection effects, we assign every subject to a scoring rule even if some other rule is a better fit. We exclude 9 subjects in the work domain and 6 in the political domain who assign lower reservation wages to lower-ranked alternatives, as well as three who are assigned to malevolent rules. We estimate OLS regressions using interval midpoints whenever best-fit scoring parameters are interval-identified. We pool data across domains to increase statistical power. The specification includes fixed effects for the decision domain, preference profile presentation modes, and the order in which the subject made decisions about the work domain and the political domain. We also cluster standard errors at the subject level where appropriate.


In column 1 of Table \ref{scoringCorrelates}, the coefficient of the synthetic scoring rule is positive and statistically significant, indicating that the as-if scoring rules are partly informed by subjects' beliefs about cardinal utilities. Because our measurements of these beliefs, and hence of the synthetic scoring parameters, are potentially noisy, that coefficient may be biased. Column 2 limits the influence of outliers in $\tilde{s}$ by substituting subjects' percentile rank within the $\tilde{s}$ distribution. While this change in the predictor variable prevents a direct comparison with the previous estimate, the statistical significance of the resulting coefficient estimate increases substantially, as expected. Columns 3 and 4 replicate these regressions including controls for risk tolerance and altruism. They yield similar estimates of the coefficients on the synthetic scoring parameter and its percentile rank. The coefficient of risk tolerance is negative, as one would expect if Planners maximize the expected utility of the representative Stakeholder assessed behind a ``veil of ignorance," assuming random assignment of rankings to Stakeholders. In that case, greater risk tolerance implies greater convexity of the scoring rule. We also find that more altruistic Planners tend to use more concave scoring rules. Additional analysis shows that the relation between synthetic and best-fitting scoring parameters is stronger for the political domain than the work domain, though not to a statistically significant extent (see Appendix \ref{app:syntheticScoringRules}).


\begin{table}[htb!]
\caption{Relation between best-fitting scoring parameters and beliefs about reservation prices. \label{scoringCorrelates}}
\begin{normalsize}
\setlength{\tabcolsep}{3pt}
\begin{center}
\begin{tabular}{lccccccc} \toprule
 & (1) & (2) & (3) & (4) \\
VARIABLE & \multicolumn{4}{c}{Estimated scoring parameter $s$} \\
\midrule
            &               &               &               &               \\

\textit{Scoring parameter $\tilde{s}$}&               &               &               &               \\
\quad Level &       0.115   &               &       0.102   &               \\
            &     (0.056)   &               &     (0.056)   &               \\
\quad \%-rank&               &       0.062   &               &       0.055   \\
            &               &     (0.023)   &               &     (0.023)   \\
\textit{Risk tolerance}&               &               &               &               \\
\quad Level &               &               &      -0.717   &               \\
            &               &               &     (0.314)   &               \\
\quad \%-rank&               &               &               &      -0.054   \\
            &               &               &               &     (0.028)   \\
\textit{Altruism}&               &               &               &               \\
\quad Level &               &               &       0.178   &               \\
            &               &               &     (0.223)   &               \\
\quad \%-rank&               &               &               &       0.048   \\
            &               &               &               &     (0.028)   \\
Political domain&      -0.111   &      -0.111   &      -0.111   &      -0.112   \\
            &     (0.010)   &     (0.010)   &     (0.010)   &     (0.010)   \\
\hline
Observations & 792 & 792 & 792 & 792 \\
Subjects & 405 & 405 & 405 & 405 \\
\bottomrule
\end{tabular}
\end{center}
\end{normalsize}
\footnotesize
\textit{\textbf{Notes:}} Parameters estimated with OLS using midpoint values of best-fit score. Regressions include all subjects with monotonic beliefs about reservation prices. Risk tolerance is measured as the certainty equivalent of an equiprobable win 40 / win 10 lottery implied by individual-level CRRA estimates. Altruism is measured as the maximum amount of Swiss Francs the Planner is willing to give up to increase each stakeholder's payoff by \$2. For subjects with multiple switches in the risk preference or altruism elicitation, the corresponding variable is set to the mean value among the other subjects. We also include two indicator variables for the presence of multiple switching points, one for each characteristic. All regressions control for the type of preference profile presentation and for whether the political domain was displayed before or after the work domain. Standard errors are clustered by subject.
\end{table}


\paragraph{Informational interventions}\label{unavailableAlternatives}

Next, we test an implication of the cardinal information mechanism. To appreciate this implication, consider the preference profiles depicted in column 1 of Table \ref{iiaProfiles}. The one in panel B is the same as the one in Panel A, except that it also lists a third option, $X$, that is not available. One Stakeholder prefers $X$ to $A$ while none prefer it to $B$. If Planners' choices depend on cardinal inferences, this additional information should reduce $A$'s attractiveness to the Planner. In contrast, if Planner's choices are based solely on the ordinal ranks among \emph{available} options, the information should have no effect. In fact, we see that the fraction selecting A falls sharply from 0.993 to 0.427, and the change is highly statistically significant (Panel D). Likewise, under the cardinal inference hypothesis, one would expect that listing the unavailable option ($X$) shown in columns 2 and 3 of Panel B would shift Planners' choices away from $A$ in these columns. The data strongly support both predictions. While an aversion to last-place ranks among $listed$ options would also predict the effects in columns 1 and 2, it would not predict the effect in column 3.


In all three columns, the option $X$ is relatively unattractive: when we make it available, very few Planners' choose it (Panel C). Consequently, if Planners use ordinal information \emph{only} to make cardinal inferences, $X$'s availability (as opposed to its mere listing) should leave the choice frequencies for the other options unchanged. We obtain mixed evidence concerning this implication. The choice distribution changes significantly for column 2 ($p<0.03$), but not for columns 1 and 3. The former finding raises the possibility that, in addition to making cardinal inferences based on ordinal information, Planners may care intrinsically about the ordinal positions of available options. We explore this possibility more thoroughly in the next subsection.

The comparison of Panels A and C also reveals a striking violation of the choice axiom known as Sen's $\alpha$. According to that axiom, removing an unchosen option from a menu should not change the chosen option. In fact, choices change dramatically.


\begin{table}[tbh!]
\caption{Information treatments \label{iiaProfiles}}
\setlength{\tabcolsep}{3pt}
\begin{center}
\begin{tabularx}{\textwidth}[t]{lccc}
\toprule
& (1) & (2) & (3) \\


\multicolumn{4}{c}{\textbf{A. Baseline profiles}} \\ \quad Profiles
&
\begin{tabular}[t]{cccccc}
\midrule
B & B & A & A & A \\
A & A & B & B & B \\
\midrule
\end{tabular}
&
\begin{tabular}[t]{cccccc}
\midrule
B & A & A & A & B \\
A & B & B & B & A \\
\midrule
\end{tabular} &
\begin{tabular}[t]{cccccc}
\midrule
A & A & B & B & C \\
B & B & A & C & A \\
C & C & C & A & B \\
\midrule
\end{tabular}
\\ \\ \quad
Choice distribution
&
\begin{tabular}[t]{cc}
A & B  \\
0.993 & 0.007   \\
\end{tabular}
&
\begin{tabular}[t]{cc}
A & B \\
0.993 & 0.007 \\
\end{tabular}
&
\begin{tabular}[t]{ccc}
A & B & C \\
0.622 & 0.368 & 0.010 \\
\end{tabular}
\\ \\


\multicolumn{4}{c}{\textbf{B. Adding information about an unavailable option $X$}} \\ \quad Profiles &
\begin{tabular}[t]{cccccc}
\midrule
B & B & A & A & A \\
A & \gray \st{X} & B & B & B \\
\gray \st{X} & A & \gray \st{X} & \gray \st{X} & \gray \st{X} \\
\midrule
\end{tabular}
&
\begin{tabular}[t]{cccccc}
\midrule
\gray \st{X} & A & A & A & B \\
B & B & B & B & \gray \st{X} \\
A & \gray \st{X} & \gray \st{X} & \gray \st{X} & A \\
\midrule
\end{tabular}&
\begin{tabular}[t]{cccccc}
\midrule
A & A & B & B & C \\
B & B & A & C & \gray \st{X} \\
C & C & C & \gray \st{X} & A \\
\gray \st{X} & \gray \st{X} & \gray \st{X} & A & B \\
\midrule
\end{tabular}


\\ \\ \quad
Choice distribution &
\begin{tabular}[t]{cc}
A & B  \\
0.427 & 0.573  \\
\end{tabular}
&
\begin{tabular}[t]{cc}
A & B \\
0.358 & 0.642 \\
\end{tabular}
&
\begin{tabular}[t]{ccc}
A & B & C \\
0.217 & 0.674 & 0.109 \\
\end{tabular}
\\
\\

\multicolumn{4}{c}{\textbf{C. Adding an available option $X$}} \\ \quad Profiles
&
\begin{tabular}[t]{cccccc}
\midrule
B & B & A & A & A \\
A & X & B & B & B \\
X & A & X & X & X \\
\midrule
\end{tabular}
&
\begin{tabular}[t]{cccccc}
\midrule
X & A & A & A & B \\
B & B & B & B & X \\
A & X & X & X & A \\
\midrule
\end{tabular}
&
\begin{tabular}[t]{cccccc}
\midrule
A & A & B & B & C \\
B & B & A & C & X \\
C & C & C & X & A \\
X & X & X & A & B \\
\midrule
\end{tabular}

\\ \\ \quad
Choice distribution  &
\begin{tabular}[t]{ccc}
A & B & X \\
0.365 & 0.630 & 0.005 \\
\end{tabular}
&
\begin{tabular}[t]{ccc}
A & B & X \\
0.240 & 0.748 & 0.012 \\
\end{tabular}&
\begin{tabular}[t]{cccc}
A & B & C & X\\
0.205 & 0.694 & 0.099 & 0.002 \\
\end{tabular}
\\ \\

\multicolumn{4}{c}{\textbf{D. $p$-values of the difference of choice-distributions}}\\
Comparison across panels \\
\quad A and B  & 0.000  & 0.000 & 0.000 \\
\quad A and C  & 0.000 & 0.000 & 0.000 \\
\quad B and C  & 0.531 & 0.021 & 1.000  \\


\\

\bottomrule
\end{tabularx}
\end{center}
\footnotesize
\textbf{\textit{Notes:}}
Column 3 of Panel A is profile 14 of Table \ref{threeOptionProfiles}. In Panel C, column 3 is profile 18 from Table \ref{fourOptionProfiles}, and columns 1 and 2 are profiles 4 and 5 from Table \ref{threeOptionProfiles}, respectively. Columns 1 and 2 of Panel A both display the distribution of choices that subjects made for the single two-alternative profile they encountered. All decisions concern the work domain. $p$-values are based on two-sample Kolmogorov-Smirnov tests for equality of distributions among options other than $X$.
\end{table}


\subsection{Do people care about ordinal information intrinsically?}\label{cardinalExperiment}

To investigate the possibility that Planners may care about ranks intrinsically, we conduct an additional experiment in which social alternatives are distributions of monetary payoffs. By overtly revealing these payoffs to Planners, we ensure that they no longer use ordinal information to draw inferences about cardinal payoffs.


\paragraph{Design}

A laboratory subject in the role of Planner chooses among vectors of payoffs for three Stakeholders. To test whether ordinal information affects Planner's choices even when inference about cardinal payoffs can no longer play a role, we use two types of treatments, \emph{Swap} and \emph{Addition}.

To understand the Swap treatments, consider the cardinal payoff profile $S1$ in column 1 of Panel A of Table \ref{tab:profilesCardinalNEW}. Assuming Stakeholders only care about their own payoffs, they rank the alternatives as shown in column 2. The Swap treatment swaps participants' cardinal payoffs within alternatives, as highlighted in profile $S1$. These particular swaps yield the cardinal payoff profile $S1'$. Notice that option A delivers a payoff of 6 to one Stakeholder, 5 to a second, and 3 to a third in both $S1$ and $S1'$. Similar statements hold for options B and C. Therefore, the swaps do not alter the relative attractiveness of the three options according to any anonymous Bergson-Samuelson social welfare function. However, the effect on ordinal rankings is potentially consequential: While the rank-dominant alternative in profile $S1$ is $B$, the rank-dominant alternative in the altered profile $S1'$ is $A$. Hence, if the frequency with which Planners select alternative $B$ is higher for profile $S1$ than for $S1'$, an intrinsic concern for (within-menu) rankings of alternatives provides a plausible explanation, but concern for anonymous cardinal payoff profiles does not.

For profiles $S1$ and $S1'$, any Planner who applies an anonymous Bergson-Samuelson social welfare function is indifferent between options A and B. Consequently, a comparison of choices for those profiles can only reveal whether a Planner breaks cardinal ties based on ordinal information. To determine whether Planners are also willing to sacrifice the quality of the cardinal outcome in order to improve the (within-menu) ordinal outcome, we also study Swap treatments in which the payoffs for option $A$ are higher than in $S1$ for all three Stakeholders by a small amount $\epsilon>0$. This change renders option A strictly better than option B according to any anonymous Bergson-Samuelson social welfare function. If the frequency with which Planners select alternative $B$ is also higher for the modified $S1$ than for the modified $S1'$, we can infer that Planners' use ordinal information is not limited to breaking cardinal ties.

Our second type of treatment, \emph{Addition}, resembles those studied in Section \ref{unavailableAlternatives}: We modify a profile by adding an alternative that Planners will likely avoid but that alters the rank distributions of the remaining alternatives. In contrast to those previous treatments, Planners know the cardinal payoffs associated with each option and are aware that the addition of the new alternative leaves those payoffs unchanged. Therefore, any Planner who applies a Bergson-Samuelson social welfare function will continue to select the same option. Accordingly, if choices shift toward the option that becomes more attractive from an ordinal perspective, an intrinsic concern for (within-menu) rankings provides a plausible explanation, but concern for cardinal payoffs does not. The profiles we use for this treatment include $A1$ in Panel B of Table \ref{tab:profilesCardinalNEW}. For this profile, all scoring rules select the majority winner $B$. Adding the unattractive alternative $X$ turns it into profile $A1'$. For this profile, scoring rules with $s> \frac{1}{2}$ will instead select option $A$. Consequently, we ask whether the addition of $X$ increases the frequency with which Planners select $A$ or leaves it unchanged.


\begin{table}
\caption{Cardinal payoff profiles \label{tab:profilesCardinalNEW}}
\begin{center}
    \setlength{\tabcolsep}{16pt}
    \begin{tabular}{cc}
    \multicolumn{2}{c}{\textbf{A. Swap treatments}} \\
    (1) & (2) \\
    \midrule
    \\
    $S1$: Original cardinal payoff profile &
    Implied ordinal preferences
    \medskip \\
    \setlength{\tabcolsep}{2pt}
	\begin{tabular}{ccccccc}
	\toprule
	Option & \multicolumn{6}{c}{Stakeholder} \\
	\cmidrule(r){2-7}
	  && 1 && 2 && 3 \\
	\midrule
	A &&  6 && 3   &$\overset{\text{\tiny swap}}{\longleftrightarrow}$& 5 \\
	B && 3 &$\overset{\text{\tiny swap}}{\longleftrightarrow}$& 5 & & 6 \\
	C & & 1 &$\overset{\text{\tiny swap}}{\longleftrightarrow}$& 4 & & 2 \\
	\bottomrule
	\end{tabular}
    &
    \setlength{\tabcolsep}{6pt}
	\begin{tabular}{lccc}
	\toprule
	Preference & \multicolumn{3}{c}{Stakeholder} \\
	\cmidrule(r){2-4}
	& 1 & 2 & 3 \\
	\midrule
	Best&A&B& B\\
	Middle&B& C&A\\
	Worst&C& A&C\\
	\bottomrule
	\end{tabular}
\\ \\
    $S1'$: Altered cardinal payoff profile &
    Implied ordinal preferences
    \medskip \\
    \setlength{\tabcolsep}{2pt}
	\begin{tabular}{ccccccc}
	\toprule
	Option & \multicolumn{6}{c}{Stakeholder} \\
	\cmidrule(r){2-7}
	  && 1 && 2 && 3 \\
	\midrule
	A &&  6 && 5   &\phantom{$\overset{\text{\tiny swap}}{\longleftrightarrow}$}& 3 \\
	B && 5 &\phantom{$\overset{\text{\tiny swap}}{\longleftrightarrow}$}& 3 & & 6 \\
	C & & 4 &\phantom{$\overset{\text{\tiny swap}}{\longleftrightarrow}$}& 1 & & 2 \\
	\bottomrule
	\end{tabular}
    &
\setlength{\tabcolsep}{6pt}
	\begin{tabular}{lccc}
	\toprule
	Preference & \multicolumn{3}{c}{Stakeholder} \\
	\cmidrule(r){2-4}
	& 1 & 2 & 3 \\
	\midrule
	Best&A&A&B\\
	Middle&B&B&A\\
	Worst&C&C&C\\
	\bottomrule
	\end{tabular} \\
	\\
	\multicolumn{2}{c}{\textbf{B. Addition treatments}}  \\
    (1) & (2) \\
    \midrule
    \\
	$A1$: Original cardinal payoff profile &
     	 Implied ordinal preferences
    	\medskip \\
    \setlength{\tabcolsep}{6pt}
	\begin{tabular}{cccc}
	\toprule
	Option & \multicolumn{3}{c}{Stakeholder} \\
	\cmidrule(r){2-4}
	& 1 & 2 & 3 \\
	\midrule
	A & 3 & 2 & 2 \\
	B & 1 & 3 & 3 \\
	\bottomrule
	\end{tabular}
	&

    \setlength{\tabcolsep}{6pt}
	\begin{tabular}{lccc}
	\toprule
	Pref. rank & \multicolumn{3}{c}{Stakeholder} \\
	\cmidrule(r){2-4}
	& 1 & 2 & 3 \\
	\midrule
	Best & A& B & B\\
	Worst & B& A & A\\
	\bottomrule
	\end{tabular}
	\\
    \\
	$A1'$: Altered cardinal payoff profile &
     	 Implied ordinal preferences
    	\medskip \\
    \setlength{\tabcolsep}{6pt}
	\begin{tabular}{cccc}
	\toprule
	Option & \multicolumn{3}{c}{Stakeholder} \\
	\cmidrule(r){2-4}
	& 1 & 2 & 3 \\
	\midrule
	A & 3 & 2 & 2 \\
	B & 1 & 3 & 3 \\
	X & 2 & 1 & 1 \\
	\bottomrule
	\end{tabular}
	&
    \setlength{\tabcolsep}{6pt}
	\begin{tabular}{lccc}
	\toprule
	Preference & \multicolumn{3}{c}{Stakeholder} \\
	\cmidrule(r){2-4}
	& 1 & 2 & 3 \\
	\midrule
	Best & A& B & B\\
	Middle & X& A & A\\
	Worst & B& X & X\\
	\bottomrule
	\end{tabular}
\\
\end{tabular}
\end{center}
\end{table}


\paragraph{Implementation}

We ran the experiment at the University of Zurich's Experimental Economics Laboratory between March and May 2023 with 584 subjects in the role of Planner. To increase statistical power, we include a second profile $A2$ for the Addition treatment,\footnote{The payoff vectors of profile $A2$ are $A: (2, 2, 2), B: (1, 3, 3), X: (3, 1, 1)$, where $X$ is the added alternative.} and have Planners make decisions for multiple rescaled versions of all profiles, with scaling factors ranging from 2 to 5. (Additional minor changes to the payoff profiles $A1$ and $A2$ prevent overlapping information in visual depictions.) Appendix \ref{appendixCardinalDesign} presents details. Planners completed the corresponding 20 tasks in random order. Each Planner knew there was a 10\% chance she would be assigned to an actual group of Stakeholders, in which case we would implement one of her decisions, selected at random. As in the foregoing experiment, this design ensures that, as long as the Planner is not indifferent about the Stakeholders' welfare, she has an incentive to reveal her true social preferences. The Planner's own payment does not depend on her selection. In addition, Stakeholders recruited on prolific.com received the payments associated with the option chosen by their assigned Planner in a randomly selected task.


\paragraph{Results}

Table \ref{mainResultsCardinal} shows our results. Because Planners rarely choose options $C$ or $X$,\footnote{The frequencies are $1.23$\%, $1.34$\%, $1.28$\%, and $0.68$\% in profiles $S1, S1', A1'$, and $A2'$, respectively. Appendix \ref{appendixCardinal} presents additional results and robustness checks.} we can focus on the distribution of choices between options $A$ and $B$.


Panel A concerns the Swap treatments. The first row shows dramatic effects of changes in rankings when all $\epsilon_i$ are zero. Planners choose option $A$ 30\% of the time in profile $S1$ (where it is rank-dominated) versus more than 70\% of the time in profile $S1'$ (where it is rank-dominant). In both cases, we can reject a choice frequency of 50\%, the most natural manifestation of indifference, at $p<0.01$. The second row of Panel A shows that this effect persists even when alternative $A$ cardinally dominates alternatives $B$ and $C$. Notably, the frequency with which Planners choose option $A$ in profile $S1$ is only 76\%, which is consistent with social preferences that value the avoidance of Stakeholders' least-preferred outcomes. Furthermore, that frequency rises to more than 90\% in $S1'$, where ordinal and cardinal considerations both favor $A$. Comparing the rows of Panel A shows that Planners also care about cardinal payoffs: they select alternative $A$ significantly more often when its payoff distribution dominates that of alternative $B$, even though the ordinal preference profile is the same.


The Addition treatments corroborate these findings (Panel B). For both profiles $A1$ and $A2$, the addition of alternative $X$ decreases the frequency with which Planners choose the majority winner, $B$, to a statistically significant degree. This violation of Sen's Alpha is consistent with the hypothesis that Planners try to avoid alternatives that Stakeholders rank lowest among the available options.


We conclude that Stakeholders' ordinal rankings over available alternatives affect Planners' choices even when cardinal information is provided.


\begin{table}[ht!]
\caption{Choices with cardinal payoff information \label{mainResultsCardinal}}
\begin{center}
\begin{tabular}{cc}
\begin{tabular}{lccc}
\multicolumn{4}{c}{\textbf{A. Swap treatments}} \medskip \\
\toprule
& \multicolumn{2}{c}{Option $A$ chosen} \\
\cmidrule(r){2-3}
Profile          & $S1$           & $S1'$ & Difference\\
\midrule
$\epsilon = 0$ & 0.300 & 0.712 & 0.412 \\  & (0.013) & (0.012) & (0.022) \\ $\epsilon > 0$ & 0.755 & 0.901 & 0.146 \\  & (0.014) & (0.010) & (0.015) \\ Difference & 0.455 & 0.189 & -0.265 \\  & (0.017) & (0.013) & (0.021) \\
\bottomrule
\end{tabular}
\medskip
&
\begin{tabular}{lccc}
\multicolumn{3}{c}{\textbf{B. Addition treatments}} \medskip \\
\toprule
& \multicolumn{2}{c}{Option $B$ chosen} \\
\cmidrule(r){2-3}
Profile          & $A1$           & $A2$\\
\midrule
$X$ unavailable & 0.220 & 0.496  \\  & (0.014) & (0.019)  \\ $X$ added & 0.175 & 0.396  \\  & (0.013) & (0.018)  \\ Difference & 0.045 & 0.099  \\  & (0.014) & (0.014)  \\
\bottomrule
\end{tabular}
\end{tabular}
\end{center}
\footnotesize
\textbf{\textit{Notes}}: Panel A shows the frequency with which subjects choose option A. Panel B shows the frequency with which subjects choose option B. Standard errors clustered by subject. Pooled across display formats.
\end{table}


\section{Robustness across samples}\label{sec:robustness}\label{genPop}


In this section, we ask whether our main conclusions extend to general population samples. We also test whether the general public in countries with divergent social and political traditions, the United States and Sweden \citep{alesinaGlaeser}, use similar or divergent criteria when aggregating ordinal preferences, and we explore external validity by asking whether ordinal aggregation preferences among the general public are related to attitudes toward political processes.


In these supplemental experiments, each social choice entails the allocation of \$20 (in the U.S.) or SEK170 (in Sweden, worth approximately \$20 at the time of the experiment) to one of four charities: Doctors without Borders, Unicef, Oxfam, and the International Fund for Animal Welfare. These organizations are well-known in both countries and represent diverse causes that have broad appeal across the political spectrum. We recruited 711 Swedish and 805 U.S. voting-age citizens through Dynata and Lucid to serve as Planners. Each Planner aggregates the preferences of same-country Stakeholders, recruited through pollfish. Planners only observe group members' ordinal preference rankings. All Planners see the same set of profiles we used for the political domain in our main experiment, as well as either the actual profile for a group of Stakeholders or a randomly generated profile.  Planners know that one of the preference profiles they consider corresponds to real Stakeholders with 10\% probability. Abridged instructions carefully explain the preference displays (see Appendix \ref{instructionsGenPop}). Subjects who failed comprehension checks were excluded. To make our samples more representative of the respective general populations, we weight observations as follows. For the U.S. sample, we use the 2018 General Social Survey \citep{GSS} to generate weights for 4 population categories defined by (i) gender (male, female), and (ii) political party preference (Democrat, Republican). For the Swedish sample, we use data from \citet{StatisticsSweden} to generate weights for 12 categories defined by (i) gender, and (ii) political party preference (Left Party, Social Democratic Party, Green Party, Centre Party, Moderate Party, Sweden Democrats). Appendix \ref{app:genPopDesign} provides additional implementation details and demographic summary statistics.


We use the Bayes classifier to assign each subject to the best-fitting rule based on their choices for the seven three-option profiles. Figure \ref{figClassificationGenPop} shows classification frequencies for the U.S. and Swedish samples. For comparison, we also include the frequencies for the Swiss student sample concerning the political domain, with the caveat that this domain is not identical to the one we use for the general population samples. To facilitate comparisons across samples, we consolidate several categories (strictly concave scoring rules, all benevolent alternatives aside from scoring rules, and all malevolent rules).


\begin{figure}[htp]
\caption{Classification of general population samples to pre-specified rules \label{figClassificationGenPop}}
\vspace{-0.3cm}
\begin{center}
\includegraphics[width=0.8\textwidth]{graphs/figure7.pdf}
\end{center}
\vspace{-0.8cm}
\footnotesize
\textbf{\textit{Notes:}} Bayes classifications based on consequential choices for the seven three-option preferences profiles indicated with stars in Table \ref{threeOptionProfiles}. General population observations weighted to make the samples representative with respect to gender and party preference. Whiskers denote 95\% confidence intervals.
\end{figure}


Differences in classification frequencies between the Swedish and U.S. general population samples are remarkably small and statistically weak: Holm-Bonferroni corrected $t$-tests for each category reject the hypothesis of equal frequency at the 5\% level only for $\frac{1}{3}<s<\frac{1}{2}$. Hence, the fundamental aggregation preferences of U.S. and Swedish citizens are similar, which suggests that they do not contribute to policy divergences. The general population distributions also resemble the distribution for the student sample despite the difference in domains. Widespread conformance with strictly concave scoring rules proves to be a robust phenomenon. Notably, the elevated frequencies of ``Other benevolent rules" are partly attributable to the antiplurality runoff rule, which is closely related to the strictly concave scoring rule alternatives. Choices for discerning four-option profiles corroborate these conclusions (see Appendix \ref{app:genPopFourOpt}). The fact that conformance with the Borda rule in Figure \ref{figClassificationGenPop} is lower than in Figure \ref{barChart} is a mechanical consequence of classifying subjects based on seven profiles rather than seventeen: with fewer profiles, it is more difficult to detect the irresoluteness that distinguishes the linear scoring rule from nearby alternatives.
The modestly higher frequencies of malevolent rules suggest that our general population samples may include slightly higher fractions of subjects who did not take the tasks seriously. These results do not substantially change if we run unweighted regressions (see Appendix \ref{app:genPopRobustness}).


Finally, we ask whether our measures of aggregation preference correlate with self-reported attitudes toward political processes. Our survey presents subjects with two hypothetical candidates for political leadership of the nation. It describes Candidate 1  as polarizing and Candidate 2 as a compromise alternative.\footnote{Candidate 1:``\textit{Most citizens either love him or hate him. There is hardly anyone who does not have a strong opinion. If candidate 1 were elected, some citizens would be exhilarated, many others would be devastated, and nobody would be indifferent.}'' Candidate 2: ``\textit{While he is nobody's greatest favorite, most citizens would be ok with candidate 2. If he were elected, nobody would be exhilarated, nobody would be devastated}.'' We randomize the order of the descriptions and the candidate labels.} We ask subjects which candidate better represents the will of the citizens of the nation. Pooling across the US and Swedish samples, we find evidence that Planners who deploy more concave scoring rules tend to have a stronger preference for the compromise candidate over the polarizing candidate, but statistical significance is not robust. Subjects also express their agreement with two statements regarding compromise versus majoritarian approaches in the political process. Statistical evidence on the relation between these responses and Planners' best-fitting scoring parameters is inconclusive. Appendix \ref{app:externalValidity} presents details.


\section{Conclusion}\label{sec:conclusion}

Our objective in this paper has been to develop an empirical understanding of ordinal aggregation preferences. Because there are many settings in which groups make collective choices based primarily on ordinal rankings, this objective is of considerable practical importance. We find that choices for the overwhelming majority of people conform to \emph{scoring rules}. The most common choice patterns correspond to the Borda and near-antiplurality rules. For a sizeable majority of subjects, choices conform to strictly concave scoring rules, which indicates a pronounced preference for compromise. Subjects are generally averse to majoritarian outcomes. Thus, many familiar rules, such as plurality and the Condorcet criterion, describe the ordinal aggregation choices of relatively few individuals. The same is true of plurality runoff, supermajority and unanimity rules. The classification's fit is excellent, and clustering analysis reveals no major omissions from our list of pre-specified rules. We find systematic and significant differences in the distributions of rules between the work domain and the political domain, but these differences are of limited magnitude. Because our classifications are highly predictive of choices out of sample, including across domains, we infer that ordinal aggregation also entails stable structural elements. While we find strong indications that subjects aggregate ordinal preferences based in part on inferences about cardinal utility, a separate experiment demonstrates that subjects also care about ordinal rankings intrinsically. It follows that (within-menu) ordinal rankings may impact perceptions of fairness even when cardinal information is available. Supplemental experiments show that the distributions of aggregation preferences in the U.S. and Sweden, countries with divergent political and social traditions, are remarkably similar, and both resemble the distribution for the student sample used in our main experiment. There is suggestive evidence that conformance with more concave scoring rules in experimental decisions correlates with a preference for electing compromise candidates.


Our analysis suggests many potential directions for future work. The extent to which our as-if characterizations generalize beyond the domain we examined (small groups and small menus) is an open question. Even if they are context-specific, they potentially illuminate attitudes that may apply more generally, for example concerning the desire for compromise solutions. We have also focused exclusively on preferences over outcomes. A complementary line of inquiry would investigate the relationships between preferences over rules revealed by choices over outcomes, and preferences over rules implied by approval of axioms (in the spirit of \citealp{NielsenRehbeck} and others). Likewise, one could also study preferences over rules revealed by choices over rules (as in  \citealp{EngelmannGruener, HoffmanRenes, Engelmannetal.MinorityProtection}), but that approach would necessarily implicate the strategic issues from which we have intentionally abstracted. It would also be of interest to investigate whether an awareness of manipulability issues factors into preferences over collective choice mechanisms.


\newpage

\setstretch{1}
\setlength{\bibsep}{0pt plus 0.3ex}


\ifx\undefined\bysame
\newcommand{\bysame}{\leavevmode\hbox to\leftmargin{\hrulefill\,\,}}
\fi
\begin{thebibliography}{xx}

\harvarditem[Alesina and Glaeser]{Alesina and Glaeser}{2004}{alesinaGlaeser}
{\bf Alesina, Alberto and Edward Glaeser}, {\it Fighting poverty in the US and
  Europe: A world of difference}, Oxford University Press, 2004.

\harvarditem[Alm{\aa}s et al.]{Alm{\aa}s, Cappelen and
  Tungodden}{2020}{Almaasetal.}
{\bf Alm{\aa}s, Ingvild, Alexander~W Cappelen, and Bertil Tungodden},
  ``Cutthroat capitalism versus cuddly socialism: Are Americans more
  meritocratic and efficiency-seeking than Scandinavians?,'' {\it Journal of
  Political Economy}, 2020, {\it 128} (5), 1753--1788.

\harvarditem[Ambuehl and Bernheim]{Ambuehl and
  Bernheim}{2025}{AmbuehlBernheimData}
{\bf Ambuehl, Sandro and B.~Douglas Bernheim}, ``Data and Code for: ``Social
  Preferences over Ordinal Outcomes'','' 2025.
\newblock Distributed by Inter-university Consortium for Political and Social
  Research, Ann Arbor, MI.

\harvarditem[Ambuehl et al.]{Ambuehl, Bernheim and
  Ockenfels}{2021a}{AmbuehlBernheimOckenfels}
{\bf \bysame{}, B~Douglas Bernheim, and Axel Ockenfels}, ``What Motivates
  Paternalism? {A}n Experimental Study,'' {\it American Economic Review}, 2021,
  {\it 111} (3).

\harvarditem[Ambuehl et al.]{Ambuehl, Blesse, D\"orrenberg, Feldhaus and
  Ockenfels}{2021b}{politiciansProject}
{\bf \bysame{}, Sebastian Blesse, Philip D\"orrenberg, Christoph Feldhaus, and
  Axel Ockenfels}, ``Politicians’ Social Welfare Criteria: An Experiment With
  German Legislators,'' {\it unpublished}, 2021.

\harvarditem[Andreoni et al.]{Andreoni, Aydin, Barton, Bernheim and
  Naecker}{2020}{Andreonietal.WhenFairIsntFair}
{\bf Andreoni, James, Deniz Aydin, Blake Barton, B~Douglas Bernheim, and
  Jeffrey Naecker}, ``When Fair Isn't Fair: Understanding Choice Reversals
  Involving Social Preferences,'' {\it Journal of Political Economy}, 2020,
  {\it 128} (5).

\harvarditem[Apesteguia et al.]{Apesteguia, Ballester and
  Ferrer}{2011}{Apesteguiaetal.Justice}
{\bf Apesteguia, Jose, Miguel~A Ballester, and Rosa Ferrer}, ``On the justice
  of decision rules,'' {\it The Review of Economic Studies}, 2011, {\it 78}
  (1), 1--16.

\harvarditem[Arrieta and Bolte]{Arrieta and Bolte}{2023}{ArrietaBolte}
{\bf Arrieta, Gonzalo and Lukas Bolte}, ``What You Don't Know May Hurt You: A
  Revealed Preferences Approach,'' {\it Working Paper}, 2023.

\harvarditem[Arrow]{Arrow}{1950}{Arrow1950}
{\bf Arrow, Kenneth~J}, ``A difficulty in the concept of social welfare,'' {\it
  Journal of Political Economy}, 1950, {\it 58} (4), 328--346.

\harvarditem[Arrow]{Arrow}{1951}{Arrow1951}
{\bf \bysame{}}, {\it Social choice and individual values}, John Wiley \& Sons,
  1951.

\harvarditem[Baker]{Baker}{2008}{Baker}
{\bf Baker, John}, ``Election of the Green Party Cathaoirleach, 2007,'' {\it
  Irish Political Studies}, 2008, {\it 23} (3), 431--440.

\harvarditem[Bolton and Ockenfels]{Bolton and
  Ockenfels}{2006}{BoltonOckenfelsComment}
{\bf Bolton, Gary~E and Axel Ockenfels}, ``Inequality aversion, efficiency, and
  maximin preferences in simple distribution experiments: comment,'' {\it
  American Economic Review}, 2006, {\it 96} (5), 1906--1911.

\harvarditem[Brandt et al.]{Brandt, Conitzer, Endriss, Lang and
  Procaccia}{2016}{ComSocHandbook}
{\bf Brandt, Felix, Vincent Conitzer, Ulle Endriss, Jerome Lang, and Ariel~D.
  Procaccia}, {\it Handbook of Computational Social Choice}, Cambridge
  University Press, 2016.

\harvarditem[Camerer et al.]{Camerer, Dreber, Forsell, Ho, Huber, Johannesson,
  Kirchler, Almenberg, Altmejd, Chan et~al.}{2016}{Camereretal.Replicability}
{\bf Camerer, Colin~F, Anna Dreber, Eskil Forsell, Teck-Hua Ho, J{\"u}rgen
  Huber, Magnus Johannesson, Michael Kirchler, Johan Almenberg, Adam Altmejd,
  Taizan Chan et~al.}, ``Evaluating replicability of laboratory experiments in
  economics,'' {\it Science}, 2016, {\it 351} (6280), 1433--1436.

\harvarditem[Costa-Gomes and Crawford]{Costa-Gomes and
  Crawford}{2006}{CostaGomesCrawford}
{\bf Costa-Gomes, Miguel~A and Vincent~P Crawford}, ``Cognition and behavior in
  two-person guessing games: An experimental study,'' {\it American Economic
  Review}, 2006, {\it 96} (5), 1737--1768.

\harvarditem[Costa-Gomes et al.]{Costa-Gomes, Crawford and
  Broseta}{2001}{CostaGomesetal.ECMA}
{\bf Costa-Gomes, Miguel, Vincent~P Crawford, and Bruno Broseta}, ``Cognition
  and behavior in normal-form games: An experimental study,'' {\it
  Econometrica}, 2001, {\it 69} (5), 1193--1235.

\harvarditem[Davern et al.]{Davern, Bautista, Freese, Morgan and
  Smith}{2021}{GSS}
{\bf Davern, Michael, Rene Bautista, Jeremy Freese, Stephen~L. Morgan, and
  Tom~W. Smith}, {\it General Social Survey 2016--2020 Panel}, Chicago: NORC,
  2021.
\newblock Machine-readable data file.

\harvarditem[Davies and Shah]{Davies and Shah}{2003}{DaviesShah}
{\bf Davies, Todd and Raja Shah}, ``Intuitive preference aggregation: Tests of
  independence and consistency,'' in ``Public Choice Society and Economic
  Science Association Annual Meetings, Nashville, TN'' 2003.

\harvarditem[{Debian Project}]{{Debian Project}}{2016}{Debian}
{\bf {Debian Project}}, ``Constitution for the Debian Project, v1.7,'' {\it
  \url{https://www.debian.org/devel/constitution}}, 2016.

\harvarditem[{Econometric Society}]{{Econometric
  Society}}{2022}{econometricSociety}
{\bf {Econometric Society}}, ``Rules and Procedures,''
  \url{https://www.econometricsociety.org/society/organization-and-governance/rules-and-procedures#45}
  May 2022.

\harvarditem[Engelmann and Gr{\"u}ner]{Engelmann and
  Gr{\"u}ner}{2017}{EngelmannGruener}
{\bf Engelmann, Dirk and Hans Gr{\"u}ner}, ``Tailored {B}ayesian Mechanisms:
  Experimental Evidence from Two-Stage Voting Games,'' {\it Unpublished
  Manuscript}, 2017.

\harvarditem[Engelmann and Strobel]{Engelmann and
  Strobel}{2004}{EngelmannStrobel}
{\bf \bysame{} and Martin Strobel}, ``Inequality aversion, efficiency, and
  maximin preferences in simple distribution experiments,'' {\it American
  Economic Review}, 2004, {\it 94} (4), 857--869.

\harvarditem[Engelmann et al.]{Engelmann, Gruener, Hoffmann and
  Possajennikov}{2020}{Engelmannetal.MinorityProtection}
{\bf \bysame{}, Hans~Peter Gruener, Timo Hoffmann, and Alex Possajennikov},
  ``Minority Protection in Voting Mechanisms: Experimental Evidence,'' {\it
  Unpublished Manuscript}, 2020.

\harvarditem[Featherstone]{Featherstone}{2019}{Featherstone.RankEfficiency}
{\bf Featherstone, Clayton~R}, ``Rank efficiency: Investigating a widespread
  ordinal welfare criterion,'' {\it unpublished}, 2019.

\harvarditem[Fehr and Charness]{Fehr and Charness}{2023}{fehrCharness}
{\bf Fehr, Ernst and Gary Charness}, ``Social preferences: fundamental
  characteristics and economic consequences,'' 2023.

\harvarditem[Freeman et al.]{Freeman, Brill and
  Conitzer}{2014}{Freemanetal.AxiomaticRunoff}
{\bf Freeman, Rupert, Markus Brill, and Vincent Conitzer}, ``On the axiomatic
  characterization of runoff voting rules,'' in ``Twenty-Eighth AAAI Conference
  on Artificial Intelligence'' 2014.

\harvarditem[Gaertner]{Gaertner}{2009}{Gaertner.HandbookChapter}
{\bf Gaertner, Wulf}, {\it Distributive justice: An overview of experimental
  evidence}, Oxford University Press,  2009.

\harvarditem[Gaertner and Schokkaert]{Gaertner and
  Schokkaert}{2012}{GaertnerSchokkaert.Book}
{\bf \bysame{} and Erik Schokkaert}, {\it Empirical social choice:
  questionnaire-experimental studies on distributive justice}, Cambridge
  University Press, 2012.

\harvarditem[Gibbard]{Gibbard}{1973}{Gibbard}
{\bf Gibbard, Allan}, ``Manipulation of voting schemes: a general result,''
  {\it Econometrica}, 1973, {\it 41} (4), 587--601.

\harvarditem[Herreiner and Puppe]{Herreiner and Puppe}{2010}{HerreinerPuppe}
{\bf Herreiner, Dorothea~K and Clemens Puppe}, ``Inequality aversion and
  efficiency with ordinal and cardinal social preferences---An experimental
  study,'' {\it Journal of Economic Behavior \& Organization}, 2010, {\it 76}
  (2), 238--253.

\harvarditem[Hoffmann and Renes]{Hoffmann and Renes}{2017}{HoffmanRenes}
{\bf Hoffmann, Timo and Sander Renes}, ``Flip a coin or vote: Choosing Group
  Decision Rules,'' {\it Unpublished Manuscript}, 2017.

\harvarditem[Holt and Laury]{Holt and Laury}{2002}{HoltLaury}
{\bf Holt, Charles~A. and Susan~K. Laury}, ``Risk Aversion and Incentive
  Effects,'' {\it American Economic Review}, 2002, {\it 92} (5), 1644--655.

\harvarditem[Huang]{Huang}{1998}{Huang.kModes}
{\bf Huang, Zhexue}, ``Extensions to the k-means algorithm for clustering large
  data sets with categorical values,'' {\it Data Mining and Knowledge
  Discovery}, 1998, {\it 2} (3), 283--304.

\harvarditem[Jackson and Yariv]{Jackson and
  Yariv}{2014}{JacksonYariv.Experiment}
{\bf Jackson, Matthew~O and Leeat Yariv}, ``Present bias and collective dynamic
  choice in the lab,'' {\it American Economic Review}, 2014, {\it 104} (12),
  4184--4204.

\harvarditem[Kara and Sertel]{Kara and Sertel}{2005}{KaraSertel}
{\bf Kara, Ay{\c{c}}a E~Giritligil and Murat~R Sertel}, ``Does majoritarian
  approval matter in selecting a social choice rule? An exploratory panel
  study,'' {\it Social Choice and Welfare}, 2005, {\it 25} (1), 43--73.

\harvarditem[Konow]{Konow}{2003}{Konow.reviewFairness}
{\bf Konow, James}, ``Which is the fairest one of all? {A} positive analysis of
  justice theories,'' {\it Journal of Economic Literature}, 2003, {\it 41} (4),
  1188--1239.

\harvarditem[Kuziemko et al.]{Kuziemko, Buell, Reich and
  Norton}{2014}{Kuziemkoetal.LastPlaceAversion}
{\bf Kuziemko, Ilyana, Ryan~W Buell, Taly Reich, and Michael~I Norton},
  ````Last-place aversion'': Evidence and redistributive implications,'' {\it
  The Quarterly Journal of Economics}, 2014, {\it 129} (1), 105--149.

\harvarditem[Martinangeli and Windsteiger]{Martinangeli and
  Windsteiger}{2020}{MartinangeliWindsteiger}
{\bf Martinangeli, Andrea~FM and Lisa Windsteiger}, ``Last word not yet spoken:
  a reinvestigation of last place aversion with aversion to rank reversals,''
  {\it Experimental Economics}, 2020, pp.~1--21.

\harvarditem[Nielsen and Rehbeck]{Nielsen and Rehbeck}{2020}{NielsenRehbeck}
{\bf Nielsen, Kirby and John Rehbeck}, ``When Choices are Mistakes,'' {\it
  Available at SSRN 3481381}, 2020.

\harvarditem[Portmann and Stojanovi{\'c}]{Portmann and
  Stojanovi{\'c}}{2019}{PortmannStojanovic}
{\bf Portmann, Lea and Nenad Stojanovi{\'c}}, ``Electoral discrimination
  against immigrant-origin candidates,'' {\it Political Behavior}, 2019, {\it
  41} (1), 105--134.

\harvarditem[Rawls]{Rawls}{1971}{rawls1971theory}
{\bf Rawls, John}, {\it A Theory of Justice}, Harvard University Press, 1971.

\harvarditem[Richie]{Richie}{2004}{Richie}
{\bf Richie, Robert}, ``Instant runoff voting: what Mexico (and others) could
  learn,'' {\it Election Law Journal}, 2004, {\it 3} (3), 501--512.

\harvarditem[Satterthwaite]{Satterthwaite}{1975}{Satterthwaite}
{\bf Satterthwaite, Mark~Allen}, ``Strategy-proofness and {A}rrow's conditions:
  Existence and correspondence theorems for voting procedures and social
  welfare functions,'' {\it Journal of Economic Theory}, 1975, {\it 10} (2),
  187--217.

\harvarditem[{Statistics Sweden}]{{Statistics Sweden}}{2018}{StatisticsSweden}
{\bf {Statistics Sweden}}, ``The Party Preference Survey in May 2018,''
  \url{https://www.scb.se/publication/34439} 2018.

\harvarditem[Thomson and Oppenheimer]{Thomson and
  Oppenheimer}{2016}{CRTalternative}
{\bf Thomson, Keela~S and Daniel~M Oppenheimer}, ``Investigating an alternate
  form of the cognitive reflection test,'' {\it Judgment and Decision Making},
  2016, {\it 11} (1), 99.

\harvarditem[Toplak]{Toplak}{2006}{Toplak}
{\bf Toplak, Jurij}, ``The parliamentary election in Slovenia, October 2004,''
  {\it Electoral Studies}, 2006, {\it 25} (4), 825--831.

\harvarditem[Webb]{Webb}{2010}{Webb.NaiveBayes}
{\bf Webb, Geoffrey~I}, ``Na{\"\i}ve Bayes,'' {\it Encyclopedia of machine
  learning}, 2010, {\it 15}, 713--714.

\end{thebibliography}


 \renewcommand{\thesubsection}{\thesection.\arabic{subsection}}

\newpage


\begin{appendix}

\renewcommand\thetable{\thesection.\arabic{table}}
\renewcommand\thefigure{\thesection.\arabic{figure}}
\setcounter{page}{0}

\begin{center}
{\Huge \textbf{ONLINE APPENDIX}}\\
\vspace{1cm}
{\Large   \textbf{\LARGE Social Preferences over Ordinal Outcomes}  \\  \bigskip \bigskip Sandro Ambuehl, B. Douglas Bernheim}
\end{center}


\pagenumbering{gobble}
\setstretch{1.2}
\addcontentsline{toc}{section}{Appendix}
\part{}
\parttoc
\setstretch{1.4}

\newpage
\pagenumbering{arabic}


\setcounter{table}{0}
\setcounter{figure}{0}


\section{Main experiment: identification and design}

\subsection{Identification}


\paragraph{Distance between rules}\label{app:distanceBetweenRules}


Figure \ref{rulesDistance} tabulates, for each pair of social choice rules, the number of preference profiles (out of the 17 we use for identification) for which their implications differ. Two patterns stand out. First, in the vast majority of cases, different rules have different implications for large numbers of profiles. Ignoring the diagonal, few of the entries in Panel A are less than 3, and many are greater than 8, indicating differences on more than half of profiles. Second, three clusters of similar rules are visible. The first and second clusters consist of the strictly convex and strictly concave scoring rules, respectively. Because these rules are members of the single family that is indexed by a continuous parameter $s$, adjacent rules tend to differ on small numbers of profiles, but more distant scoring rules are more easily differentiated. The third cluster consists of the scoring runoff rules with $s<1$. Observe also that the Condorcet rule as well as the runoff rules, align more closely with convex scoring rules than with concave scoring rules. Hence, a preference for compromise may push subjects away from Condorcet extensions and runoff rules.


\begin{figure}[htbp!]
\caption{Distance between rules on the 17 profiles used in the work domain\label{rulesDistance}}
\begin{center}
\begin{tikzpicture}[draw=black, line width=0.05pt]\matrix (heatmap) [matrix of nodes, nodes in empty cells, row sep = -1.5\pgflinewidth, column sep=-1\pgflinewidth, nodes={draw, minimum size=0.475cm, anchor=center, align=center, inner sep=0, font=\tiny}]{
|[fill=none, draw=none, text=black, text width = 2.5cm]| & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=0$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $0<s<\frac{1}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{1}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{1}{3}<s<\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{1}{2}<s<\frac{3}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{3}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{3}{5}<s<\frac{2}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{2}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{2}{3}<s<\frac{3}{4}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{3}{4}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{3}{4}<s<\frac{4}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{4}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{4}{5}<s<1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $0\leq s\leq\frac{1}{4}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $\frac{1}{4}<s<\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $s=\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $\frac{1}{2}<s<1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $s=1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Condorcet & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Supermajority & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Unanimity\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=0$& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $0<s<\frac{1}{3}$& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{1}{3}$& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!94, draw=black, text=white]| 16\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{1}{3}<s<\frac{1}{2}$& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{1}{2}$& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!88, draw=black, text=white]| 15\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{1}{2}<s<\frac{3}{5}$& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{3}{5}$& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{3}{5}<s<\frac{2}{3}$& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{2}{3}$& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{2}{3}<s<\frac{3}{4}$& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{3}{4}$& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!94, draw=black, text=white]| 16\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{3}{4}<s<\frac{4}{5}$& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{4}{5}$& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{4}{5}<s<1$& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=1$& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $0\leq s\leq\frac{1}{4}$& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $\frac{1}{4}<s<\frac{1}{2}$& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $s=\frac{1}{2}$& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!94, draw=black, text=white]| 16\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $\frac{1}{2}<s<1$& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $s=1$& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Condorcet& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!88, draw=black, text=white]| 15\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Supermajority& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!47, draw=black, text=black]| 8\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Unanimity& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!0, draw=black, text=black]| 0\\
};
\end{tikzpicture}
\end{center}
\footnotesize
\textbf{\textit{Notes:}} This graph plots the set of 23 benevolent rules on both the horizontal and vertical axes. Each cell reports the number of profiles (out of the 17 used in the work domain) for which a given pair of rules differ from each other. We use the definition that two rules differ on a profile if they select a different subset of options (distance = 1); otherwise they do not differ on that profile (distance = 0).
\end{figure}


\paragraph{Excluded three-option profiles}\label{excludedProfiles}

Given anonymity and neutrality, our setting of five Stakeholders and three alternatives allows for 42 different preference profiles. The work domain in our experiment uses the 17 profiles that provide the largest amount of discrimination between our pre-specified rules. Table \ref{excludedProfilesTable} lists the remaining 25 profiles that are not included in our experiment. These omitted profiles cannot expand the set of scoring rules, scoring runoffs, or $q$-majority rules we can distinguish. In the case of scoring rules and $q$-majority rules, this can be shown analytically. For the case of scoring runoff rules, we show this point using brute-force computer scripts.

Figure \ref{rulesDistanceOmitted} shows that these profiles would also provide little or no additional ability to distinguish between our pre-specified rules.


\begin{table}[htbp]
\caption{Excluded three-alternative profiles  \label{excludedProfilesTable}}
\begin{center}
\begin{small}
\setlength{\tabcolsep}{3pt}
\begin{tabular}{ccccc}
\toprule
\begin{tabular}{llllll}
 & A & A & A & A & A\\
 & B & B & B & B & B\\
 & C & C & C & C & C\\
& & & & & \\
 & A & A & A & A & A\\
 & C & C & B & B & B\\
 & B & B & C & C & C\\
& & & & & \\
 & C & C & A & A & A\\
 & A & A & B & B & B\\
 & B & B & C & C & C\\
& & & & & \\
 & B & B & A & A & A\\
 & A & A & B & B & B\\
 & C & C & C & C & C\\
& & & & & \\
 & A & A & A & A & A\\
 & C & B & B & B & B\\
 & B & C & C & C & C\\
& & & & & \\
\end{tabular}
&
\begin{tabular}{llllll}
 & A & A & A & B & B\\
 & C & B & B & A & A\\
 & B & C & C & C & C\\
& & & & & \\
 & A & A & A & B & B\\
 & B & C & C & A & A\\
 & C & B & B & C & C\\
& & & & & \\
 & C & C & A & A & A\\
 & A & B & B & B & B\\
 & B & A & C & C & C\\
& & & & & \\
 & B & B & A & A & A\\
 & A & C & C & C & B\\
 & C & A & B & B & C\\
& & & & & \\
 & B & B & A & A & A\\
 & A & C & C & B & B\\
 & C & A & B & C & C\\
& & & & & \\
\end{tabular}
&
\begin{tabular}{llllll}
 & B & A & A & A & A\\
 & C & B & B & B & C\\
 & A & C & C & C & B\\
& & & & & \\
 & B & A & A & A & A\\
 & C & B & B & C & C\\
 & A & C & C & B & B\\
& & & & & \\
 & C & A & A & A & A\\
 & B & C & B & B & B\\
 & A & B & C & C & C\\
& & & & & \\
 & C & B & A & A & A\\
 & A & C & B & B & B\\
 & B & A & C & C & C\\
& & & & & \\
 & C & B & A & A & A\\
 & A & C & B & B & C\\
 & B & A & C & C & B\\
& & & & & \\
\end{tabular}
&
\begin{tabular}{llllll}
 & B & C & A & A & A\\
 & A & B & C & B & B\\
 & C & A & B & C & C\\
& & & & & \\
 & C & B & B & A & A\\
 & A & A & A & B & B\\
 & B & C & C & C & C\\
& & & & & \\
 & C & B & B & A & A\\
 & A & A & A & B & C\\
 & B & C & C & C & B\\
& & & & & \\
 & B & C & A & A & A\\
 & A & A & C & B & B\\
 & C & B & B & C & C\\
& & & & & \\
 & B & C & A & A & A\\
 & A & A & B & B & B\\
 & C & B & C & C & C\\
& & & & & \\
\end{tabular}
&
\begin{tabular}{llllll}
 & C & A & A & A & A\\
 & A & B & B & B & B\\
 & B & C & C & C & C\\
& & & & & \\
 & C & A & A & A & A\\
 & A & B & B & B & C\\
 & B & C & C & C & B\\
& & & & & \\
 & B & A & A & A & A\\
 & A & C & C & B & B\\
 & C & B & B & C & C\\
& & & & & \\
 & B & A & A & A & A\\
 & A & C & B & B & B\\
 & C & B & C & C & C\\
& & & & & \\
 & B & A & A & A & A\\
 & A & B & B & B & B\\
 & C & C & C & C & C\\
& & & & & \\
\end{tabular} \\
\bottomrule
\end{tabular}

\end{small}
\end{center}
\footnotesize
\emph{Notes:} Most our pre-specified rules make the same prediction about which option will be chosen for most of the profiles in this table. Note that for the majority of these profiles, the preference rank distributions of the alternatives are fully ordered by stochastic dominance, \emph{and} the option with the lowest (least preferred) rank distribution is statewise dominated by another option.
\end{table}


\begin{figure}[htbp!]
\caption{Distance between rules used on the 25 omitted profiles\label{rulesDistanceOmitted}}
\begin{center}
\begin{tikzpicture}[draw=black, line width=0.05pt]\matrix (heatmap) [matrix of nodes, nodes in empty cells, row sep = -1.5\pgflinewidth, column sep=-1\pgflinewidth, nodes={draw, minimum size=0.475cm, anchor=center, align=center, inner sep=0, font=\tiny}]{
|[fill=none, draw=none, text=black, text width = 2.5cm]| & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=0$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $0<s<\frac{1}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{1}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{1}{3}<s<\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{1}{2}<s<\frac{3}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{3}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{3}{5}<s<\frac{2}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{2}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{2}{3}<s<\frac{3}{4}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{3}{4}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{3}{4}<s<\frac{4}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{4}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{4}{5}<s<1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $0\leq s\leq\frac{1}{4}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $\frac{1}{4}<s<\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $s=\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $\frac{1}{2}<s<1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $s=1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Condorcet & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Supermajority & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Unanimity\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=0$& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $0<s<\frac{1}{3}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{1}{3}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{1}{3}<s<\frac{1}{2}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{1}{2}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{1}{2}<s<\frac{3}{5}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{3}{5}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{3}{5}<s<\frac{2}{3}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{2}{3}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{2}{3}<s<\frac{3}{4}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{3}{4}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{3}{4}<s<\frac{4}{5}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{4}{5}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{4}{5}<s<1$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=1$& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!52, draw=black, text=black]| 13& |[fill=airforceblue!84, draw=black, text=white]| 21\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $0\leq s\leq\frac{1}{4}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $\frac{1}{4}<s<\frac{1}{2}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $s=\frac{1}{2}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $\frac{1}{2}<s<1$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $s=1$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Condorcet& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Supermajority& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!52, draw=black, text=black]| 13& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!72, draw=black, text=white]| 18\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Unanimity& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!84, draw=black, text=white]| 21& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!72, draw=black, text=white]| 18& |[fill=airforceblue!0, draw=black, text=black]| 0\\
};
\end{tikzpicture}
\end{center}
\footnotesize
\textbf{\textit{Notes:}} This graph plots the set of 23 benevolent rules on both the horizontal and vertical axes. Each cell reports the number of profiles (out of the 25 profiles omitted from the experiment) for which a given pair of rules differ from each other. We use the definition that two rules differ on a profile if they select a different subset of options (distance = 1); otherwise they do not differ on that profile (distance = 0).
\end{figure}


\newpage
\paragraph{Distance between rules on profiles used the in political domain}\label{rulesDistPolit}

Figure \ref{rulesDistancePolitical} displays the distance between any pair of our pre-specified rules on the preference profiles we use for the political domain. Entries of zero off the diagonal indicate that the corresponding pair of rules cannot be separately identified using either of our methods. Moreover, using incentivized choice alone, we cannot separately identify any rule that differs from two other rules on only a single profile and whose choices are the union of the choices of the two other rules. This case occurs for some point-identified scoring rules with $0<s<1$.


\begin{figure}[H]
\caption{Distance between rules on the 7 profiles used in the political domain\label{rulesDistancePolitical}}
\begin{center}
\begin{tikzpicture}[draw=black, line width=0.05pt]\matrix (heatmap) [matrix of nodes, nodes in empty cells, row sep = -1.5\pgflinewidth, column sep=-1\pgflinewidth, nodes={draw, minimum size=0.475cm, anchor=center, align=center, inner sep=0, font=\tiny}]{
|[fill=none, draw=none, text=black, text width = 2.5cm]| & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=0$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $0<s<\frac{1}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{1}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{1}{3}<s<\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{1}{2}<s<\frac{3}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{3}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{3}{5}<s<\frac{2}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{2}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{2}{3}<s<\frac{3}{4}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{3}{4}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{3}{4}<s<\frac{4}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{4}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{4}{5}<s<1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $0\leq s\leq\frac{1}{4}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $\frac{1}{4}<s<\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $s=\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $\frac{1}{2}<s<1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $s=1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Condorcet & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Supermajority & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Unanimity\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=0$& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $0<s<\frac{1}{3}$& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{1}{3}$& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{1}{3}<s<\frac{1}{2}$& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{1}{2}$& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!86, draw=black, text=white]| 6\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{1}{2}<s<\frac{3}{5}$& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{3}{5}$& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{3}{5}<s<\frac{2}{3}$& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{2}{3}$& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{2}{3}<s<\frac{3}{4}$& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{3}{4}$& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{3}{4}<s<\frac{4}{5}$& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{4}{5}$& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{4}{5}<s<1$& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=1$& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $0\leq s\leq\frac{1}{4}$& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $\frac{1}{4}<s<\frac{1}{2}$& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $s=\frac{1}{2}$& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $\frac{1}{2}<s<1$& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $s=1$& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Condorcet& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Supermajority& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!43, draw=black, text=black]| 3\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Unanimity& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!0, draw=black, text=black]| 0\\
};
\end{tikzpicture}
\end{center}
\footnotesize
\textbf{\textit{Notes:}} This graph plots the set of 23 benevolent rules on both the horizontal and vertical axes. Each cell reports the number of profiles (out of the 7 used in the political domain) for which a given pair of rules differ from each other. We use the definition that two rules differ on a profile if they select a different subset of options (distance = 1); otherwise they do not differ on that profile (distance = 0).
\end{figure}


\newpage
\subsection{Design details}

\paragraph{Overview}
Table \ref{experimentOverview} presents a schematic overview of the experiment.


\begin{table}[htb!]
\caption{Schematic overview of the experiment \label{experimentOverview}}

\begin{tabular}{|p{0.97\textwidth}|}

\hline \\

\textbf{Part 0: Initial instructions} \\ \medskip

\textbf{Part A: Task assigment}\\
\begin{enumerate}[1.]
\item Instructions concerning task assignment
\item Instructions about the interface that displays preference profiles
\item 25 task assignment decisions (intermingled)
\item 3 task assignment decisions with unavailable options (intermingled)
\end{enumerate}

\medskip
\textbf{Part B: Donation to a political party} \\
\begin{enumerate}[1.]
\item Instructions concerning the donation to a political party
\item 12 party donation decisions
\end{enumerate}

\medskip
\textbf{Part C: Further elicitations}
\begin{enumerate}[1.]
\item Beliefs
	\bi
	\item 5 rounds of incentivized belief elicitation about workers' reservation wages, followed by an unincentivized elicitation of own reservation wages for completing each of the five tasks
	\item 6 rounds of incentivized belief elicitation about citizens' political preferences, followed by an unincentivized elicitation of own willingness to pay to trigger or prevent the donation to each of the political parties
	\ei
\item Preferences
	\bi
	\item Risk preferences
	\item Altruism
	\ei
\item Other characteristics
	\bi
	\item Demographic information
	\item Cognitive Reflection Test
	\item Knowledge about social choice theory
	\ei
\end{enumerate}
\\
\hline
\end{tabular} \medskip\\
\footnotesize
\textbf{\textit{Notes:}} Each stage in Part C directly follows instructions concerning that stage. Half of the subjects proceed through the experiment in the order displayed. For the other half, Part B and Part A are interchanged. The latter subjects receive the instructions about the interface that displays preference profiles in Part B instead of in Part A. For those subjects, the two stages of Part C.1 are also interchanged.
\end{table}

\paragraph{Work tasks}\label{app:designDetails}


In the work domain, Planners decide about assignments of the following five tasks to Stakeholders: (i) \emph{Image labeling.} The worker views a sequence of 400 images and identifies each by clicking a button. (ii) \emph{Hate speech filtering.} The worker views 400 messages posted on twitter.com and indicates whether each includes hate speech such as racist or sexist statements. (iii) \emph{Audio transcription.} The worker listens to a sequence of 400 words and, in each case, identifies the word by clicking a button. (iv) \emph{Movie reviews classification.} The worker classifies 400 movie reviews according to whether they are positive or negative. (v) \emph{Assigning apprentices to mentors.} The worker finds a pairwise stable match between five hypothetical apprentices and five hypothetical mentors knowing their preferences. The worker completes 20 rounds of this task. Both Planners and Stakeholders not only read descriptions of each task, but also try each of them out  before making decisions (except for the matching task, a single round of which some subjects find time-consuming).


\paragraph{Alternative display formats}\label{additionalDisplayFormats}

In order to ensure our results are not an inadvertent artifact of a particular presentation of preference information, we consider \emph{all} structurally distinct ways of displaying preference profiles. Because a preference profile is an array of three-tuples of the form (option, worker, rank) that indicate the preference rank a particular worker assigns to a given option, there are three such possibilities: rows and columns can be ranks and workers (in which case options appear in cells), ranks and options (in which case workers appear in cells), or workers and options (in which case ranks appear in cells). We exposed the first possibility Figure \ref{socialArchitectInterface0} in the main text. Figures \ref{socialArchitectInterface1} and \ref{socialArchitectInterface2} show the second and third possibilities, respectively. We provide the same tools for exploration (hiding, highlighting, and rearranging) in all display versions. Each Planner sees one and only one presentation format, which we select at random.

For our experiment with Swedish and US general population respondents, we only use the first and second display version. Not only do they yield similar results in our main experiment as the third display version, but they are also more natural to process.

\begin{figure}[htb!]
\caption{Planners' decision interface, version 2 (individuals in cells) \label{socialArchitectInterface1}}
\begin{center}
\begin{tabular}{|c|}
\hline \\ \medskip
\includegraphics[height = 0.4\textheight]{graphs/figureA4}\\
\hline
\end{tabular}\\
\end{center}
\footnotesize \textbf{\textit{Notes:}} Subjects can drag and drop columns, click a button to shuffle columns, hide choice options, highlight workers, and hide workers. If a subject hides an option, only two rows are shown; the remaining rows are labeled `Best of 2' and `Worst of 2.' Symbols for options and colors for worker symbols are randomly drawn each round. The order of options is randomly drawn each round.
\end{figure}


\begin{figure}[htp!]
\caption{Planners' decision interface, version 3 (ranks in cells) \label{socialArchitectInterface2}}
\begin{center}
\begin{tabular}{|c|}
\hline \\ \medskip
\includegraphics[height = 0.4\textheight, angle=270,origin=c]{graphs/figureA5}\\
\hline
\end{tabular}\\
\end{center}
\footnotesize \textbf{\textit{Notes:}} Subjects can drag and drop columns, drag and drop rows, click a button to shuffle columns, click a button to shuffle rows, highlight preference ranks, hide choice options, and hide workers. If a subject hides an option, only two symbols, labeled `best' and `worst,' are shown. The colors of the symbols labeled `best' and `worst' differ depending on whether two or three options are displayed, to signify that the best (worst) option among two is not necessarily best (worst) among three. Symbols for options and colors for worker symbols are randomly drawn each round. The orders of workers and options are randomly drawn each round.  Colors for the preference ranks are randomly drawn on the individual level but remain constant throughout the experiment.
\end{figure}

\FloatBarrier


\paragraph{Details about decisions in the political domain} There are two differences between the political and work domain that are not mentioned in the main text. First, to prevent Planners from drawing inferences about parties from preference distributions, we select the group of five citizens from a sample that equally represents those who self-identify as politically left, right, and center. Planners are aware of the sample's composition. The second difference concerns the process for eliciting Stakeholders' preferences: citizens' expressed preferences determine the recipient of the donation with 2.5\% probability, versus 5\% probability for the analogous contingency in the work domain.

\paragraph{Additional elicitations}\label{additionalElicitations}

\textit{Beliefs about reservation prices.} In the elicitation of belies about stakeholder reservation prices in the work domain, possible answers range from \$0 to \$10 in increments of \$0.50. Half of the subjects see these questions ordered by rank (first, second, third), and half see them ordered by reverse rank (third, second, first). Before subjects answer these questions, we remind them about the price and income level in the US and the standard wages of mTurk workers. The instructions for these tasks explain the concept of WTP using an example for which we randomize the elicited valuation (either \$5 or \$1). We assume that the Planner's inferences about positional money-metric utility do not vary from one profile to another, and consequently elicit these beliefs only once.

The elicitations in the political domain are the same as for the work domain, with two exceptions. First, Possible predictions for the WTP to trigger the donation lie in the set \{$>$15, 15, 13, 11, 9, 7, 5, 3, 1, -1, -3, -5 ,-7, -9, -11, -13, -15, $<$-15\}. Over the five rounds, Planners make predictions for citizens with different first names rather than different cities of residence. Second, to increase statistical power, Planners answer five (nearly) redundant versions of these questions, where each version specifies the city and state where the worker lives. We use two sets of five city labels, randomly assigned, so we can determine whether the labels are consequential.

If one of the subject's reservation price predictions ends up determining her payment, she receives Fr. 30 minus Fr. 3 for every dollar by which her prediction differs from the truth, which we assess using incentivized multiple decision lists in preliminary sessions involving a separate set of workers. Planners also indicate their own reservation price for completing each of the five tasks, but these responses are unincentivized.


\textit{Risk preferences.} On each line of a multiple decision list, subjects choose between two lotteries. The first lottery pays Fr. 23 with probability $p$ or Fr. 15 with probability $(1-p)$. The second pays Fr. 38 with probability $p$ or Fr. 3 with probability $(1-p)$. The list includes 11 binary choices with $p\in\{0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1\}$. In a second otherwise identical decision list, the first lottery pays either Fr. 20 or Fr. 13, while the second pays either Fr. 34 or Fr. 5.

\textit{Altruism.} Subjects complete a multiple decision list involving the following pairs of alternatives: ``\emph{Increase each of the five workers' payoffs by \$2. Decrease my own study payment by CHF $X$},'' or ``\emph{Do not increase the group members' payoffs. Leave my own study payment unchanged.}'' Each line employs a different value of $X$ in the set $\{0, 1, 2, 3, 4, 6, 8, 10, 12, 14, 16, 18, 20\}$. In an otherwise identical second list, the first option is \emph{``Increase each of the five workers' payoffs by Fr. $X$  (exchanged to USD). Decrease my own study payment by Fr. $5X$},'' and the values of $X$  lie in \{$0.2, 0.4, 0.6, 0.8, 1, 1.2, 1.4, 1.6, 1.8, 2$\}. Notice that the first list implicates both efficiency and equity, while the second implicates only equity.
Subjects complete a multiple decision list involving the following pairs of alternatives: ``\emph{Increase each of the five workers' payoffs by \$2. Decrease my own study payment by CHF $X$},'' or ``\emph{Do not increase the group members' payoffs. Leave my own study payment unchanged.}'' Each line employs a different value of $X$ in the set $\{0, 1, 2, 3, 4, 6, 8, 10, 12, 14, 16, 18, 20\}$. In an otherwise identical second list, the first option is \emph{``Increase each of the five workers' payoffs by Fr. $X$  (exchanged to USD). Decrease my own study payment by Fr. $5X$},'' and the values of $X$  lie in \{$0.2, 0.4, 0.6, 0.8, 1, 1.2, 1.4, 1.6, 1.8, 2$\}. Notice that the first list implicates both efficiency and equity, while the second implicates only equity.


\textit{Demographics questions} We elicit the following demographic characteristics: gender, age, field of study and degree level they are working towards, grades in university entrance exams in mathematics and in their first language, canton in which they completed their university entrance exam, their main language, whether they live with their parents, their number of siblings, monthly spending, religiosity, religion, political stance, and political party they voted for in the last election of the Swiss National Council.

\textit{Questions about background in social choice theory} Subjects report whether they have taken a class covering pertinent topics. We illustrate a three-option, three-citizen cyclical preference profile and ask subjects to name the paradox. We also ask subjects to name the Borda rule and Arrow's impossibility theorem based on verbal descriptions.


\paragraph{Comprehension checks} Each of the two comprehension checks consists of nine questions. The first set concerns the preference displays and the second concerns the general decision environment. Subjects must answer all nine questions correctly to proceed. If they make a mistake, we ask them to review the instructions and try again, but we do not tell them which question(s) they missed. This feature prevents the subject from trying to pass the comprehension check by trial and error (the chance of passing either of the comprehension checks by chance is less than 0.2\%). The comprehension questions appear in Appendix \ref{instructionsMain}.


\newpage
\subsection{Bayes classifier}

\paragraph{Derivation of the Bayesian posterior}\label{bayesClassifierDerivation}

Here, we derive the explicit expression for the Bayesian posteriors, $P(R, \epsilon | c)$, that our Bayesian classifier maximizes. For each preference profile $t$, a social choice rule $R$ prescribes a subset $S_R^t \subseteq \{A, B, C\}$ of admissible options. For each $t$, the subject makes a choice $c_t \in \{A, B, C\}$. If the individual follows rule $R$ with error probability $\epsilon$ and behaves according to the assumptions listed in Section \ref{bayesClassifierIntroduction}, then the probability of choosing each alternative is given by the following expressions for $X, Y, Z \in \{A, B, C\}$ with $X$, $Y$, and $Z$ mutually distinct from each other.
\bmathnn
P\big(c_t = X | S_R^t = \{X\}; \epsilon \big) &=& 1 - \frac{2}{3}\epsilon \\
P\big(c_t = Y | S_R^t = \{X\}; \epsilon \big) &=& \frac{1}{3}\epsilon \\
P\big(c_t = X | S_R^t = \{X, Y\}; \epsilon \big) &=& \frac{1}{2} - \frac{1}{6}\epsilon \\
P\big(c_t = Z | S_R^t = \{X, Y\}; \epsilon \big) &=& \frac{1}{3}\epsilon \\
P\big(c_t = X | S_R^t = \{A,B,C\}; \epsilon \big) &=& \frac{1}{3}
\emathnn
Moreover, by the assumption of conditional independence across rounds, the probability of observing choice sequence $c = (c_1, \ldots, c_T)$ from a subject who follows rule $R$ is given by
$
P(c | R) = \prod_{t = 1}^T P\big(c_t | S_{R}^t\big).
$
Given the assumption of uniform prior probabilities across rules and error probabilities, we derive the prior probability of observing choice sequence $c$ as
$
P(c) = \sum_{R} \frac{1}{N_R} \int_0^1 P(c | R; \epsilon)d\epsilon,
$
where $N_ R$ is the total number of rules.
By Bayes' rule, the posterior associated with rule $R$ and error probability $\epsilon$ conditional on the sequence of choices $c$ is thus given by
\bmath\label{posterior}
P(R, \epsilon | c) = \frac{P(c | R; \epsilon) \mu_i}{P(c)}.
\emath

\paragraph{Monte Carlo Simulations}
\label{BayesMonteCarlo}

We use Monte Carlo simulations to test (i) whether the Bayesian classifier reliably detects the use of pre-specified rules, and (ii) whether noise introduces bias.

To answer the first question, we simulate a sample of 1,000 subjects. We uniformly randomly assign each simulated subject to one of the identifiable benevolent rules in each domain. Each simulated subject follows the assigned rule exactly and randomizes uniformly among all tied options in case of irresoluteness. We then run the Bayesian classifier on this sample of simulated subjects, both using all three-option profiles available in the work domain, and using the three-option profiles available in the political domain. Figure \ref{figClassifierVtruth} shows the results. Focusing on the profiles available in the work domain (Panel A), three features stand out. First, the data are generally tightly centered around the diagonal. Second, subjects following a massively irresolute rule (supermajority and unanimity) are frequently confused for following another rule. Yet, even in these cases, the rule generating the choices receives non-trivial weight. Hence if the classifier assigns zero weight to these two rules, it is unlikely that any subject actually followed one of these rules. Third, while scoring rules with a parameter between the boundaries of identifiable intervals are correctly classified in all cases, scoring rules with a parameter on the interval boundaries are sometimes confused for those with a parameter just above or just below the interval boundary. The reason is mechanical. In case of a scoring parameter on the interval boundary (henceforth: a point-identified scoring rule), the set of chosen options is the union of the options chosen by the neighboring interval-identified scoring rules. By assumption, subjects uniformly randomize in case of ties. If a point-identified scoring rule is irresolute on two profiles, and the neighboring interval-identified scoring rules are resolute on those profiles, for instance, there is a 50\% chance that the randomization over the ties happens to coincide exactly with the choices prescribed by one of the neighboring interval-identified scoring rules.  Panel B performs the same exercise but restricts the available data to the profiles available in the political domain and the set of rules to the rules identifiable in that domain. Qualitatively, we observe the same results as in the work domain.

\begin{figure}
\caption{Bayesian classifier if rules are followed exactly \label{figClassifierVtruth}}
\begin{center}
A. Work domain \\
\includegraphics[width=1\textwidth]{graphs/figureA6_A.pdf} \\
B. Political domain \\
\includegraphics[width=1\textwidth]{graphs/figureA6_B.pdf}
\end{center}
\footnotesize
\textbf{\textit{Notes}}: Each of 1,000 simulated subjects is randomly uniformly assigned one identifiable  benevolent rule in the respective domain and follows the rule exactly. In case of ties, simulated subjects randomize uniformly among all tied options. We run the Bayesian classifier on the simulated data. Fractions of subjects are indicated by the sizes of circles.
\end{figure}


To answer the second question, we simulate 4,000 subjects who choose uniformly randomly from all options in each round.  We then run the Bayesian classifier on this sample of simulated subjects, both using all three-option profiles available in the work domain, and using the three-option profiles available in the political domain. Figure \ref{noiseClassification} shows the resulting distribution of best-fitting types. Panel A uses the profiles available in the work domain. Unsurprisingly, close to half of simulated subjects are assigned to a malevolent rule.\footnote{Because our selection of profiles is not random, deviations from 50\% are expected.} Within the set of benevolent rules, we see that the least resolute rules, supermajority and unanimity, attract by far the largest fraction of subjects. The remaining rules all attract similar numbers of subjects, between one and six percent. Notably, neither of the modes we observe in our experimental data, Borda and near-antiplurality, attract a disproportionate fraction of randomly generated subjects.

Panel B uses the set of profiles available in the political domain. We find that 43.1\% of simulated subjects are classified as malevolent. Unlike in the work domain, we do not observe that randomly generated subjects are classified as following the unanimity rule disproportionately often. A possible reason is that the substantially smaller number of rounds in the political domain makes it more likely that a random vector of selections matches the other rules sufficiently well, especially given the leeway afforded by the high estimated noise parameter.

Overall, we conclude that our classification results in Section \ref{bayesClassifierResults} are not an artifact of classifying noisy data.


\begin{figure}
\caption{Bayesian classifier on uniformly random noise \label{noiseClassification}}
\begin{center}
A. Work domain \\
 (44.8\% classified as malevolent)\\
\includegraphics[width=0.6\textwidth]{graphs/figureA7_A} \\
B. Political domain\\
 (43.1\% classified as malevolent)\\
\includegraphics[width=0.6\textwidth]{graphs/figureA7_B}
\end{center}
\footnotesize

\textbf{\textit{Notes}}: The category ``Plurality equivalent'' includes scoring rules with $0\leq s < \frac{1}{3}$, any Condorcet extension, and any scoring runoff rule with $s<1$.'' Each of 4,000 simulated subjects makes uniformly random choices from all three options. We run the Bayesian classifier on the simulated data. Graphs display the distribution over benevolent rules only.
\end{figure}


\newpage

\setcounter{table}{0}
\setcounter{figure}{0}

\section{Main experiment: supplementary results}


\subsection{Classification using indifference data} \label{app:indifferenceData}

Next to selecting one option for each profile, Planners also indicate any alternatives they consider equally good as the one they have selected. While this expression of indifference has no consequences within our experiment, two considerations mitigate the usual concerns about hypothetical choices: first, we ask Planners to report indifference that they presumably would have recognized when making the associated consequential choice moments prior; second, misrepresenting indifference information would not serve any other plausible objective (e.g., enhancement of social image).

While the revelation of indifference can, in some cases, be incentivized by randomizing among the pertinent options, two considerations make this approach unsuitable for our purposes. First, the procedure changes the nature of social alternatives by introducing lotteries over assignments, thereby implicating workers' preferences over lotteries. Second, the social choice rules we consider do not specify random resolution. Adding that provision to a rule amounts to creating a non-standard extension.

We use an appropriately adjusted version of our Bayes classifier to analyze the indifference data. We continue to impose assumptions (i) and (iii), along with a slightly modified version of assumption (ii): when deviating from her rule, the subject randomizes uniformly over the seven subsets of options, rather than the three options. Under these assumptions, a rule maximizes the posterior probability if and only if it maximizes the number of profiles for which it predicts the correct subset. We break ties at random. Information on indifference  allows us to dispense with assumption (iv). In generating random benchmark data, we generate indifference data by designating each of the unchosen options as equally good based on independent Bernoulli draws. We choose the Bernoulli probability to match the average size of the best-option sets according to the actual subjects.


The average Planner expresses irresoluteness for 18\% of the profiles. In comparison, the set of our benevolent pre-specified rules produces a tie for 18.4\% of the profiles, on average. Focusing only on the Condorcet rule and the set of all scoring rules we can identify, this figure falls to 14.3\%. Planners designate all options as equally good for only 1.54\% of profiles.


\begin{figure}[htb!]
\caption{Best fitting pre-specified rules using indifference data\label{barChartIndifference}}
\begin{center}
\begin{tabular}{c}
\includegraphics[width = 0.8\textwidth]{graphs/figureB1} \\
\end{tabular}
\end{center}
\footnotesize
\textit{\textbf{Notes:}} The random-choice benchmark for malevolent rules exceeds the range of the graph.
\end{figure}

Figure \ref{barChartIndifference} displays the results. Unlike in the case of incentivized choices, indifference data lets us identify all 23 benevolent rules and their malevolent counterparts, yielding a total of 46 pre-specified rules. The results closely resemble those from our main specification (Figure \ref{barChart}). The perhaps most salient difference are the less pronounced modes at the linear and most concave scoring rules, and the slightly increased fraction of subjects assigned to the scoring rule with $\frac{1}{3}<s<\frac{1}{2}$. The overwhelming majority of choices continues to be consistent with a weakly concave scoring rule. Other rules continue to garner vanishing support.

The classifier based on indifference-data continues to achieve a good fit. Subjects' responses match the sets of options selected by their assigned rule for 14 of 17 profiles (83.5\%). In contrast, for our random-choice benchmark, the corresponding number is 43.2\%. As an alternative way of making this comparison, one can think of an empirical fingerprint as consisting of $3 \times 17 = 51$ binary variables, each of which indicates whether a given option is selected for a given profile. The average subject's choice coincides with the prediction of the best-fitting rule in 47 of these 51 cases (91.8\%), which is far higher than the random-choice benchmark (63.9\%).


\subsection{Corroboration based on four-option social choice problems}\label{analysisClassSeparating}\label{app:fourOptionProfiles}

Four-option profiles provide additional opportunities to distinguish cleanly between classes of social choice rules. Specifically, there are two four-option profiles, labeled ``Condorcet-separating 1'' and ``Condorcet-separating 2'' in Table \ref{fourOptionProfiles} (numbered 18 and 19), for which the Condorcet winner is \emph{rank-dominated}. These profiles distinguish between all Condorcet extensions and the entire class of \emph{proper} scoring rules (ones that do not assign the same score to any two ranks) because the former must select the Condorcet winner while the latter cannot select a rank-dominated option.

We randomly intermingle both of these profiles with the three-option profiles in both domain blocks. While four-option problems are more cognitively demanding, recall that the experimental interface allows subjects to hide options, which makes it easy to identify Condorcet winners.

\begin{table}[htb]
\caption{Four-alternative profiles \label{fourOptionProfiles}}
\setlength{\tabcolsep}{3pt}
\begin{small}
\begin{center}
\begin{tabular}{lccccccccccccc}
\toprule
Label  & Index & \multicolumn{5}{c}{Profile}   & Option selected& Condorcet & Plurality\\
&&&&&&&  by some proper  &  & (* = runoff)\\
&&&&&&&  scoring rule  & &   \\ \midrule
&                     &   &   &   &   &   &                          \\
Condorcet-separating 1 & 18                    & A & A & B & B & C & B or C      & A     & \{A$^*$,B\}    \\
                    & & B & B & A & C & D &                  &                 \\
                    & & C & C & C & D & A &                  &                 \\
                    & & D & D & D & A & B &                  &               \medskip  \\
Condorcet-separating 2 & 19                   & A & A & B & B & C & B or C   & A    & \{A$^*$,B\}    \\
                    &&  B & B & A & D & D &                  &                 \\
                    &&  C & C & C & C & A &                  &                \\
                    && D & D & D & A & B &                  &               \medskip  \\
Runoff-separating 1 &22 & A & A & B & D & D & A or B or C&  \{A, B, C, D\} & \{A,D$^*$\}   \\
  && B & B & C & A & C &              &                  \\
  && C & C & D & C & B &              &                  \\
  && D & D & A & B & A &              &                \medskip   \\
Runoff-separating 2 & 23 & A & A & C & D & D & A or B or C & \{A, B, C, D\} & \{A,D$^*$\}  \\
 & & B & B & B & A & B &              &                   \\
 & & C & C & D & C & C &              &                  \\
 & & D & D & A & B & A &              &   \medskip \\

\bottomrule
\end{tabular}
\end{center}
\end{small}
\footnotesize
\textbf{\emph{Notes:}} We display each preference profile as a 4$\times$5-matrix; columns correspond to workers, rows to preference ranks. The $r$th row shows a worker's $r$th-ranked alternative. For Condorcet-cyclical profiles, we indicate the set of options in the top-cycle.
\end{table}

As shown in the left half of Table \ref{separatingProfiles} (which pertains to the work domain), subjects choose the Condorcet winner roughly one-fifth of the time for each of these profiles. However, only 11.6\% consistently choose the Condorcet winner for both profiles.\footnote{This frequency, while relatively small, is significantly higher than would be observed by chance if the distributions of choices were independent across profiles, $0.205\times 0.212 = 0.043$ ($\chi^2$-test, $p<0.001$).} In contrast, because a negligible fraction of subjects choose option D, roughly 80\% of the individual choices are consistent with a proper scoring rule. Moreover, for 69.5\% of subjects, the pair of choices is consistent with some scoring rule in the following class: $[1, s_1, s_2, 0]$ where $s_1 = (2/3)^\gamma$, $s_2 = (1/3)^\gamma$, and $\gamma \in [0, \infty]$.


\begin{table}
\caption{Choices on class-separating profiles \label{separatingProfiles}}
\begin{center}
\begin{tabular}{lccccp{0.5cm}cccccccc}
\toprule
Domain & \multicolumn{4}{c}{Work} && \multicolumn{4}{c}{Politics} \\
\midrule
Option & A & B & C & D && A & B & C & D  \\
\cmidrule(r){2-5} \cmidrule(r){7-10}
Condorcet winner  & \checkmark &&&&& \checkmark \\
Optimal for some scoring rule &  &\checkmark&\checkmark&&& &\checkmark&\checkmark \\
\cmidrule(r){2-5} \cmidrule(r){7-10}

Condorcet-separating 1	 & 	0.205	 & 	0.694	 & 	0.099	 & 	0.002	 & 	 & 	0.188	 & 	0.746	 & 	0.064	 & 	0.002	 & 	\\
Condorcet-separating 2	 & 	0.212	 & 	0.719	 & 	0.064	 & 	0.005	 & 	 & 	0.188	 & 	0.800	 & 	0.012	 & 	0.000	 & 	\\
Consistent Condorcet	& \multicolumn{4}{c}{	0.116	} & & \multicolumn{4}{c}{	0.111	} 		 		 	 		 		 		 		 	\\
Consistent Scoring	& \multicolumn{4}{c}{	0.695	} & & \multicolumn{4}{c}{	0.732	} 		 		 	 		 		 		 		 	\\
\cmidrule(r){2-5} \cmidrule(r){7-10}
	 & 		 & 		 & 		 & 		 & 	 & 		 & 		 & 		 & 		 & 	\\
 Option & A & B & C & D && A & B & C & D  \\
\cmidrule(r){2-5} \cmidrule(r){7-10}
Plurality-runoff winner  &&&&\checkmark&& &&&\checkmark  \\
Optimal for some scoring rule & \checkmark &\checkmark&\checkmark&&& \checkmark&\checkmark&\checkmark \\
\cmidrule(r){2-5} \cmidrule(r){7-10}

Runoff-separating 1	 & 	0.257	 & 	0.385	 & 	0.314	 & 	0.044	 & 	 & 	0.353	 & 	0.405	 & 	0.188	 & 	0.054	 & 	\\
Runoff-separating 2	 & 	0.202	 & 	0.615	 & 	0.141	 & 	0.042	 & 	 & 	0.259	 & 	0.643	 & 	0.042	 & 	0.056	 & 	\\
Consistent Runoff	& \multicolumn{4}{c}{	0.012	} & & \multicolumn{4}{c}{	0.028	} 		 		 	 		 		 		 		 	\\
Consistent Scoring	& \multicolumn{4}{c}{	0.884	} & & \multicolumn{4}{c}{	0.930	} 		 		 	 		 		 		 		 	\\
\bottomrule
\end{tabular}
\end{center}
\footnotesize
\textbf{\textit{Notes:}} This table displays the fraction of subjects choosing each of the four options in each of the class-separating profiles. The first 262 subjects were not presented with the profile labelled Runoff-separating 2 in the political domain. The fraction of subjects consistently choosing in accordance with the plurality runoff rule in that domain is based on the remaining subjects.
\end{table}

The preceding estimates likely overstate the fraction of subjects who more broadly support Condorcet outcomes, because other rules can rationalize the selection of option A for profiles 18 and 19. As indicated in Table \ref{fourOptionProfiles}, plurality rule ($s_1 = s_2 = 0$) prescribes options A and B, while the plurality runoff rule prescribes A. Any scoring rule with $s_1 = s_2 < \frac{1}{2}$ also prescribes either A or B. One way to distinguish between Condorcet and plurality runoff on the one hand, and scoring rules with $s_1=s_2<\frac{1}{2}$ on the other, is to examine the indifference data. Of those who select A for both profiles, 36.2\% rate B as equally good, which suggests that more than a third of these selections reflect a scoring rule with $s_1=s_2<\frac{1}{2}$. Accordingly, choices for our four-option profiles imply that the fraction of subjects who implicitly follow some Condorcet rule in the work domain is no higher than (1 - 0.362)$\cdot$11.6\% = 7.4\%, which is consistent with our findings for three-option problems.

Profiles 22 and 23 in Table \ref{fourOptionProfiles}, which we also intermingled with the three-option profiles, leverage the principle of rank-dominance to provide clean separation between proper scoring rules and the plurality runoff rule. As shown in the left half of Table \ref{separatingProfiles}, subjects choose option D, the plurality runoff winner, roughly 5\% of the time. Only 1.2\% of subjects choose D for both profiles,\footnote{Though small, this fraction is significantly higher than would be observed by chance if the distributions of choices were independent across profiles, $0.044\times0.042=0.001848$ ($\chi^2$-test, $p<0.001$).} whereas 88.4\% of subjects behave as if both choices reflect a single scoring rule from the family specified above. Accordingly, only a small fraction of the Condorcet-consistent behavior for profiles 18 and 19 is likely attributable to use of the plurality runoff rule.

Four-option profiles such as 22 and 23 also enable more nuanced distinctions among rules. For example, they can provide a foundation for distinguishing concave scoring from a general aversion to lowest ranks; see Appendix \ref{lastPlaceAversion}.

We also note that our results for the four-option class-separating profiles in Table \ref{separatingProfiles} exhibit strikingly little domain sensitivity; choices in the work and political domains resemble each other. It follows that people are generally attracted to the selections of scoring rules and generally averse to those of majoritarian criteria.


\subsubsection{Last-place aversion}\label{lastPlaceAversion}

Does the concave aggregation we document in section \ref{classification} simply represent linear aggregation with a discount for receiving one's least-preferred alternative (last-place aversion; formalized as using scoring vectors that assign score $\frac{k-1}{K-1}$ to any option ranked $k\geq 2$, but score $-d<0$ to the option ranked last in the case of $K$ alternatives), or does it reflect globally concave aggregation in the sense that subjects' choices are described by score vectors that are strictly concave across all ranks (for instance by instance, by using scoring vectors that assign score $\left(\frac{k-1}{K-1}\right)^\gamma$ to the option ranked $k$ in the case of $K$ alternatives)? Choices in the runoff-separating profiles of Table \ref{fourOptionProfiles} show that globally concave aggregation plays a substantial role. In these profiles, the choice of option $A$ is consistent with a scoring rule with $s\leq 0.5$ whereas the choice of $B$ or $C$ is consistent with $s\in [\frac{1}{2}, \frac{2}{3}]$ and $s\in [\frac{2}{3}, 1]$, respectively, where $s = (\frac{1}{2})^\gamma$. Importantly, in these profiles, option $B$ is ranked last by one individual, and option $A$ is ranked last by two individuals. In the three-option profiles 3 and 4, 64\% of subjects choose the option consistent with a weakly concave scoring rule. If last-place aversion explains this choice pattern, then, in the four-option profiles 22 and 23, we should observe that at least 64\% of subjects avoid the last-place-generating options ($A$ or $B$). In contrast, the fraction of subjects selecting either option A or option B is given by 64.2\% and 81.7\% for the first and second runoff-separating profiles, respectively. Hence, last-place aversion cannot be the sole reason for the choice patterns we observe in the three-option profiles.

\FloatBarrier

\subsection{Response times by best-fitting rule}\label{appendixResponseTimes}

For each aggregation rule, Figure \ref{figResponseTimeByClassification} shows the median response times of subjects classified as best-fitting that rule. The most popular rules are not associated with unusually short response times. This suggests that subjects do not adopt specific rules just to save time.

\begin{figure}[H]
\caption{Median response time by classification based on incentivized data \label{figResponseTimeByClassification}}
\begin{center}
\includegraphics[width = 0.7\textwidth]{graphs/figureB2} \\
\end{center}
\footnotesize
\textbf{\textit{Notes:}} Spikes show 95\% confidence intervals with standard errors clustered by subject.
\end{figure}


\FloatBarrier

\subsection{Effects of preference displays}\label{analysisPreferenceDisplay}


Here, we study the influence of the format in which we presented preference profiles on our classification results. Figure \ref{barChartsByDisplay} displays the classification to pre-specified rules separately by display format. Graphs on the left hand side use incentivized choices only while those on the right hand side make use of indifference statements.

In each case we see that (i) malevolent and qualified majority rules receive vanishing support, (ii) runoff rules receive minor support, and (iii) the vast majority of subjects follow a concave scoring rule. Differences across the graphs mainly concern the modes of Borda and near-antiplurality. Relying on incentivized data alone, both of these rules emerge as the modal choices in each display version. While Borda is the more pronounced mode in display versions 1 and 3, near-antiplurality is the more pronounced mode in display version 2. This same pattern emerges to a larger extent if we incorporate indifference data for classification. Overall we conclude that our main results are robust to the preference display used, except that we cannot reliably distinguish whether Borda or near-antiplurality is the more pronounced mode.


\begin{figure}[htbp]
\caption{Best fitting pre-specified rules, work domain, by display\label{barChartsByDisplay}}
\begin{center}
\begin{tabular}{cc}
\multicolumn{2}{c}{\textbf{Display 1 (options in cells)}}\\
\midrule
Incentivized choice only & Using indifference data \\
\includegraphics[width = 0.475\textwidth]{graphs/figureB3_display1_col1} &
\includegraphics[width = 0.475\textwidth]{graphs/figureB3_display1_col2} \\
\multicolumn{2}{c}{\textbf{Display 2 (stakeholders in cells)}}\\
\midrule
Incentivized choice only & Using indifference data \\
\includegraphics[width = 0.475\textwidth]{graphs/figureB3_display2_col1} &
\includegraphics[width = 0.475\textwidth]{graphs/figureB3_display2_col2} \\
\multicolumn{2}{c}{\textbf{Display 3 (ranks in cells)}}\\
\midrule
Incentivized choice only & Using indifference data \\
\includegraphics[width = 0.475\textwidth]{graphs/figureB3_display3_col1} &
\includegraphics[width = 0.475\textwidth]{graphs/figureB3_display3_col2} \\
\end{tabular}
\end{center}
\footnotesize
\textbf{\textit{Notes:}} Displays 1, 2, and 3 are shown in Figures \ref{socialArchitectInterface0}, \ref{socialArchitectInterface1}, and  \ref{socialArchitectInterface2}, respectively.
\end{figure}

\newpage

\FloatBarrier

\subsection{Endogenous clusters}\label{appendixClustering}\label{app:clusteringDetails}

The $k$-modes algorithm we use begins by arbitrarily selecting $k$ subjects as initial cluster centers. Then it iterates two steps. First, each subject is assigned to the cluster for which the center matches her choices on the largest number of preference profiles. Second, cluster centers are updated: the new cluster center consists of the vector of modal choices for subjects assigned to that cluster. The algorithm terminates once the cluster centers stabilize.

We search for clusters in the work domain fixing $k=1, 2, 3, 5$, and $10$. For $k=1$, we run the algorithm 405 times, using each subject's choices as an initial cluster center once. For $k\geq2$, we run the algorithm 1000 times, in each case randomly setting the initial cluster centers equal to the choices of $k$ randomly selected subjects (excluding those who perfectly conform to pre-specified rules), and retaining the solution with the lowest total within-cluster distance.\footnote{We use the Hamming distance, i.e. the number of profiles on which two choice sequences differ from each other.} We limit the pre-specified rules to all scoring rules, all scoring runoff rules and all Condorcet extensions, which have a total of 298 resolute components. We exclude unanimity because the resolute components (of which there are $2^3\times3^{14} > 38\times10^6$) encompass all choice patterns that are consistent with the Pareto principle. Similarly, we exclude supermajority because it is massively irresolute, so the number of resolute components is enormous. These exclusions are likely inconsequential given the small number of subjects assigned to these rules in Section \ref{classification}, and in any case the exclusion of an empirically important rule can only increase the fraction of subjects assigned to endogenous clusters. Other exclusions are attributable to redundancies.

Table \ref{clusteringResults} displays the endogenous clusters uncovered by our clustering algorithm and the fractions of subjects assigned to each.


\begin{table}[htbp!]
\caption{Clustering results \label{clusteringResults}}
\small
\begin{center}
\begin{tabularx}{\textwidth}{l *{17}{Y}}
\toprule
Profile & 1 & 2 & 3 & 4 & 5 & 6 & 7 & 8 & 9 & 10 & 11 & 12 & 13 & 14 & 15 & 16 & 17\\
\midrule
Condorcet& A& B& A& A& A& A& A& A& A& A& A& ABC& ABC& A& A& A& A\\
Scoring \\
\quad $\bar{s}$ & $\nicefrac{1}{3}$	& $\nicefrac{1}{3}$ & $\nicefrac{1}{2}$ & $\nicefrac{1}{2}$ & $\nicefrac{1}{2}$	& $\nicefrac{3}{5}$ & $\nicefrac{2}{3}$ & $\nicefrac{2}{3}$ & $\nicefrac{3}{4}$	& $\nicefrac{3}{4}$ & $\nicefrac{4}{5}$ & 0 & $\nicefrac{1}{2}$ & - & 0 & 0 & 0 \\
\quad $s < \bar{s}$ & A & \mbox{A$^{a)}$} & A & A & A & A & A & A & A & A & A & \mbox{-$^{b)}$} & AC & AB & \mbox{-$^{a)}$} & \mbox{-$^{b)}$} & \mbox{-$^{a)}$}\\
\quad $s > \bar{s}$& B& B& B& B& B& B& B& B& B& B& B& B& B& AB& A$^{c)}$& A& A\\
\midrule
\\ \multicolumn{18}{c}{Endogenous clusters, $k=   1$} \\
\cmidrule(r){2-18}
\% subjects \\
\quad 4.69&B & B & B & B & B & B & B & B & A & B & B & B & B & A & A & A & A\\
Total: 4.69 \\
\\ \multicolumn{18}{c}{Endogenous clusters, $k=   2$} \\
\cmidrule(r){2-18}
\% subjects \\
\quad 4.69&B & B & B & B & B & B & B & B & A & B & B & B & B & A & A & A & A\\
\quad 1.73&B & B & B & A & B & B & A & B & B & A & B & B & B & B & A & A & A\\
Total: 6.42 \\
\\ \multicolumn{18}{c}{Endogenous clusters, $k=   3$} \\
\cmidrule(r){2-18}
\% subjects \\
\quad 4.69&B & B & B & B & B & B & B & B & A & B & B & B & B & A & A & A & A\\
\quad 1.73&B & B & B & A & B & B & A & B & B & A & B & B & B & B & A & A & A\\
\quad 0.25&B & B & B & B & C & C & B & C & C & B & C & A & B & C & C & A & B\\
Total: 6.67 \\
\\ \multicolumn{18}{c}{Endogenous clusters, $k=   5$} \\
\cmidrule(r){2-18}
\% subjects \\
\quad 4.69&B & B & B & B & B & B & B & B & A & B & B & B & B & A & A & A & A\\
\quad 1.73&B & B & B & B & B & B & B & B & A & B & B & B & B & B & A & A & A\\
\quad 1.73&B & B & B & A & B & B & A & B & B & A & B & B & B & B & A & A & A\\
\quad 1.48&B & B & A & A & B & B & B & A & A & A & A & B & B & B & A & A & A\\
\quad 0.74&B & B & A & B & B & B & B & A & C & B & B & B & B & C & C & A & A\\
Total: 10.37 \\
\\ \multicolumn{18}{c}{Endogenous clusters, $k=  10$} \\
\cmidrule(r){2-18}
\% subjects \\
\quad 4.69&B & B & B & B & B & B & B & B & A & B & B & B & B & A & A & A & A\\
\quad 1.73&B & B & B & A & B & B & A & B & B & A & B & B & B & B & A & A & A\\
\quad 1.48&B & B & A & A & B & B & B & A & A & A & A & B & B & A & A & A & A\\
\quad 1.23&B & B & A & A & B & B & A & A & A & A & A & B & B & A & A & A & A\\
\quad 0.99&B & B & B & B & B & B & A & A & B & A & A & B & B & B & A & A & A\\
\quad 0.74&B & B & B & B & B & B & A & B & A & A & A & B & C & A & A & A & A\\
\quad 0.74&B & B & B & B & B & B & B & A & B & A & B & B & B & A & A & A & A\\
\quad 0.25&B & B & B & B & C & C & B & C & C & B & C & A & B & C & C & A & B\\
\quad 0.25&C & C & C & A & C & B & A & A & B & C & C & C & A & C & C & C & A\\
\quad 0.25&B & C & A & A & A & A & B & B & A & B & B & B & B & A & C & B & A\\
Total: 12.35 \\
\bottomrule \end{tabularx}
\end{center}
\footnotesize
$^{a)}$ For $s = 0$, \{A,C\} is selected. \\
$^{b)}$ For $s = 0$, \{A,B\} is selected. \\
$^{c)}$ For $s = 1$, \{A,B\} is selected. \medskip \\
\footnotesize
\textbf{\textit{Notes:}} This table shows the clusters that emerge endogenously (in addition to those associated with our pre-specified rules) from application of a $k$-modes algorithm, along with the fractions of subjects assigned to each such cluster. For ease of comparison, the top section of the table shows choices for selected pre-specified rules.
\end{table}

\newpage
\subsection{Cardinal inferences}\label{app:syntheticScoringRules}


Table \ref{scoringCorrelatesDomain} replicates Table \ref{scoringCorrelates} separately for each domain. We observe parameter estimates on $\tilde{s}$ that are comparable in magnitude to those we obtain in the pooled sample. The estimates are larger in the political domain, and only statistically significant if we use the percentile rank of $\tilde{s}$ as a right-hand-side variable. Due to the smaller number of observations, a drop in statistical significance is expected. One possible explanation for the higher estimated coefficients in the political domain compared to the work domain is that participants possess more precise beliefs about the political domain, due to their greater familiarity with the Swiss political landscape than with the working conditions of US-based mTurk workers.


\begin{table}[htb!]
\caption{Relation between best-fitting and synthetic scoring parameters by domain \label{scoringCorrelatesDomain}}
\begin{normalsize}
\setlength{\tabcolsep}{3pt}
\begin{center}
\begin{tabular}{lcccccccc} \toprule
 & (1) & (2) & (3) & (4) & (5) & (6) & (7) & (8) \\
VARIABLE & \multicolumn{8}{c}{Estimated scoring parameter $s$} \\
\midrule
 & \multicolumn{4}{c}{Politics} & \multicolumn{4}{c}{Work} \\
\cmidrule(r){2-5} \cmidrule(r){6-9}
            &               &               &               &               &               &               &               &               \\
\textit{Scoring parameter $\tilde{s}$}&               &               &               &               &               &               &               &               \\
\quad Level &       0.123   &               &       0.117   &               &       0.098   &               &       0.081   &               \\
            &     (0.084)   &               &     (0.085)   &               &     (0.074)   &               &     (0.074)   &               \\
\quad \%-rank&               &       0.072   &               &       0.065   &               &       0.049   &               &       0.039   \\
            &               &     (0.036)   &               &     (0.036)   &               &     (0.031)   &               &     (0.032)   \\
\textit{Risk tolerance}&               &               &               &               &               &               &               &               \\
\quad Level &               &               &      -0.915   &               &               &               &      -0.536   &               \\
            &               &               &     (0.414)   &               &               &               &     (0.348)   &               \\
\quad \%-rank&               &               &               &      -0.070   &               &               &               &      -0.039   \\
            &               &               &               &     (0.036)   &               &               &               &     (0.031)   \\
\textit{Altruism}&               &               &               &               &               &               &               &               \\
\quad Level &               &               &       0.030   &               &               &               &       0.314   &               \\
            &               &               &     (0.286)   &               &               &               &     (0.251)   &               \\
\quad \%-rank&               &               &               &       0.041   &               &               &               &       0.052   \\
            &               &               &               &     (0.038)   &               &               &               &     (0.032)   \\
\hline
Observations & 398 & 398 & 398 & 398 & 394 & 394 & 394 & 394 \\
Subjects & 398 & 398 & 398 & 398 & 394 & 394 & 394 & 394 \\
\bottomrule
\end{tabular}
\end{center}
\end{normalsize}
\footnotesize
\textit{\textbf{Notes:}} Parameters estimated with OLS using midpoint values of best-fit score. Regressions include all subjects with monotonic beliefs about reservation prices. Risk tolerance is measured as the certainty equivalent of an equiprobable win 40 / win 10 lottery implied by individual-level CRRA estimates. Altruism is measured as the maximum amount of Swiss Francs the Planner is willing to give up to increase each stakeholder's payoff by \$2. For subjects with multiple switches in the risk preference or altruism elicitation, the corresponding variable is set to the mean value among the other subjects. We also include two indicator variables for the presence of multiple switching points, one for each characteristic. All regressions control for the type of preference profile presentation and for whether the political domain was displayed before or after the work domain. Standard errors are clustered by subject.
\end{table}


\newpage

\setcounter{table}{0}
\setcounter{figure}{0}

\section{Cardinal outcomes experiment} \label{appendixCardinal}

\setcounter{figure}{0}
\setcounter{table}{0}


\subsection{Design details}\label{appendixCardinalDesign}

Planners complete 20 decision tasks in random order, 12 concerning rescaled versions of the Swap treatment, and 8 concerning rescaled versions of the Addition treatment. The purpose of including re-scaled versions while disguising similarities is to augment sample size. We randomize the order in which options and Stakeholders are listed, which makes it difficult to spot the relationships between the decision tasks.

As in the main experiment, we ensure that our results are not an inadvertent artifact of a particular way of presenting outcome profiles by using \emph{all} three structurally distinct versions of conveying that information. Figures \ref{cardinalDisp0}, \ref{cardinalDisp1}, and \ref{cardinalDisp2} show a screenshot of each version. Each subject is randomly assigned to one of these display versions and completes the entire experiment in that version.

\paragraph{Parameters}


\textit{Swap treatments.} The following twelve tasks make up the Swap treatments. For two tasks, we use profiles $S1$ and $S1'$ with all amounts multiplied by a factor of two, and with $\epsilon_i=0$ for all $i$. Two other tasks are identical except that the strictly positive $\epsilon_i$ equal $0.25$. An additional group of four tasks parallel the first four, except that we use a multiplicative factor of 2.5, and we set the strictly positive values of $\epsilon_1$ equal to $1/5$. Yet another group of four tasks parallel the first two groups, except that we use a multiplication factor 3 and set the strictly positive values of $\epsilon_1$ equal to $1/6$.

The cardinal advantage we confer on option $A$ depends on the display format. With the format in Figure \ref{cardinalDisp0}, it is $\epsilon_1 = \epsilon_2 = \epsilon_3$ throughout, whereas with the remaining two formats, $\epsilon_1=\epsilon_2=0$ throughout.


\textit{Addition treatments.} The eight tasks that make up the Addition treatments belong to four pairs. Paired tasks are identical except that one includes option $X$ while the other does not. One pair is based on profile $A1$ with payoffs multiplied by 5 and $\delta = 0.4$. A second pair is based on $A1$ with payoffs multiplied by factor $4$ and $\delta = 0.5$. The remaining four rounds parallel the first four, except we use profile $A2$ instead of $A1$, and set $\nu=0.4$ and $\nu=0.5$ when the scaling factor is 5 and 4, respectively. With the display format for which no visual overlap occurs, we set $\nu= 0$ throughout.


\paragraph{Implementation}

Subjects read all instructions on-screen in English. We reproduce these instructions in Appendix \ref{instructionsCardinal}. Only those who passed two comprehension checks were allowed to complete the tasks. The first check ensures that subjects understand the payoff information displayed in the tasks. The second ensures that they understand the consequences of selecting any given payoff vector for the group of Stakeholders. We allow subjects who do not pass a check to revisit the instructions and try again.


\begin{figure}
\caption{Planner's decision interface, version 1 (axes stakeholders and payoffs) \label{cardinalDisp0}}
\begin{center}
\begin{tabular}{|c|}
\hline
\includegraphics[width=0.7\textwidth, angle=270,origin=c]{graphs/figureC1}
\\ \hline
\end{tabular}
\end{center}
\end{figure}


\begin{figure}
\caption{Planner's decision interface, version 2 (axes options and payoffs) \label{cardinalDisp1}}
\begin{center}
\begin{tabular}{|c|}
\hline
\includegraphics[width=0.7\textwidth]{graphs/figureC2}
\\ \hline
\end{tabular}
\end{center}
\end{figure}


\begin{figure}
\caption{Planner's decision interface, version 3 (axes individuals and options) \label{cardinalDisp2}}
\begin{center}
\begin{tabular}{|c|}
\hline
\includegraphics[width=0.7\textwidth, angle=270,origin=c]{graphs/figureC3}
\\ \hline
\end{tabular}
\end{center}
\end{figure}


\subsection{Supplementary results}\label{appendixCardinalResults}

In this section, we provide more detailed statistical analysis of the resulting choices. In particular, we provide separate results for each display format, and we perform robustness checks for treatments involving profiles $A1$ and $A2$.

Table \ref{choiceDistributionsCardinal} shows the choice distributions for each payoff profile category, both pooled across display formats and separately for each display format. Table \ref{statTestsCardinal} performs statistical tests of treatment effects. The first column pools across display formats and includes display format fixed effects. Columns 2, 3, and 4 then show results separately for each display format. We cluster all standard errors at the subject level.


\begin{table}
\caption{Choice distributions \label{choiceDistributionsCardinal}}
\begin{center}
\begin{tabular}{lcccccccccc}
\toprule
Profile          & $S1$           & $S1'$ & $S1$ & $S1'$ & $A1$    & $A1$ & $A2$    & $A2$ \\
 & $\epsilon_i = 0$ & $\epsilon_i = 0$ & $\epsilon_i > 0$ & $\epsilon_i > 0$ \\
Option $X$ available &&&&& Yes & No & Yes & No \\
\midrule
\multicolumn{9}{l}{Choice (pooled across displays)} \\
\quad A &     0.300 &     0.712 &     0.755 &     0.901 &     0.812 &     0.780 &     0.597 &     0.504 \\ \quad B &     0.687 &     0.275 &     0.233 &     0.085 &     0.175 &     0.220 &     0.396 &     0.496 \\ \quad C &     0.013 &     0.013 &     0.012 &     0.014 & - & - & - & - \\ \quad X & - & - & - & - &     0.013 & - &     0.007 & - \medskip \\ \multicolumn{9}{l}{Choice (display 1)} \\ \quad A &     0.440 &     0.644 &     0.842 &     0.874 &     0.842 &     0.879 &     0.729 &     0.687 \\ \quad B &     0.547 &     0.335 &     0.142 &     0.114 &     0.139 &     0.121 &     0.266 &     0.313 \\ \quad C &     0.012 &     0.021 &     0.016 &     0.012 & - & - & - & - \\\quad X & - & - & - & - &     0.018 & - &     0.005 & - \medskip \\ \multicolumn{9}{l}{Choice (display 2)} \\ \quad A &     0.293 &     0.673 &     0.750 &     0.910 &     0.855 &     0.795 &     0.565 &     0.458 \\ \quad B &     0.698 &     0.317 &     0.242 &     0.082 &     0.135 &     0.205 &     0.425 &     0.542 \\ \quad C &     0.008 &     0.010 &     0.008 &     0.008 & - & - & - & - \\\quad X & - & - & - & - &     0.010 & - &     0.010 & - \medskip \\ \multicolumn{9}{l}{Choice (display 3)} \\ \quad A &     0.170 &     0.818 &     0.675 &     0.919 &     0.740 &     0.668 &     0.500 &     0.374 \\ \quad B &     0.813 &     0.174 &     0.313 &     0.060 &     0.250 &     0.332 &     0.495 &     0.626 \\ \quad C &     0.017 &     0.009 &     0.012 &     0.021 & - & - & - & - \\\quad X & - & - & - & - &     0.010 & - &     0.005 & - \medskip \\
\bottomrule
\end{tabular}
\end{center}
\end{table}


Panel A of Table \ref{statTestsCardinal} compares the choice frequency of option $A$ for the case in which $\epsilon_i = 0$ for all $i$ across profiles $S1$ and $S1'$. While both baseline choice probability and effect size vary substantially across display formats, we observe large treatment effects for each of them. Panel B considers the parallel comparison for the case in which $\epsilon_i > 0$ for some $i$. Recall that for display format 1, we set $\epsilon_i >0$ for all $i$, whereas for display formats 2 and 3 we set $\epsilon_3 > 0$ but keep $\epsilon_1 = \epsilon_2 = 0$. While we find strong positive effects for display formats 2 and 3, we find a substantially smaller effect for display format 1. Possibly because we conferred a stronger cardinal advantage upon option $A$ in that display format, the baseline probability of selecting option $A$ is higher than for the other display formats. Panel C shows the effect of increasing $\epsilon_i$ separately for each display format with profile $S1$. Panel D shows the corresponding regressions for profile $S1'$. We see strong and highly statistically significant treatment effects for each display format and each of the two profiles.

\begin{table}
\caption{Statistical tests \label{statTestsCardinal}}
\begin{center}
\textbf{A. $S1$ vs. $S1'$ when $\epsilon_i = 0$ for all $i$}\\
\begin{tabular}{lcccc} \toprule
 & (1) & (2) & (3) & (4) \\
VARIABLES & \multicolumn{4}{c}{Option $A$ chosen} \\ \cmidrule(r){2-5}
Display format & All & 1 & 2 & 3 \\
\midrule
Profile S1' & 0.412 & 0.204 & 0.380 & 0.648 \\
 & (0.022) & (0.035) & (0.039) & (0.031) \\
Constant & 0.336 & 0.440 & 0.293 & 0.170 \\
 & (0.017) & (0.023) & (0.023) & (0.018) \\
 &  &  &  &  \\
Display format FE & Yes &  &  &  \\
Observations & 3,504 & 1,140 & 1,200 & 1,164 \\
 Subjects & 584 & 190 & 200 & 194 \\ \bottomrule
\end{tabular}
\medskip\\
\textbf{B. $S1$ vs. $S1'$ when $\epsilon_i > 0$ for some $i$}\\
\begin{tabular}{lcccc} \toprule
 & (1) & (2) & (3) & (4) \\
VARIABLES & \multicolumn{4}{c}{Option $A$ chosen} \\ \cmidrule(r){2-5}
Display format & All & 1 & 2 & 3 \\
\midrule
Profile S1' & 0.146 & 0.032 & 0.160 & 0.244 \\
 & (0.015) & (0.022) & (0.025) & (0.026) \\
Constant & 0.785 & 0.842 & 0.750 & 0.675 \\
 & (0.017) & (0.020) & (0.026) & (0.027) \\
 &  &  &  &  \\
Display format FE & Yes &  &  &  \\
Observations & 3,504 & 1,140 & 1,200 & 1,164 \\
 Subjects & 584 & 190 & 200 & 194 \\ \bottomrule
\end{tabular}
\medskip\\
\textbf{C. Effect of increasing $\epsilon_i$ in profile $S1$}\\
\begin{tabular}{lcccc} \toprule
 & (1) & (2) & (3) & (4) \\
VARIABLES & \multicolumn{4}{c}{Option $A$ chosen} \\ \cmidrule(r){2-5}
Display format & All & 1 & 2 & 3 \\
\midrule
Increased epsilon & 0.455 & 0.402 & 0.457 & 0.505 \\
 & (0.017) & (0.027) & (0.030) & (0.029) \\
Constant & 0.414 & 0.440 & 0.293 & 0.170 \\
 & (0.019) & (0.023) & (0.023) & (0.018) \\
 &  &  &  &  \\
Display format FE & Yes &  &  &  \\
Observations & 3,504 & 1,140 & 1,200 & 1,164 \\
 Subjects & 584 & 190 & 200 & 194 \\ \bottomrule
\end{tabular}
\medskip\\
\textbf{D. Effect of increasing $\epsilon_i$ in profile $S1'$}\\
\begin{tabular}{lcccc} \toprule
 & (1) & (2) & (3) & (4) \\
VARIABLES & \multicolumn{4}{c}{Option $A$ chosen} \\ \cmidrule(r){2-5}
Display format & All & 1 & 2 & 3 \\
\midrule
Increased epsilon & 0.189 & 0.230 & 0.237 & 0.101 \\
 & (0.013) & (0.023) & (0.025) & (0.020) \\
Constant & 0.664 & 0.644 & 0.673 & 0.818 \\
 & (0.017) & (0.020) & (0.023) & (0.020) \\
 &  &  &  &  \\
Display format FE & Yes &  &  &  \\
Observations & 3,504 & 1,140 & 1,200 & 1,164 \\
 Subjects & 584 & 190 & 200 & 194 \\ \bottomrule
\end{tabular}
\end{center}
\footnotesize
\end{table}

Panels A and B of Table \ref{statTestsCardinalSen}, respectively, show the effects of removing alternative $X$ from profiles $A1$ and $A2$. For profile $A1$, we see strong and significant effects with display formats 2 and 3. For display format 1, we observe a statistically insignificant effect with the opposite sign. Columns 5 to 8 repeat columns 1 to 4, but only include observations for which option $X$ was not chosen. The results are largely similar. For profile $A2$, we observe statistically significant effects (at the 5\% level) separately for each display format. Again, these results remain largely unchanged when we exclude observations for which the subjects selected $X$.

\begin{table}
\caption{Statistical tests \label{statTestsCardinalSen}}
\begin{center}
\textbf{A. $A1$ by availability of option $X$} \\
\setlength{\tabcolsep}{2pt}
\begin{tabular}{lcccccccc} \toprule
 & (1) & (2) & (3) & (4) & (5) & (6) & (7) & (8) \\
VARIABLES & \multicolumn{8}{c}{Option $A$ chosen} \\ \cmidrule(r){2-9}
Display format & All & 1 & 2 & 3 & All & 1 & 2 & 3 \\
\midrule
Option $X$ unavailable & -0.033 & 0.037 & -0.060 & -0.072 & -0.043 & 0.021 & -0.069 & -0.080 \\
 & (0.014) & (0.024) & (0.021) & (0.028) & (0.014) & (0.023) & (0.022) & (0.027) \\
Constant & 0.877 & 0.842 & 0.855 & 0.740 & 0.890 & 0.858 & 0.864 & 0.747 \\
 & (0.019) & (0.022) & (0.021) & (0.025) & (0.018) & (0.021) & (0.020) & (0.025) \\
 &  &  &  &  &  &  &  &  \\
Display format FE & Yes &  &  &  & Yes &  &  &  \\
Observations & 2,336 & 760 & 800 & 776 & 2,321 & 753 & 796 & 772 \\
 Subjects & 584 & 190 & 200 & 194 & 584 & 190 & 200 & 194 \\ \bottomrule
\end{tabular}
\medskip
\medskip\\
\textbf{B. $A2$ by availability of option $X$} \\
\setlength{\tabcolsep}{2pt}
\begin{tabular}{lcccccccc} \toprule
 & (1) & (2) & (3) & (4) & (5) & (6) & (7) & (8) \\
VARIABLES & \multicolumn{8}{c}{Option $A$ chosen} \\ \cmidrule(r){2-9}
Display format & All & 1 & 2 & 3 & All & 1 & 2 & 3 \\\midrule
Option $X$ unavailable & -0.092 & -0.042 & -0.108 & -0.126 & -0.097 & -0.046 & -0.113 & -0.129 \\
 & (0.015) & (0.021) & (0.025) & (0.029) & (0.015) & (0.021) & (0.025) & (0.028) \\
Constant & 0.754 & 0.729 & 0.565 & 0.500 & 0.758 & 0.733 & 0.571 & 0.503 \\
 & (0.029) & (0.030) & (0.032) & (0.032) & (0.029) & (0.029) & (0.032) & (0.032) \\
 &  &  &  &  &  &  &  &  \\
Display format FE & Yes &  &  &  & Yes &  &  &  \\
Observations & 2,336 & 760 & 800 & 776 & 2,328 & 758 & 796 & 774 \\
 Subjects & 584 & 190 & 200 & 194 & 584 & 190 & 200 & 194 \\ \bottomrule
\end{tabular}
\end{center}
\footnotesize
\end{table}


\newpage

\setcounter{table}{0}
\setcounter{figure}{0}

\section{General population experiment}

\setcounter{figure}{0}
\setcounter{table}{0}

\subsection{Design details and summary statistics}\label{app:genPopDesign}

Instructions are abridged versions from the laboratory experiment, but include the detailed presentation of the preference display. We only use versions 1 and 2 of presenting preference profiles (see Figures \ref{socialArchitectInterface0} and \ref{socialArchitectInterface1}, respectively), since version 3 is often perceived as less intuitive.

A native speaker of Swedish at a commercial translation agency translated the survey into Swedish. We aimed for 1000 respondents in each country. We began sampling with Dynata until no further subjects could be recruited. We then continued sampling the same survey with Lucid until no further subjects could be recruited (potential repeat participants were automatically filtered out by the Qualtrics survey). We retain subjects who participated through Dynata if they correctly answered the comprehension check about the preference display. For subjects recruited through Lucid we added a filter such that subjects could complete the survey only if they correctly answered these comprehension check questions. Because of these requirements, which are more stringent than typical for the subject population, we managed to obtain 711 subjects in Sweden and a comparable 805 subjects in the US. We recruited all Stakeholders with pollfish.

We exclude subjects who failed the comprehension check. Those recruited with Lucid were terminated upon failing the check. Those recruited with Dynata who failed the check could nonetheless finish the study, but we do not include them in our analysis.

Table \ref{demGenPop} presents the distribution of the demographic characteristics of the general population samples.

\begin{table}[htbp]
\caption{Demographic characteristics of the general population samples \label{demGenPop}}
\begin{small}
\begin{center}
\begin{tabular}{lcc}
\toprule
Variable & USA	 & Sweden \\
\midrule
\textbf{Gender}	&		&		\\
Male	&	0.509	&	0.560	\\
Female	&	0.489	&	0.433	\\
Non-binary	&	0.001	&	0.007	\\
\textbf{Age}	&		&		\\
18-25	&	0.098	&	0.184	\\
26-35	&	0.088	&	0.188	\\
36-45	&	0.062	&	0.141	\\
46-55	&	0.155	&	0.179	\\
56-65	&	0.160	&	0.191	\\
66-75	&	0.200	&	0.100	\\
76-85	&	0.214	&	0.015	\\
$>$85	&	0.022	&	0.001	\\
\textbf{Race}	&		&		\\
White (non-hispanic)	&	0.625	&		\\
Black	&	0.057	&		\\
White (hispanic)	&	0.255	&		\\
Other	&	0.063	&		\\
Native Swedish	&		&	0.778	\\
Other European background	&		&	0.124	\\
Other	&		&	0.098	\\
\textbf{Political party preference}	&		&		\\
Republican or other right-learning party	&	0.477	&		\\
Democrat or other left-learning party	&	0.523	&		\\
Left Party	&		&	0.149	\\
Social Democratic Party	&		&	0.249	\\
Green Party	&		&	0.065	\\
Centre Party	&		&	0.089	\\
Moderate Party	&		&	0.239	\\
Sweden Democrats	&		&	0.210	\\
\textbf{Education}	&		&		\\
Primary school	&	0.000	&	0.052	\\
Some high school	&	0.005	&	0.058	\\
High school or GED	&	0.145	&	0.266	\\
Some college	&	0.194	&	0.159	\\
Associate's or Bachelor's degree	&	0.419	&	0.412	\\
Master's degree	&	0.189	&	0.030	\\
Doctoral degree	&	0.048	&	0.024	\\
\textbf{Income bracket}	&		&		\\
$<$ USD 50k, $<$ SEK 500k	&	0.420	&	0.779	\\
between USD 50k and 100k or SEK 500k and 1,000k	&	0.349	&	0.208	\\
$>$ USD 100k, $>$ SEK 1,000k	&	0.231	&	0.013	\\
\textbf{Marital status}	&		&		\\
Married	&	0.511	&	0.370	\\
Widowed	&	0.097	&	0.038	\\
Divorced	&	0.142	&	0.090	\\
Separated	&	0.010	&	0.032	\\
Never married	&	0.241	&	0.470	\\
\\ \textbf{N}	&	805.000	&	711.000	\\
	&		&		\\
\bottomrule
\end{tabular}
\end{center}
\end{small}
\footnotesize
\end{table}


\subsection{Corroboration based on four-option profiles}\label{app:genPopFourOpt}

The general population experiment also includes the two four-option profiles of Table \ref{separatingProfiles} that separate between the class of proper scoring rules and the class of Condorcet extensions.

We find that only 8.0\% of U.S. subjects and 5.8\% of Swedish subjects consistently choose the Condorcet winner for both Condorcet-separating profiles (unweighted estimates). Moreover, 23.4\% (U.S.) and 14.6\% (Sweden) of those subjects express indifference between options for at least one of these profiles, which suggests that they follow plurality rule. That leaves roughly 5\% to 6\% of subjects in the Condorcet category. Likewise, small minorities (3.1\% of U.S. subjects and 1.7\% of Swedish subjects) choose the plurality runoff winner in both runoff-separating profiles. All of these results closely parallel our findings for the student sample.

\subsection{Robustness to survey weights}
\label{app:genPopRobustness}

Figure \ref{figClassificationGenPopUnweighted} replicates Figure \ref{figClassificationGenPop} without using survey weights. Classification frequencies remain very similar across the general population samples.

\begin{figure}[htp]
\caption{Classification of general population samples to pre-specified rules \label{figClassificationGenPopUnweighted}}
\vspace{-0.3cm}
\begin{center}
\includegraphics[width=0.8\textwidth]{graphs/figureD1}
\end{center}
\vspace{-0.8cm}
\footnotesize
\textbf{\textit{Notes:}} Bayes classifications based on consequential choices for the seven three-option preferences profiles indicated with stars in Table \ref{threeOptionProfiles}. Whiskers denote 95\% confidence intervals.
\end{figure}

We also explore how the frequencies change when we weigh observations to make the U.S. sample more representative with respect to race. (We leave the weights for the Swedish sample unchanged since \citet{StatisticsSweden} does not provide suitable data on race categories.) Using the 2018 General Social Survey \citep{GSS} we generate weights for 16 population categories, defined by (i) gender, (ii) political party preference, and (iii) race (white, hispanic, black, other). The differences between the U.S. and Swedish samples remain small and retain their direction, though the difference in the fraction of subjects assigned to strictly concave scoring rules now also becomes statistically significant. This result might be fragile, as it is driven by disproportionately large weights assigned to black participants, who represent only 5.7\% of the sample.


\subsection{Relation to attitudes about political representation}\label{app:externalValidity}

Here, we detail the relation between subjects' self-reported attitudes about political representation and our behavioral measure of aggregation preference. Next to rating, for each hypothetical political leader described in Section \ref{genPop}, the extent to which that person represents the will of the citizens, subjects also indicate their level of agreement or disagreement with each of the following two statements: ``\textit{The political system should strive for compromise solutions that everyone can live with even if the result is nobody's absolute favorite},'' and ``\textit{What the majority wants is right for a country, even if that makes some citizens suffer.}''


\begin{table}[htb!]
\setlength{\tabcolsep}{2pt}
\caption{Relation between behavioral measures of aggregation preference and attitudes about political representation \label{externalValidity}}
\begin{center}
\begin{tabular}{lcccc} \toprule
& (1) & (2) & (3) & (4) \\
VARIABLES & \multicolumn{2}{c}{Prefers compromise } & \multicolumn{2}{c}{Prefers compromise} \\
& \multicolumn{2}{c}{ candidate} & \multicolumn{2}{c}{ solutions} \medskip \\
Mean of the dep. var. & \multicolumn{2}{c}{0.801} & \multicolumn{2}{c}{0.687} \medskip \\
\cmidrule(r){2-3}\cmidrule(r){4-5}
            &            &            &            &            \\

Scoring parameter&       0.085&       0.081&       0.096&       0.078\\
            &     (0.042)&     (0.042)&     (0.056)&     (0.056)\\
& & & & \\
Demographic controls & & \checkmark & & \checkmark \\
N           &        1411&        1411&        1411&        1411\\
\bottomrule
\end{tabular}
\end{center}
\footnotesize
\textbf{\textit{Notes:}} OLS regressions. All regressions control for sample provider and display fixed effects. Demographic controls include nationality, gender, age, marital status, percentile rank of respondents' education within the sample in their respective country, indicators for being unemployed and for being part-time employed, as well as the percentile rank of income within the sample for the respondents' country.
\end{table}

Table \ref{externalValidity} shows the relation between these attitudes and our behavioral measure. For columns 1 and 2, the dependent variable is a binary indicator of preference for the compromise candidate. For columns 3 and 4, it is an index of preference for compromise policies constructed as follows: we assign values 0, $\frac{1}{3}$, $\frac{2}{3}$, and 1 to the responses `strongly disagree', `disagree', `agree', and `strongly agree' for the first of the two statement above, invert the scores for the second, and average responses across them. Each column reports an OLS regression pooling over U.S. and Swedish subjects. The main independent variable is the subject's scoring parameter, which we compute by restricting the Bayes classification to scoring rules and assigning interval midpoints. In all cases, we control for sample provider and display fixed effects. While the results are not strong, they are nevertheless suggestive. As expected, those who deploy more concave scoring rules tend to prefer greater compromise in political processes. The relationship is statistically significant at the 5\% level in column 1, at the 10\% level in columns 2 and 3, and statistically insignificant but directionally consistent in column 4.


\newpage
\section{Experiment instructions}


\subsection{Main experiment}\label{instructionsMain}
\includepdf[pages=-, landscape=true]{arrowProjectInstructions.pdf}

\subsection{Cardinal outcomes experiment}\label{instructionsCardinal}
\includepdf[pages=-, landscape=true]{instructionsCardinal.pdf}


\subsection{General population experiment}\label{instructionsGenPop}
\includepdf[pages=-, landscape=true]{arrowProjectInstructionsRepresentative.pdf}

\subsection{Elicitation of mTurk worker Stakeholder preferences}
\includepdf[pages=-, landscape=true]{arrowProjectInstructionsWorkers.pdf}

\subsection{Elicitation of Swiss citizen Stakeholder preferences}
\begin{center}
\includegraphics[width=\textwidth]{graphs/pollfishSurveyScreenshots.pdf}
\end{center}

\addcontentsline{toc}{section}{References}


\end{appendix}


\end{document}


\documentclass[smallheadings]{article}

\usepackage{tabularx,booktabs, placeins}
\newcolumntype{Y}{>{\centering\arraybackslash}X}

\usepackage{float}

\usepackage[]{graphicx}
\usepackage{mathrsfs}
\usepackage{mdwlist}
\usepackage{amssymb}
\usepackage{eucal}
\usepackage{units}
\usepackage{afterpage}


\usepackage{soul}

\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{epstopdf}
\usepackage{dsfont}
\usepackage{bm}
\usepackage{enumerate}
\usepackage[usenames, table]{xcolor}
\usepackage{hhline}
\definecolor{blue}{rgb}{0,0,1}
\definecolor{purple}{rgb}{0.6,0,1}
\usepackage{natbib}
\usepackage{multirow}
\usepackage[margin=1.2in]{geometry}
\usepackage{pdfpages}
\usepackage{float}
\usepackage{setspace}

\usepackage{pdflscape}
\usepackage{marvosym}


\usepackage{minitoc}
\usepackage[toc,page,header]{appendix}

\usepackage{multibib}
\newcites{appendix}{References}
\renewcommand \thepart{}
\renewcommand \partname{}


\usepackage[colorlinks = true, citecolor = blue, linkcolor = blue, urlcolor  = blue,]{hyperref}

\usepackage{rotating}
\usepackage[labelfont=bf]{caption}


\usepackage{array}
\newcolumntype{L}[1]{>{\raggedright\let\newline\\\arraybackslash\hspace{0pt}}m{#1}}
\newcolumntype{C}[1]{>{\centering\let\newline\\\arraybackslash\hspace{0pt}}m{#1}}
\newcolumntype{R}[1]{>{\raggedleft\let\newline\\\arraybackslash\hspace{0pt}}m{#1}}
\newcolumntype{Y}{>{\centering\arraybackslash}X}

\clubpenalty = 1000
\widowpenalty = 1000

\DeclareMathOperator{\plim}{plim}

\renewcommand{\theenumi}{(\roman{enumi})}
\renewcommand{\labelenumi}{\theenumi}
\newcommand{\bmath}{\begin{eqnarray}}
\newcommand{\emath}{\end{eqnarray}}
\newcommand{\bmathnn}{\begin{eqnarray*}}
\newcommand{\emathnn}{\end{eqnarray*}}
\newcommand{\bi}{\begin{itemize}}
\newcommand{\ei}{\end{itemize}}
\newcommand{\var}{\mathrm{var}}
\newcommand{\supp}{\mathrm{supp}}
\newcommand{\logit}{\mathrm{logit}}
\newcommand{\sech}{\mathrm{sech}}
\newcommand{\abs}{\mathrm{abs}}
\newcommand{\sgn}{\mathrm{sgn}}
\newcommand{\del}{\partial}
\newcommand{\w}{\omega}


\newcommand{\red}{\color{red}}
\newcommand{\gray}{\color{gray}}

\newcommand{\blue}{\color{blue}}
\newcommand{\green}{\color{green}}
\newcommand{\black}{\color{black}}
\newcommand{\purple}{\color{purple}}

\usepackage{ bbold }
\newcommand{\one}{\mathbb{1}}


\setcounter{MaxMatrixCols}{10}

\theoremstyle{plain}
\newtheorem{proposition}{Proposition}
\newtheorem{prediction}{Prediction}
\newtheorem*{example}{Example}
\setcounter{section}{0}
\newtheorem{theorem}[proposition]{Theorem}
\newtheorem{lemma}{Lemma}
\newtheorem{corollary}[proposition]{Corollary}
\newtheorem{result}{Result}
\newtheorem{hypothesis}{Hypothesis}
\theoremstyle{definition}
\newtheorem{definition}{Definition}
\newtheorem{assumption}{Assumption}
\newtheorem*{remark}{Remark}
\theoremstyle{plain}
\DeclareMathOperator*{\argmax}{argmax}
\DeclareMathOperator*{\argmin}{argmin}
\DeclareMathOperator{\core}{Core}
\DeclareMathOperator{\cov}{cov}
\usepackage{setspace}
\setstretch{1.4}


\usepackage{tikz}
\usetikzlibrary{fit,matrix}
\usepackage{collcell}
\definecolor{airforceblue}{rgb}{0.29, 0.44, 0.54}
\newcommand{\hhlline}[3]{\draw (#1-#2-2.north west) -- (#1-#2-#3.north east);}
\newcommand{\vllline}[3]{\draw (#1-5-#3.north west) -- (#1-#2-#3.north west);}
\newcommand{\vrlline}[3]{\draw (#1-5-#3.north east) -- (#1-#2-#3.north east);}

 \renewcommand{\thesection}{\Alph{section}}
 \renewcommand{\thesubsection}{\thesection.\arabic{subsection}}


\defcitealias{SEP.voting}{Stanford Encyclopedia of Philosophy (2016)}


\usepackage{xr}
\externaldocument{Social_Preferences_Over_Ordinal_Outcomes}


\begin{document}


\doparttoc
\faketableofcontents

\begin{appendix}

\renewcommand\thetable{\thesection.\arabic{table}}
\renewcommand\thefigure{\thesection.\arabic{figure}}
\setcounter{page}{0}

\begin{center}
{\Huge \textbf{ONLINE APPENDIX}}\\
\vspace{1cm}
{\Large   \textbf{\LARGE Social Preferences over Ordinal Outcomes}  \\  \bigskip \bigskip Sandro Ambuehl, B. Douglas Bernheim}
\end{center}


\pagenumbering{gobble}
\setstretch{1.2}
\addcontentsline{toc}{section}{Appendix}
\vspace{-0.9cm}
\part{}
\parttoc
\setstretch{1.4}

\newpage
\pagenumbering{arabic}


\setcounter{table}{0}
\setcounter{figure}{0}


\section{Main experiment: identification and design}

\subsection{Identification}


\paragraph{Distance between rules}\label{app:distanceBetweenRules}


Figure \ref{rulesDistance} tabulates, for each pair of social choice rules, the number of preference profiles (out of the 17 we use for identification) for which their implications differ. Two patterns stand out. First, in the vast majority of cases, different rules have different implications for large numbers of profiles. Ignoring the diagonal, few of the entries in Panel A are less than 3, and many are greater than 8, indicating differences on more than half of profiles. Second, three clusters of similar rules are visible. The first and second clusters consist of the strictly convex and strictly concave scoring rules, respectively. Because these rules are members of the single family that is indexed by a continuous parameter $s$, adjacent rules tend to differ on small numbers of profiles, but more distant scoring rules are more easily differentiated. The third cluster consists of the scoring runoff rules with $s<1$. Observe also that the Condorcet rule as well as the runoff rules, align more closely with convex scoring rules than with concave scoring rules. Hence, a preference for compromise may push subjects away from Condorcet extensions and runoff rules.


\begin{figure}[htbp!]
\caption{Distance between rules on the 17 profiles used in the work domain\label{rulesDistance}}
\begin{center}
\begin{tikzpicture}[draw=black, line width=0.05pt]\matrix (heatmap) [matrix of nodes, nodes in empty cells, row sep = -1.5\pgflinewidth, column sep=-1\pgflinewidth, nodes={draw, minimum size=0.475cm, anchor=center, align=center, inner sep=0, font=\tiny}]{
|[fill=none, draw=none, text=black, text width = 2.5cm]| & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=0$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $0<s<\frac{1}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{1}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{1}{3}<s<\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{1}{2}<s<\frac{3}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{3}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{3}{5}<s<\frac{2}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{2}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{2}{3}<s<\frac{3}{4}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{3}{4}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{3}{4}<s<\frac{4}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{4}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{4}{5}<s<1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $0\leq s\leq\frac{1}{4}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $\frac{1}{4}<s<\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $s=\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $\frac{1}{2}<s<1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $s=1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Condorcet & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Supermajority & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Unanimity\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=0$& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $0<s<\frac{1}{3}$& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{1}{3}$& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!94, draw=black, text=white]| 16\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{1}{3}<s<\frac{1}{2}$& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{1}{2}$& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!88, draw=black, text=white]| 15\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{1}{2}<s<\frac{3}{5}$& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{3}{5}$& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{3}{5}<s<\frac{2}{3}$& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{2}{3}$& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{2}{3}<s<\frac{3}{4}$& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{3}{4}$& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!94, draw=black, text=white]| 16\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{3}{4}<s<\frac{4}{5}$& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{4}{5}$& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{4}{5}<s<1$& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=1$& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $0\leq s\leq\frac{1}{4}$& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $\frac{1}{4}<s<\frac{1}{2}$& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $s=\frac{1}{2}$& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!94, draw=black, text=white]| 16\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $\frac{1}{2}<s<1$& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!65, draw=black, text=white]| 11& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $s=1$& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!53, draw=black, text=black]| 9& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!100, draw=black, text=white]| 17\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Condorcet& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!24, draw=black, text=black]| 4& |[fill=airforceblue!35, draw=black, text=black]| 6& |[fill=airforceblue!41, draw=black, text=black]| 7& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!18, draw=black, text=black]| 3& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!6, draw=black, text=black]| 1& |[fill=airforceblue!12, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 5& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!88, draw=black, text=white]| 15\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Supermajority& |[fill=airforceblue!71, draw=black, text=white]| 12& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!59, draw=black, text=black]| 10& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!76, draw=black, text=white]| 13& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!82, draw=black, text=white]| 14& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!47, draw=black, text=black]| 8\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Unanimity& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!94, draw=black, text=white]| 16& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!100, draw=black, text=white]| 17& |[fill=airforceblue!88, draw=black, text=white]| 15& |[fill=airforceblue!47, draw=black, text=black]| 8& |[fill=airforceblue!0, draw=black, text=black]| 0\\
};
\end{tikzpicture}
\end{center}
\footnotesize
\textbf{\textit{Notes:}} This graph plots the set of 23 benevolent rules on both the horizontal and vertical axes. Each cell reports the number of profiles (out of the 17 used in the work domain) for which a given pair of rules differ from each other. We use the definition that two rules differ on a profile if they select a different subset of options (distance = 1); otherwise they do not differ on that profile (distance = 0).
\end{figure}


\paragraph{Excluded three-option profiles}\label{excludedProfiles}

Given anonymity and neutrality, our setting of five Stakeholders and three alternatives allows for 42 different preference profiles. The work domain in our experiment uses the 17 profiles that provide the largest amount of discrimination between our pre-specified rules. Table \ref{excludedProfilesTable} lists the remaining 25 profiles that are not included in our experiment. These omitted profiles cannot expand the set of scoring rules, scoring runoffs, or $q$-majority rules we can distinguish. In the case of scoring rules and $q$-majority rules, this can be shown analytically. For the case of scoring runoff rules, we show this point using brute-force computer scripts.

Figure \ref{rulesDistanceOmitted} shows that these profiles would also provide little or no additional ability to distinguish between our pre-specified rules.


\begin{table}[htbp]
\caption{Excluded three-alternative profiles  \label{excludedProfilesTable}}
\begin{center}
\begin{small}
\setlength{\tabcolsep}{3pt}
\begin{tabular}{ccccc}
\toprule
\begin{tabular}{llllll}
 & A & A & A & A & A\\
 & B & B & B & B & B\\
 & C & C & C & C & C\\
& & & & & \\
 & A & A & A & A & A\\
 & C & C & B & B & B\\
 & B & B & C & C & C\\
& & & & & \\
 & C & C & A & A & A\\
 & A & A & B & B & B\\
 & B & B & C & C & C\\
& & & & & \\
 & B & B & A & A & A\\
 & A & A & B & B & B\\
 & C & C & C & C & C\\
& & & & & \\
 & A & A & A & A & A\\
 & C & B & B & B & B\\
 & B & C & C & C & C\\
& & & & & \\
\end{tabular}
&
\begin{tabular}{llllll}
 & A & A & A & B & B\\
 & C & B & B & A & A\\
 & B & C & C & C & C\\
& & & & & \\
 & A & A & A & B & B\\
 & B & C & C & A & A\\
 & C & B & B & C & C\\
& & & & & \\
 & C & C & A & A & A\\
 & A & B & B & B & B\\
 & B & A & C & C & C\\
& & & & & \\
 & B & B & A & A & A\\
 & A & C & C & C & B\\
 & C & A & B & B & C\\
& & & & & \\
 & B & B & A & A & A\\
 & A & C & C & B & B\\
 & C & A & B & C & C\\
& & & & & \\
\end{tabular}
&
\begin{tabular}{llllll}
 & B & A & A & A & A\\
 & C & B & B & B & C\\
 & A & C & C & C & B\\
& & & & & \\
 & B & A & A & A & A\\
 & C & B & B & C & C\\
 & A & C & C & B & B\\
& & & & & \\
 & C & A & A & A & A\\
 & B & C & B & B & B\\
 & A & B & C & C & C\\
& & & & & \\
 & C & B & A & A & A\\
 & A & C & B & B & B\\
 & B & A & C & C & C\\
& & & & & \\
 & C & B & A & A & A\\
 & A & C & B & B & C\\
 & B & A & C & C & B\\
& & & & & \\
\end{tabular}
&
\begin{tabular}{llllll}
 & B & C & A & A & A\\
 & A & B & C & B & B\\
 & C & A & B & C & C\\
& & & & & \\
 & C & B & B & A & A\\
 & A & A & A & B & B\\
 & B & C & C & C & C\\
& & & & & \\
 & C & B & B & A & A\\
 & A & A & A & B & C\\
 & B & C & C & C & B\\
& & & & & \\
 & B & C & A & A & A\\
 & A & A & C & B & B\\
 & C & B & B & C & C\\
& & & & & \\
 & B & C & A & A & A\\
 & A & A & B & B & B\\
 & C & B & C & C & C\\
& & & & & \\
\end{tabular}
&
\begin{tabular}{llllll}
 & C & A & A & A & A\\
 & A & B & B & B & B\\
 & B & C & C & C & C\\
& & & & & \\
 & C & A & A & A & A\\
 & A & B & B & B & C\\
 & B & C & C & C & B\\
& & & & & \\
 & B & A & A & A & A\\
 & A & C & C & B & B\\
 & C & B & B & C & C\\
& & & & & \\
 & B & A & A & A & A\\
 & A & C & B & B & B\\
 & C & B & C & C & C\\
& & & & & \\
 & B & A & A & A & A\\
 & A & B & B & B & B\\
 & C & C & C & C & C\\
& & & & & \\
\end{tabular} \\
\bottomrule
\end{tabular}

\end{small}
\end{center}
\footnotesize
\emph{Notes:} Most our pre-specified rules make the same prediction about which option will be chosen for most of the profiles in this table. Note that for the majority of these profiles, the preference rank distributions of the alternatives are fully ordered by stochastic dominance, \emph{and} the option with the lowest (least preferred) rank distribution is statewise dominated by another option.
\end{table}


\begin{figure}[htbp!]
\caption{Distance between rules used on the 25 omitted profiles\label{rulesDistanceOmitted}}
\begin{center}
\begin{tikzpicture}[draw=black, line width=0.05pt]\matrix (heatmap) [matrix of nodes, nodes in empty cells, row sep = -1.5\pgflinewidth, column sep=-1\pgflinewidth, nodes={draw, minimum size=0.475cm, anchor=center, align=center, inner sep=0, font=\tiny}]{
|[fill=none, draw=none, text=black, text width = 2.5cm]| & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=0$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $0<s<\frac{1}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{1}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{1}{3}<s<\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{1}{2}<s<\frac{3}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{3}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{3}{5}<s<\frac{2}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{2}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{2}{3}<s<\frac{3}{4}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{3}{4}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{3}{4}<s<\frac{4}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{4}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{4}{5}<s<1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $0\leq s\leq\frac{1}{4}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $\frac{1}{4}<s<\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $s=\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $\frac{1}{2}<s<1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $s=1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Condorcet & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Supermajority & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Unanimity\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=0$& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $0<s<\frac{1}{3}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{1}{3}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{1}{3}<s<\frac{1}{2}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{1}{2}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{1}{2}<s<\frac{3}{5}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{3}{5}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{3}{5}<s<\frac{2}{3}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{2}{3}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{2}{3}<s<\frac{3}{4}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{3}{4}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{3}{4}<s<\frac{4}{5}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{4}{5}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{4}{5}<s<1$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=1$& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!52, draw=black, text=black]| 13& |[fill=airforceblue!84, draw=black, text=white]| 21\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $0\leq s\leq\frac{1}{4}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $\frac{1}{4}<s<\frac{1}{2}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $s=\frac{1}{2}$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $\frac{1}{2}<s<1$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $s=1$& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Condorcet& |[fill=airforceblue!8, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!88, draw=black, text=white]| 22\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Supermajority& |[fill=airforceblue!36, draw=black, text=black]| 9& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!52, draw=black, text=black]| 13& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!44, draw=black, text=black]| 11& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!72, draw=black, text=white]| 18\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Unanimity& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!84, draw=black, text=white]| 21& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!88, draw=black, text=white]| 22& |[fill=airforceblue!72, draw=black, text=white]| 18& |[fill=airforceblue!0, draw=black, text=black]| 0\\
};
\end{tikzpicture}
\end{center}
\footnotesize
\textbf{\textit{Notes:}} This graph plots the set of 23 benevolent rules on both the horizontal and vertical axes. Each cell reports the number of profiles (out of the 25 profiles omitted from the experiment) for which a given pair of rules differ from each other. We use the definition that two rules differ on a profile if they select a different subset of options (distance = 1); otherwise they do not differ on that profile (distance = 0).
\end{figure}


\newpage
\paragraph{Distance between rules on profiles used the in political domain}\label{rulesDistPolit}

Figure \ref{rulesDistancePolitical} displays the distance between any pair of our pre-specified rules on the preference profiles we use for the political domain. Entries of zero off the diagonal indicate that the corresponding pair of rules cannot be separately identified using either of our methods. Moreover, using incentivized choice alone, we cannot separately identify any rule that differs from two other rules on only a single profile and whose choices are the union of the choices of the two other rules. This case occurs for some point-identified scoring rules with $0<s<1$.


\begin{figure}[H]
\caption{Distance between rules on the 7 profiles used in the political domain\label{rulesDistancePolitical}}
\begin{center}
\begin{tikzpicture}[draw=black, line width=0.05pt]\matrix (heatmap) [matrix of nodes, nodes in empty cells, row sep = -1.5\pgflinewidth, column sep=-1\pgflinewidth, nodes={draw, minimum size=0.475cm, anchor=center, align=center, inner sep=0, font=\tiny}]{
|[fill=none, draw=none, text=black, text width = 2.5cm]| & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=0$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $0<s<\frac{1}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{1}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{1}{3}<s<\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{1}{2}<s<\frac{3}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{3}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{3}{5}<s<\frac{2}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{2}{3}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{2}{3}<s<\frac{3}{4}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{3}{4}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{3}{4}<s<\frac{4}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=\frac{4}{5}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $\frac{4}{5}<s<1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Scoring, $s=1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $0\leq s\leq\frac{1}{4}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $\frac{1}{4}<s<\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $s=\frac{1}{2}$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $\frac{1}{2}<s<1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Runoff, $s=1$ & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Condorcet & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Supermajority & |[fill=none, draw=none, text=black, rotate=90, anchor=west]| Unanimity\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=0$& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $0<s<\frac{1}{3}$& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{1}{3}$& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{1}{3}<s<\frac{1}{2}$& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{1}{2}$& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!86, draw=black, text=white]| 6\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{1}{2}<s<\frac{3}{5}$& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{3}{5}$& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{3}{5}<s<\frac{2}{3}$& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{2}{3}$& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{2}{3}<s<\frac{3}{4}$& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{3}{4}$& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{3}{4}<s<\frac{4}{5}$& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=\frac{4}{5}$& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $\frac{4}{5}<s<1$& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Scoring, $s=1$& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $0\leq s\leq\frac{1}{4}$& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $\frac{1}{4}<s<\frac{1}{2}$& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $s=\frac{1}{2}$& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $\frac{1}{2}<s<1$& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Runoff, $s=1$& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Condorcet& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!14, draw=black, text=black]| 1& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!29, draw=black, text=black]| 2& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Supermajority& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!57, draw=black, text=black]| 4& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!71, draw=black, text=white]| 5& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!0, draw=black, text=black]| 0& |[fill=airforceblue!43, draw=black, text=black]| 3\\
|[fill=none, draw=none, text=black, text width=2.5cm, anchor=west, align=left]| Unanimity& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!86, draw=black, text=white]| 6& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!100, draw=black, text=white]| 7& |[fill=airforceblue!43, draw=black, text=black]| 3& |[fill=airforceblue!0, draw=black, text=black]| 0\\
};
\end{tikzpicture}
\end{center}
\footnotesize
\textbf{\textit{Notes:}} This graph plots the set of 23 benevolent rules on both the horizontal and vertical axes. Each cell reports the number of profiles (out of the 7 used in the political domain) for which a given pair of rules differ from each other. We use the definition that two rules differ on a profile if they select a different subset of options (distance = 1); otherwise they do not differ on that profile (distance = 0).
\end{figure}


\newpage
\subsection{Design details}

\paragraph{Overview}
Table \ref{experimentOverview} presents a schematic overview of the experiment.


\begin{table}[htb!]
\caption{Schematic overview of the experiment \label{experimentOverview}}

\begin{tabular}{|p{0.97\textwidth}|}

\hline \\

\textbf{Part 0: Initial instructions} \\ \medskip

\textbf{Part A: Task assigment}\\
\begin{enumerate}[1.]
\item Instructions concerning task assignment
\item Instructions about the interface that displays preference profiles
\item 25 task assignment decisions (intermingled)
\item 3 task assignment decisions with unavailable options (intermingled)
\end{enumerate}

\medskip
\textbf{Part B: Donation to a political party} \\
\begin{enumerate}[1.]
\item Instructions concerning the donation to a political party
\item 12 party donation decisions
\end{enumerate}

\medskip
\textbf{Part C: Further elicitations}
\begin{enumerate}[1.]
\item Beliefs
	\bi
	\item 5 rounds of incentivized belief elicitation about workers' reservation wages, followed by an unincentivized elicitation of own reservation wages for completing each of the five tasks
	\item 6 rounds of incentivized belief elicitation about citizens' political preferences, followed by an unincentivized elicitation of own willingness to pay to trigger or prevent the donation to each of the political parties
	\ei
\item Preferences
	\bi
	\item Risk preferences
	\item Altruism
	\ei
\item Other characteristics
	\bi
	\item Demographic information
	\item Cognitive Reflection Test
	\item Knowledge about social choice theory
	\ei
\end{enumerate}
\\
\hline
\end{tabular} \medskip\\
\footnotesize
\textbf{\textit{Notes:}} Each stage in Part C directly follows instructions concerning that stage. Half of the subjects proceed through the experiment in the order displayed. For the other half, Part B and Part A are interchanged. The latter subjects receive the instructions about the interface that displays preference profiles in Part B instead of in Part A. For those subjects, the two stages of Part C.1 are also interchanged.
\end{table}

\paragraph{Work tasks}\label{app:designDetails}


In the work domain, Planners decide about assignments of the following five tasks to Stakeholders: (i) \emph{Image labeling.} The worker views a sequence of 400 images and identifies each by clicking a button. (ii) \emph{Hate speech filtering.} The worker views 400 messages posted on twitter.com and indicates whether each includes hate speech such as racist or sexist statements. (iii) \emph{Audio transcription.} The worker listens to a sequence of 400 words and, in each case, identifies the word by clicking a button. (iv) \emph{Movie reviews classification.} The worker classifies 400 movie reviews according to whether they are positive or negative. (v) \emph{Assigning apprentices to mentors.} The worker finds a pairwise stable match between five hypothetical apprentices and five hypothetical mentors knowing their preferences. The worker completes 20 rounds of this task. Both Planners and Stakeholders not only read descriptions of each task, but also try each of them out  before making decisions (except for the matching task, a single round of which some subjects find time-consuming).


\paragraph{Alternative display formats}\label{additionalDisplayFormats}

In order to ensure our results are not an inadvertent artifact of a particular presentation of preference information, we consider \emph{all} structurally distinct ways of displaying preference profiles. Because a preference profile is an array of three-tuples of the form (option, worker, rank) that indicate the preference rank a particular worker assigns to a given option, there are three such possibilities: rows and columns can be ranks and workers (in which case options appear in cells), ranks and options (in which case workers appear in cells), or workers and options (in which case ranks appear in cells). We exposed the first possibility Figure \ref{socialArchitectInterface0} in the main text. Figures \ref{socialArchitectInterface1} and \ref{socialArchitectInterface2} show the second and third possibilities, respectively. We provide the same tools for exploration (hiding, highlighting, and rearranging) in all display versions. Each Planner sees one and only one presentation format, which we select at random.

For our experiment with Swedish and US general population respondents, we only use the first and second display version. Not only do they yield similar results in our main experiment as the third display version, but they are also more natural to process.

\begin{figure}[htb!]
\caption{Planners' decision interface, version 2 (individuals in cells) \label{socialArchitectInterface1}}
\begin{center}
\begin{tabular}{|c|}
\hline \\ \medskip
\includegraphics[height = 0.4\textheight]{graphs/figureA4}\\
\hline
\end{tabular}\\
\end{center}
\footnotesize \textbf{\textit{Notes:}} Subjects can drag and drop columns, click a button to shuffle columns, hide choice options, highlight workers, and hide workers. If a subject hides an option, only two rows are shown; the remaining rows are labeled `Best of 2' and `Worst of 2.' Symbols for options and colors for worker symbols are randomly drawn each round. The order of options is randomly drawn each round.
\end{figure}


\begin{figure}[htp!]
\caption{Planners' decision interface, version 3 (ranks in cells) \label{socialArchitectInterface2}}
\begin{center}
\begin{tabular}{|c|}
\hline \\ \medskip
\includegraphics[height = 0.4\textheight, angle=270,origin=c]{graphs/figureA5}\\
\hline
\end{tabular}\\
\end{center}
\footnotesize \textbf{\textit{Notes:}} Subjects can drag and drop columns, drag and drop rows, click a button to shuffle columns, click a button to shuffle rows, highlight preference ranks, hide choice options, and hide workers. If a subject hides an option, only two symbols, labeled `best' and `worst,' are shown. The colors of the symbols labeled `best' and `worst' differ depending on whether two or three options are displayed, to signify that the best (worst) option among two is not necessarily best (worst) among three. Symbols for options and colors for worker symbols are randomly drawn each round. The orders of workers and options are randomly drawn each round.  Colors for the preference ranks are randomly drawn on the individual level but remain constant throughout the experiment.
\end{figure}

\FloatBarrier


\paragraph{Details about decisions in the political domain} There are two differences between the political and work domain that are not mentioned in the main text. First, to prevent Planners from drawing inferences about parties from preference distributions, we select the group of five citizens from a sample that equally represents those who self-identify as politically left, right, and center. Planners are aware of the sample's composition. The second difference concerns the process for eliciting Stakeholders' preferences: citizens' expressed preferences determine the recipient of the donation with 2.5\% probability, versus 5\% probability for the analogous contingency in the work domain.

\paragraph{Additional elicitations}\label{additionalElicitations}

\textit{Beliefs about reservation prices.} In the elicitation of belies about stakeholder reservation prices in the work domain, possible answers range from \$0 to \$10 in increments of \$0.50. Half of the subjects see these questions ordered by rank (first, second, third), and half see them ordered by reverse rank (third, second, first). Before subjects answer these questions, we remind them about the price and income level in the US and the standard wages of mTurk workers. The instructions for these tasks explain the concept of WTP using an example for which we randomize the elicited valuation (either \$5 or \$1). We assume that the Planner's inferences about positional money-metric utility do not vary from one profile to another, and consequently elicit these beliefs only once.

The elicitations in the political domain are the same as for the work domain, with two exceptions. First, Possible predictions for the WTP to trigger the donation lie in the set \{$>$15, 15, 13, 11, 9, 7, 5, 3, 1, -1, -3, -5 ,-7, -9, -11, -13, -15, $<$-15\}. Over the five rounds, Planners make predictions for citizens with different first names rather than different cities of residence. Second, to increase statistical power, Planners answer five (nearly) redundant versions of these questions, where each version specifies the city and state where the worker lives. We use two sets of five city labels, randomly assigned, so we can determine whether the labels are consequential.

If one of the subject's reservation price predictions ends up determining her payment, she receives Fr. 30 minus Fr. 3 for every dollar by which her prediction differs from the truth, which we assess using incentivized multiple decision lists in preliminary sessions involving a separate set of workers. Planners also indicate their own reservation price for completing each of the five tasks, but these responses are unincentivized.


\textit{Risk preferences.} On each line of a multiple decision list, subjects choose between two lotteries. The first lottery pays Fr. 23 with probability $p$ or Fr. 15 with probability $(1-p)$. The second pays Fr. 38 with probability $p$ or Fr. 3 with probability $(1-p)$. The list includes 11 binary choices with $p\in\{0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1\}$. In a second otherwise identical decision list, the first lottery pays either Fr. 20 or Fr. 13, while the second pays either Fr. 34 or Fr. 5.

\textit{Altruism.} Subjects complete a multiple decision list involving the following pairs of alternatives: ``\emph{Increase each of the five workers' payoffs by \$2. Decrease my own study payment by CHF $X$},'' or ``\emph{Do not increase the group members' payoffs. Leave my own study payment unchanged.}'' Each line employs a different value of $X$ in the set $\{0, 1, 2, 3, 4, 6, 8, 10, 12, 14, 16, 18, 20\}$. In an otherwise identical second list, the first option is \emph{``Increase each of the five workers' payoffs by Fr. $X$  (exchanged to USD). Decrease my own study payment by Fr. $5X$},'' and the values of $X$  lie in \{$0.2, 0.4, 0.6, 0.8, 1, 1.2, 1.4, 1.6, 1.8, 2$\}. Notice that the first list implicates both efficiency and equity, while the second implicates only equity.
Subjects complete a multiple decision list involving the following pairs of alternatives: ``\emph{Increase each of the five workers' payoffs by \$2. Decrease my own study payment by CHF $X$},'' or ``\emph{Do not increase the group members' payoffs. Leave my own study payment unchanged.}'' Each line employs a different value of $X$ in the set $\{0, 1, 2, 3, 4, 6, 8, 10, 12, 14, 16, 18, 20\}$. In an otherwise identical second list, the first option is \emph{``Increase each of the five workers' payoffs by Fr. $X$  (exchanged to USD). Decrease my own study payment by Fr. $5X$},'' and the values of $X$  lie in \{$0.2, 0.4, 0.6, 0.8, 1, 1.2, 1.4, 1.6, 1.8, 2$\}. Notice that the first list implicates both efficiency and equity, while the second implicates only equity.


\textit{Demographics questions} We elicit the following demographic characteristics: gender, age, field of study and degree level they are working towards, grades in university entrance exams in mathematics and in their first language, canton in which they completed their university entrance exam, their main language, whether they live with their parents, their number of siblings, monthly spending, religiosity, religion, political stance, and political party they voted for in the last election of the Swiss National Council.

\textit{Questions about background in social choice theory} Subjects report whether they have taken a class covering pertinent topics. We illustrate a three-option, three-citizen cyclical preference profile and ask subjects to name the paradox. We also ask subjects to name the Borda rule and Arrow's impossibility theorem based on verbal descriptions.


\paragraph{Comprehension checks} Each of the two comprehension checks consists of nine questions. The first set concerns the preference displays and the second concerns the general decision environment. Subjects must answer all nine questions correctly to proceed. If they make a mistake, we ask them to review the instructions and try again, but we do not tell them which question(s) they missed. This feature prevents the subject from trying to pass the comprehension check by trial and error (the chance of passing either of the comprehension checks by chance is less than 0.2\%). The comprehension questions appear in Appendix \ref{instructionsMain}.


\newpage
\subsection{Bayes classifier}

\paragraph{Derivation of the Bayesian posterior}\label{bayesClassifierDerivation}

Here, we derive the explicit expression for the Bayesian posteriors, $P(R, \epsilon | c)$, that our Bayesian classifier maximizes. For each preference profile $t$, a social choice rule $R$ prescribes a subset $S_R^t \subseteq \{A, B, C\}$ of admissible options. For each $t$, the subject makes a choice $c_t \in \{A, B, C\}$. If the individual follows rule $R$ with error probability $\epsilon$ and behaves according to the assumptions listed in Section \ref{bayesClassifierIntroduction}, then the probability of choosing each alternative is given by the following expressions for $X, Y, Z \in \{A, B, C\}$ with $X$, $Y$, and $Z$ mutually distinct from each other.
\bmathnn
P\big(c_t = X | S_R^t = \{X\}; \epsilon \big) &=& 1 - \frac{2}{3}\epsilon \\
P\big(c_t = Y | S_R^t = \{X\}; \epsilon \big) &=& \frac{1}{3}\epsilon \\
P\big(c_t = X | S_R^t = \{X, Y\}; \epsilon \big) &=& \frac{1}{2} - \frac{1}{6}\epsilon \\
P\big(c_t = Z | S_R^t = \{X, Y\}; \epsilon \big) &=& \frac{1}{3}\epsilon \\
P\big(c_t = X | S_R^t = \{A,B,C\}; \epsilon \big) &=& \frac{1}{3}
\emathnn
Moreover, by the assumption of conditional independence across rounds, the probability of observing choice sequence $c = (c_1, \ldots, c_T)$ from a subject who follows rule $R$ is given by
$
P(c | R) = \prod_{t = 1}^T P\big(c_t | S_{R}^t\big).
$
Given the assumption of uniform prior probabilities across rules and error probabilities, we derive the prior probability of observing choice sequence $c$ as
$
P(c) = \sum_{R} \frac{1}{N_R} \int_0^1 P(c | R; \epsilon)d\epsilon,
$
where $N_ R$ is the total number of rules.
By Bayes' rule, the posterior associated with rule $R$ and error probability $\epsilon$ conditional on the sequence of choices $c$ is thus given by
\bmath\label{posterior}
P(R, \epsilon | c) = \frac{P(c | R; \epsilon) \mu_i}{P(c)}.
\emath

\paragraph{Monte Carlo Simulations}
\label{BayesMonteCarlo}

We use Monte Carlo simulations to test (i) whether the Bayesian classifier reliably detects the use of pre-specified rules, and (ii) whether noise introduces bias.

To answer the first question, we simulate a sample of 1,000 subjects. We uniformly randomly assign each simulated subject to one of the identifiable benevolent rules in each domain. Each simulated subject follows the assigned rule exactly and randomizes uniformly among all tied options in case of irresoluteness. We then run the Bayesian classifier on this sample of simulated subjects, both using all three-option profiles available in the work domain, and using the three-option profiles available in the political domain. Figure \ref{figClassifierVtruth} shows the results. Focusing on the profiles available in the work domain (Panel A), three features stand out. First, the data are generally tightly centered around the diagonal. Second, subjects following a massively irresolute rule (supermajority and unanimity) are frequently confused for following another rule. Yet, even in these cases, the rule generating the choices receives non-trivial weight. Hence if the classifier assigns zero weight to these two rules, it is unlikely that any subject actually followed one of these rules. Third, while scoring rules with a parameter between the boundaries of identifiable intervals are correctly classified in all cases, scoring rules with a parameter on the interval boundaries are sometimes confused for those with a parameter just above or just below the interval boundary. The reason is mechanical. In case of a scoring parameter on the interval boundary (henceforth: a point-identified scoring rule), the set of chosen options is the union of the options chosen by the neighboring interval-identified scoring rules. By assumption, subjects uniformly randomize in case of ties. If a point-identified scoring rule is irresolute on two profiles, and the neighboring interval-identified scoring rules are resolute on those profiles, for instance, there is a 50\% chance that the randomization over the ties happens to coincide exactly with the choices prescribed by one of the neighboring interval-identified scoring rules.  Panel B performs the same exercise but restricts the available data to the profiles available in the political domain and the set of rules to the rules identifiable in that domain. Qualitatively, we observe the same results as in the work domain.

\begin{figure}
\caption{Bayesian classifier if rules are followed exactly \label{figClassifierVtruth}}
\begin{center}
A. Work domain \\
\includegraphics[width=1\textwidth]{graphs/figureA6_A.pdf} \\
B. Political domain \\
\includegraphics[width=1\textwidth]{graphs/figureA6_B.pdf}
\end{center}
\footnotesize
\textbf{\textit{Notes}}: Each of 1,000 simulated subjects is randomly uniformly assigned one identifiable  benevolent rule in the respective domain and follows the rule exactly. In case of ties, simulated subjects randomize uniformly among all tied options. We run the Bayesian classifier on the simulated data. Fractions of subjects are indicated by the sizes of circles.
\end{figure}


To answer the second question, we simulate 4,000 subjects who choose uniformly randomly from all options in each round.  We then run the Bayesian classifier on this sample of simulated subjects, both using all three-option profiles available in the work domain, and using the three-option profiles available in the political domain. Figure \ref{noiseClassification} shows the resulting distribution of best-fitting types. Panel A uses the profiles available in the work domain. Unsurprisingly, close to half of simulated subjects are assigned to a malevolent rule.\footnote{Because our selection of profiles is not random, deviations from 50\% are expected.} Within the set of benevolent rules, we see that the least resolute rules, supermajority and unanimity, attract by far the largest fraction of subjects. The remaining rules all attract similar numbers of subjects, between one and six percent. Notably, neither of the modes we observe in our experimental data, Borda and near-antiplurality, attract a disproportionate fraction of randomly generated subjects.

Panel B uses the set of profiles available in the political domain. We find that 43.1\% of simulated subjects are classified as malevolent. Unlike in the work domain, we do not observe that randomly generated subjects are classified as following the unanimity rule disproportionately often. A possible reason is that the substantially smaller number of rounds in the political domain makes it more likely that a random vector of selections matches the other rules sufficiently well, especially given the leeway afforded by the high estimated noise parameter.

Overall, we conclude that our classification results in Section \ref{bayesClassifierResults} are not an artifact of classifying noisy data.


\begin{figure}
\caption{Bayesian classifier on uniformly random noise \label{noiseClassification}}
\begin{center}
A. Work domain \\
 (44.8\% classified as malevolent)\\
\includegraphics[width=0.6\textwidth]{graphs/figureA7_A} \\
B. Political domain\\
 (43.1\% classified as malevolent)\\
\includegraphics[width=0.6\textwidth]{graphs/figureA7_B}
\end{center}
\footnotesize

\textbf{\textit{Notes}}: The category ``Plurality equivalent'' includes scoring rules with $0\leq s < \frac{1}{3}$, any Condorcet extension, and any scoring runoff rule with $s<1$.'' Each of 4,000 simulated subjects makes uniformly random choices from all three options. We run the Bayesian classifier on the simulated data. Graphs display the distribution over benevolent rules only.
\end{figure}


\newpage

\setcounter{table}{0}
\setcounter{figure}{0}

\section{Main experiment: supplementary results}


\subsection{Classification using indifference data} \label{app:indifferenceData}

Next to selecting one option for each profile, Planners also indicate any alternatives they consider equally good as the one they have selected. While this expression of indifference has no consequences within our experiment, two considerations mitigate the usual concerns about hypothetical choices: first, we ask Planners to report indifference that they presumably would have recognized when making the associated consequential choice moments prior; second, misrepresenting indifference information would not serve any other plausible objective (e.g., enhancement of social image).

While the revelation of indifference can, in some cases, be incentivized by randomizing among the pertinent options, two considerations make this approach unsuitable for our purposes. First, the procedure changes the nature of social alternatives by introducing lotteries over assignments, thereby implicating workers' preferences over lotteries. Second, the social choice rules we consider do not specify random resolution. Adding that provision to a rule amounts to creating a non-standard extension.

We use an appropriately adjusted version of our Bayes classifier to analyze the indifference data. We continue to impose assumptions (i) and (iii), along with a slightly modified version of assumption (ii): when deviating from her rule, the subject randomizes uniformly over the seven subsets of options, rather than the three options. Under these assumptions, a rule maximizes the posterior probability if and only if it maximizes the number of profiles for which it predicts the correct subset. We break ties at random. Information on indifference  allows us to dispense with assumption (iv). In generating random benchmark data, we generate indifference data by designating each of the unchosen options as equally good based on independent Bernoulli draws. We choose the Bernoulli probability to match the average size of the best-option sets according to the actual subjects.


The average Planner expresses irresoluteness for 18\% of the profiles. In comparison, the set of our benevolent pre-specified rules produces a tie for 18.4\% of the profiles, on average. Focusing only on the Condorcet rule and the set of all scoring rules we can identify, this figure falls to 14.3\%. Planners designate all options as equally good for only 1.54\% of profiles.


\begin{figure}[htb!]
\caption{Best fitting pre-specified rules using indifference data\label{barChartIndifference}}
\begin{center}
\begin{tabular}{c}
\includegraphics[width = 0.8\textwidth]{graphs/figureB1} \\
\end{tabular}
\end{center}
\footnotesize
\textit{\textbf{Notes:}} The random-choice benchmark for malevolent rules exceeds the range of the graph.
\end{figure}

Figure \ref{barChartIndifference} displays the results. Unlike in the case of incentivized choices, indifference data lets us identify all 23 benevolent rules and their malevolent counterparts, yielding a total of 46 pre-specified rules. The results closely resemble those from our main specification (Figure \ref{barChart}). The perhaps most salient difference are the less pronounced modes at the linear and most concave scoring rules, and the slightly increased fraction of subjects assigned to the scoring rule with $\frac{1}{3}<s<\frac{1}{2}$. The overwhelming majority of choices continues to be consistent with a weakly concave scoring rule. Other rules continue to garner vanishing support.

The classifier based on indifference-data continues to achieve a good fit. Subjects' responses match the sets of options selected by their assigned rule for 14 of 17 profiles (83.5\%). In contrast, for our random-choice benchmark, the corresponding number is 43.2\%. As an alternative way of making this comparison, one can think of an empirical fingerprint as consisting of $3 \times 17 = 51$ binary variables, each of which indicates whether a given option is selected for a given profile. The average subject's choice coincides with the prediction of the best-fitting rule in 47 of these 51 cases (91.8\%), which is far higher than the random-choice benchmark (63.9\%).


\subsection{Corroboration based on four-option social choice problems}\label{analysisClassSeparating}\label{app:fourOptionProfiles}

Four-option profiles provide additional opportunities to distinguish cleanly between classes of social choice rules. Specifically, there are two four-option profiles, labeled ``Condorcet-separating 1'' and ``Condorcet-separating 2'' in Table \ref{fourOptionProfiles} (numbered 18 and 19), for which the Condorcet winner is \emph{rank-dominated}. These profiles distinguish between all Condorcet extensions and the entire class of \emph{proper} scoring rules (ones that do not assign the same score to any two ranks) because the former must select the Condorcet winner while the latter cannot select a rank-dominated option.

We randomly intermingle both of these profiles with the three-option profiles in both domain blocks. While four-option problems are more cognitively demanding, recall that the experimental interface allows subjects to hide options, which makes it easy to identify Condorcet winners.

\begin{table}[htb]
\caption{Four-alternative profiles \label{fourOptionProfiles}}
\setlength{\tabcolsep}{3pt}
\begin{small}
\begin{center}
\begin{tabular}{lccccccccccccc}
\toprule
Label  & Index & \multicolumn{5}{c}{Profile}   & Option selected& Condorcet & Plurality\\
&&&&&&&  by some proper  &  & (* = runoff)\\
&&&&&&&  scoring rule  & &   \\ \midrule
&                     &   &   &   &   &   &                          \\
Condorcet-separating 1 & 18                    & A & A & B & B & C & B or C      & A     & \{A$^*$,B\}    \\
                    & & B & B & A & C & D &                  &                 \\
                    & & C & C & C & D & A &                  &                 \\
                    & & D & D & D & A & B &                  &               \medskip  \\
Condorcet-separating 2 & 19                   & A & A & B & B & C & B or C   & A    & \{A$^*$,B\}    \\
                    &&  B & B & A & D & D &                  &                 \\
                    &&  C & C & C & C & A &                  &                \\
                    && D & D & D & A & B &                  &               \medskip  \\
Runoff-separating 1 &22 & A & A & B & D & D & A or B or C&  \{A, B, C, D\} & \{A,D$^*$\}   \\
  && B & B & C & A & C &              &                  \\
  && C & C & D & C & B &              &                  \\
  && D & D & A & B & A &              &                \medskip   \\
Runoff-separating 2 & 23 & A & A & C & D & D & A or B or C & \{A, B, C, D\} & \{A,D$^*$\}  \\
 & & B & B & B & A & B &              &                   \\
 & & C & C & D & C & C &              &                  \\
 & & D & D & A & B & A &              &   \medskip \\

\bottomrule
\end{tabular}
\end{center}
\end{small}
\footnotesize
\textbf{\emph{Notes:}} We display each preference profile as a 4$\times$5-matrix; columns correspond to workers, rows to preference ranks. The $r$th row shows a worker's $r$th-ranked alternative. For Condorcet-cyclical profiles, we indicate the set of options in the top-cycle.
\end{table}

As shown in the left half of Table \ref{separatingProfiles} (which pertains to the work domain), subjects choose the Condorcet winner roughly one-fifth of the time for each of these profiles. However, only 11.6\% consistently choose the Condorcet winner for both profiles.\footnote{This frequency, while relatively small, is significantly higher than would be observed by chance if the distributions of choices were independent across profiles, $0.205\times 0.212 = 0.043$ ($\chi^2$-test, $p<0.001$).} In contrast, because a negligible fraction of subjects choose option D, roughly 80\% of the individual choices are consistent with a proper scoring rule. Moreover, for 69.5\% of subjects, the pair of choices is consistent with some scoring rule in the following class: $[1, s_1, s_2, 0]$ where $s_1 = (2/3)^\gamma$, $s_2 = (1/3)^\gamma$, and $\gamma \in [0, \infty]$.


\begin{table}
\caption{Choices on class-separating profiles \label{separatingProfiles}}
\begin{center}
\begin{tabular}{lccccp{0.5cm}cccccccc}
\toprule
Domain & \multicolumn{4}{c}{Work} && \multicolumn{4}{c}{Politics} \\
\midrule
Option & A & B & C & D && A & B & C & D  \\
\cmidrule(r){2-5} \cmidrule(r){7-10}
Condorcet winner  & \checkmark &&&&& \checkmark \\
Optimal for some scoring rule &  &\checkmark&\checkmark&&& &\checkmark&\checkmark \\
\cmidrule(r){2-5} \cmidrule(r){7-10}

Condorcet-separating 1	 & 	0.205	 & 	0.694	 & 	0.099	 & 	0.002	 & 	 & 	0.188	 & 	0.746	 & 	0.064	 & 	0.002	 & 	\\
Condorcet-separating 2	 & 	0.212	 & 	0.719	 & 	0.064	 & 	0.005	 & 	 & 	0.188	 & 	0.800	 & 	0.012	 & 	0.000	 & 	\\
Consistent Condorcet	& \multicolumn{4}{c}{	0.116	} & & \multicolumn{4}{c}{	0.111	} 		 		 	 		 		 		 		 	\\
Consistent Scoring	& \multicolumn{4}{c}{	0.695	} & & \multicolumn{4}{c}{	0.732	} 		 		 	 		 		 		 		 	\\
\cmidrule(r){2-5} \cmidrule(r){7-10}
	 & 		 & 		 & 		 & 		 & 	 & 		 & 		 & 		 & 		 & 	\\
 Option & A & B & C & D && A & B & C & D  \\
\cmidrule(r){2-5} \cmidrule(r){7-10}
Plurality-runoff winner  &&&&\checkmark&& &&&\checkmark  \\
Optimal for some scoring rule & \checkmark &\checkmark&\checkmark&&& \checkmark&\checkmark&\checkmark \\
\cmidrule(r){2-5} \cmidrule(r){7-10}

Runoff-separating 1	 & 	0.257	 & 	0.385	 & 	0.314	 & 	0.044	 & 	 & 	0.353	 & 	0.405	 & 	0.188	 & 	0.054	 & 	\\
Runoff-separating 2	 & 	0.202	 & 	0.615	 & 	0.141	 & 	0.042	 & 	 & 	0.259	 & 	0.643	 & 	0.042	 & 	0.056	 & 	\\
Consistent Runoff	& \multicolumn{4}{c}{	0.012	} & & \multicolumn{4}{c}{	0.028	} 		 		 	 		 		 		 		 	\\
Consistent Scoring	& \multicolumn{4}{c}{	0.884	} & & \multicolumn{4}{c}{	0.930	} 		 		 	 		 		 		 		 	\\
\bottomrule
\end{tabular}
\end{center}
\footnotesize
\textbf{\textit{Notes:}} This table displays the fraction of subjects choosing each of the four options in each of the class-separating profiles. The first 262 subjects were not presented with the profile labelled Runoff-separating 2 in the political domain. The fraction of subjects consistently choosing in accordance with the plurality runoff rule in that domain is based on the remaining subjects.
\end{table}

The preceding estimates likely overstate the fraction of subjects who more broadly support Condorcet outcomes, because other rules can rationalize the selection of option A for profiles 18 and 19. As indicated in Table \ref{fourOptionProfiles}, plurality rule ($s_1 = s_2 = 0$) prescribes options A and B, while the plurality runoff rule prescribes A. Any scoring rule with $s_1 = s_2 < \frac{1}{2}$ also prescribes either A or B. One way to distinguish between Condorcet and plurality runoff on the one hand, and scoring rules with $s_1=s_2<\frac{1}{2}$ on the other, is to examine the indifference data. Of those who select A for both profiles, 36.2\% rate B as equally good, which suggests that more than a third of these selections reflect a scoring rule with $s_1=s_2<\frac{1}{2}$. Accordingly, choices for our four-option profiles imply that the fraction of subjects who implicitly follow some Condorcet rule in the work domain is no higher than (1 - 0.362)$\cdot$11.6\% = 7.4\%, which is consistent with our findings for three-option problems.

Profiles 22 and 23 in Table \ref{fourOptionProfiles}, which we also intermingled with the three-option profiles, leverage the principle of rank-dominance to provide clean separation between proper scoring rules and the plurality runoff rule. As shown in the left half of Table \ref{separatingProfiles}, subjects choose option D, the plurality runoff winner, roughly 5\% of the time. Only 1.2\% of subjects choose D for both profiles,\footnote{Though small, this fraction is significantly higher than would be observed by chance if the distributions of choices were independent across profiles, $0.044\times0.042=0.001848$ ($\chi^2$-test, $p<0.001$).} whereas 88.4\% of subjects behave as if both choices reflect a single scoring rule from the family specified above. Accordingly, only a small fraction of the Condorcet-consistent behavior for profiles 18 and 19 is likely attributable to use of the plurality runoff rule.

Four-option profiles such as 22 and 23 also enable more nuanced distinctions among rules. For example, they can provide a foundation for distinguishing concave scoring from a general aversion to lowest ranks; see Appendix \ref{lastPlaceAversion}.

We also note that our results for the four-option class-separating profiles in Table \ref{separatingProfiles} exhibit strikingly little domain sensitivity; choices in the work and political domains resemble each other. It follows that people are generally attracted to the selections of scoring rules and generally averse to those of majoritarian criteria.


\subsubsection{Last-place aversion}\label{lastPlaceAversion}

Does the concave aggregation we document in section \ref{classification} simply represent linear aggregation with a discount for receiving one's least-preferred alternative (last-place aversion; formalized as using scoring vectors that assign score $\frac{k-1}{K-1}$ to any option ranked $k\geq 2$, but score $-d<0$ to the option ranked last in the case of $K$ alternatives), or does it reflect globally concave aggregation in the sense that subjects' choices are described by score vectors that are strictly concave across all ranks (for instance by instance, by using scoring vectors that assign score $\left(\frac{k-1}{K-1}\right)^\gamma$ to the option ranked $k$ in the case of $K$ alternatives)? Choices in the runoff-separating profiles of Table \ref{fourOptionProfiles} show that globally concave aggregation plays a substantial role. In these profiles, the choice of option $A$ is consistent with a scoring rule with $s\leq 0.5$ whereas the choice of $B$ or $C$ is consistent with $s\in [\frac{1}{2}, \frac{2}{3}]$ and $s\in [\frac{2}{3}, 1]$, respectively, where $s = (\frac{1}{2})^\gamma$. Importantly, in these profiles, option $B$ is ranked last by one individual, and option $A$ is ranked last by two individuals. In the three-option profiles 3 and 4, 64\% of subjects choose the option consistent with a weakly concave scoring rule. If last-place aversion explains this choice pattern, then, in the four-option profiles 22 and 23, we should observe that at least 64\% of subjects avoid the last-place-generating options ($A$ or $B$). In contrast, the fraction of subjects selecting either option A or option B is given by 64.2\% and 81.7\% for the first and second runoff-separating profiles, respectively. Hence, last-place aversion cannot be the sole reason for the choice patterns we observe in the three-option profiles.

\FloatBarrier

\subsection{Response times by best-fitting rule}\label{appendixResponseTimes}

For each aggregation rule, Figure \ref{figResponseTimeByClassification} shows the median response times of subjects classified as best-fitting that rule. The most popular rules are not associated with unusually short response times. This suggests that subjects do not adopt specific rules just to save time.

\begin{figure}[H]
\caption{Median response time by classification based on incentivized data \label{figResponseTimeByClassification}}
\begin{center}
\includegraphics[width = 0.7\textwidth]{graphs/figureB2} \\
\end{center}
\footnotesize
\textbf{\textit{Notes:}} Spikes show 95\% confidence intervals with standard errors clustered by subject.
\end{figure}


\FloatBarrier

\subsection{Effects of preference displays}\label{analysisPreferenceDisplay}


Here, we study the influence of the format in which we presented preference profiles on our classification results. Figure \ref{barChartsByDisplay} displays the classification to pre-specified rules separately by display format. Graphs on the left hand side use incentivized choices only while those on the right hand side make use of indifference statements.

In each case we see that (i) malevolent and qualified majority rules receive vanishing support, (ii) runoff rules receive minor support, and (iii) the vast majority of subjects follow a concave scoring rule. Differences across the graphs mainly concern the modes of Borda and near-antiplurality. Relying on incentivized data alone, both of these rules emerge as the modal choices in each display version. While Borda is the more pronounced mode in display versions 1 and 3, near-antiplurality is the more pronounced mode in display version 2. This same pattern emerges to a larger extent if we incorporate indifference data for classification. Overall we conclude that our main results are robust to the preference display used, except that we cannot reliably distinguish whether Borda or near-antiplurality is the more pronounced mode.


\begin{figure}[htbp]
\caption{Best fitting pre-specified rules, work domain, by display\label{barChartsByDisplay}}
\begin{center}
\begin{tabular}{cc}
\multicolumn{2}{c}{\textbf{Display 1 (options in cells)}}\\
\midrule
Incentivized choice only & Using indifference data \\
\includegraphics[width = 0.475\textwidth]{graphs/figureB3_display1_col1} &
\includegraphics[width = 0.475\textwidth]{graphs/figureB3_display1_col2} \\
\multicolumn{2}{c}{\textbf{Display 2 (stakeholders in cells)}}\\
\midrule
Incentivized choice only & Using indifference data \\
\includegraphics[width = 0.475\textwidth]{graphs/figureB3_display2_col1} &
\includegraphics[width = 0.475\textwidth]{graphs/figureB3_display2_col2} \\
\multicolumn{2}{c}{\textbf{Display 3 (ranks in cells)}}\\
\midrule
Incentivized choice only & Using indifference data \\
\includegraphics[width = 0.475\textwidth]{graphs/figureB3_display3_col1} &
\includegraphics[width = 0.475\textwidth]{graphs/figureB3_display3_col2} \\
\end{tabular}
\end{center}
\footnotesize
\textbf{\textit{Notes:}} Displays 1, 2, and 3 are shown in Figures \ref{socialArchitectInterface0}, \ref{socialArchitectInterface1}, and  \ref{socialArchitectInterface2}, respectively.
\end{figure}

\newpage

\FloatBarrier

\subsection{Endogenous clusters}\label{appendixClustering}\label{app:clusteringDetails}

The $k$-modes algorithm we use begins by arbitrarily selecting $k$ subjects as initial cluster centers. Then it iterates two steps. First, each subject is assigned to the cluster for which the center matches her choices on the largest number of preference profiles. Second, cluster centers are updated: the new cluster center consists of the vector of modal choices for subjects assigned to that cluster. The algorithm terminates once the cluster centers stabilize.

We search for clusters in the work domain fixing $k=1, 2, 3, 5$, and $10$. For $k=1$, we run the algorithm 405 times, using each subject's choices as an initial cluster center once. For $k\geq2$, we run the algorithm 1000 times, in each case randomly setting the initial cluster centers equal to the choices of $k$ randomly selected subjects (excluding those who perfectly conform to pre-specified rules), and retaining the solution with the lowest total within-cluster distance.\footnote{We use the Hamming distance, i.e. the number of profiles on which two choice sequences differ from each other.} We limit the pre-specified rules to all scoring rules, all scoring runoff rules and all Condorcet extensions, which have a total of 298 resolute components. We exclude unanimity because the resolute components (of which there are $2^3\times3^{14} > 38\times10^6$) encompass all choice patterns that are consistent with the Pareto principle. Similarly, we exclude supermajority because it is massively irresolute, so the number of resolute components is enormous. These exclusions are likely inconsequential given the small number of subjects assigned to these rules in Section \ref{classification}, and in any case the exclusion of an empirically important rule can only increase the fraction of subjects assigned to endogenous clusters. Other exclusions are attributable to redundancies.

Table \ref{clusteringResults} displays the endogenous clusters uncovered by our clustering algorithm and the fractions of subjects assigned to each.


\begin{table}[htbp!]
\caption{Clustering results \label{clusteringResults}}
\small
\begin{center}
\begin{tabularx}{\textwidth}{l *{17}{Y}}
\toprule
Profile & 1 & 2 & 3 & 4 & 5 & 6 & 7 & 8 & 9 & 10 & 11 & 12 & 13 & 14 & 15 & 16 & 17\\
\midrule
Condorcet& A& B& A& A& A& A& A& A& A& A& A& ABC& ABC& A& A& A& A\\
Scoring \\
\quad $\bar{s}$ & $\nicefrac{1}{3}$	& $\nicefrac{1}{3}$ & $\nicefrac{1}{2}$ & $\nicefrac{1}{2}$ & $\nicefrac{1}{2}$	& $\nicefrac{3}{5}$ & $\nicefrac{2}{3}$ & $\nicefrac{2}{3}$ & $\nicefrac{3}{4}$	& $\nicefrac{3}{4}$ & $\nicefrac{4}{5}$ & 0 & $\nicefrac{1}{2}$ & - & 0 & 0 & 0 \\
\quad $s < \bar{s}$ & A & \mbox{A$^{a)}$} & A & A & A & A & A & A & A & A & A & \mbox{-$^{b)}$} & AC & AB & \mbox{-$^{a)}$} & \mbox{-$^{b)}$} & \mbox{-$^{a)}$}\\
\quad $s > \bar{s}$& B& B& B& B& B& B& B& B& B& B& B& B& B& AB& A$^{c)}$& A& A\\
\midrule
\\ \multicolumn{18}{c}{Endogenous clusters, $k=   1$} \\
\cmidrule(r){2-18}
\% subjects \\
\quad 4.69&B & B & B & B & B & B & B & B & A & B & B & B & B & A & A & A & A\\
Total: 4.69 \\
\\ \multicolumn{18}{c}{Endogenous clusters, $k=   2$} \\
\cmidrule(r){2-18}
\% subjects \\
\quad 4.69&B & B & B & B & B & B & B & B & A & B & B & B & B & A & A & A & A\\
\quad 1.73&B & B & B & A & B & B & A & B & B & A & B & B & B & B & A & A & A\\
Total: 6.42 \\
\\ \multicolumn{18}{c}{Endogenous clusters, $k=   3$} \\
\cmidrule(r){2-18}
\% subjects \\
\quad 4.69&B & B & B & B & B & B & B & B & A & B & B & B & B & A & A & A & A\\
\quad 1.73&B & B & B & A & B & B & A & B & B & A & B & B & B & B & A & A & A\\
\quad 0.25&B & B & B & B & C & C & B & C & C & B & C & A & B & C & C & A & B\\
Total: 6.67 \\
\\ \multicolumn{18}{c}{Endogenous clusters, $k=   5$} \\
\cmidrule(r){2-18}
\% subjects \\
\quad 4.69&B & B & B & B & B & B & B & B & A & B & B & B & B & A & A & A & A\\
\quad 1.73&B & B & B & B & B & B & B & B & A & B & B & B & B & B & A & A & A\\
\quad 1.73&B & B & B & A & B & B & A & B & B & A & B & B & B & B & A & A & A\\
\quad 1.48&B & B & A & A & B & B & B & A & A & A & A & B & B & B & A & A & A\\
\quad 0.74&B & B & A & B & B & B & B & A & C & B & B & B & B & C & C & A & A\\
Total: 10.37 \\
\\ \multicolumn{18}{c}{Endogenous clusters, $k=  10$} \\
\cmidrule(r){2-18}
\% subjects \\
\quad 4.69&B & B & B & B & B & B & B & B & A & B & B & B & B & A & A & A & A\\
\quad 1.73&B & B & B & A & B & B & A & B & B & A & B & B & B & B & A & A & A\\
\quad 1.48&B & B & A & A & B & B & B & A & A & A & A & B & B & A & A & A & A\\
\quad 1.23&B & B & A & A & B & B & A & A & A & A & A & B & B & A & A & A & A\\
\quad 0.99&B & B & B & B & B & B & A & A & B & A & A & B & B & B & A & A & A\\
\quad 0.74&B & B & B & B & B & B & A & B & A & A & A & B & C & A & A & A & A\\
\quad 0.74&B & B & B & B & B & B & B & A & B & A & B & B & B & A & A & A & A\\
\quad 0.25&B & B & B & B & C & C & B & C & C & B & C & A & B & C & C & A & B\\
\quad 0.25&C & C & C & A & C & B & A & A & B & C & C & C & A & C & C & C & A\\
\quad 0.25&B & C & A & A & A & A & B & B & A & B & B & B & B & A & C & B & A\\
Total: 12.35 \\
\bottomrule \end{tabularx}
\end{center}
\footnotesize
$^{a)}$ For $s = 0$, \{A,C\} is selected. \\
$^{b)}$ For $s = 0$, \{A,B\} is selected. \\
$^{c)}$ For $s = 1$, \{A,B\} is selected. \medskip \\
\footnotesize
\textbf{\textit{Notes:}} This table shows the clusters that emerge endogenously (in addition to those associated with our pre-specified rules) from application of a $k$-modes algorithm, along with the fractions of subjects assigned to each such cluster. For ease of comparison, the top section of the table shows choices for selected pre-specified rules.
\end{table}

\newpage
\subsection{Cardinal inferences}\label{app:syntheticScoringRules}


Table \ref{scoringCorrelatesDomain} replicates Table \ref{scoringCorrelates} separately for each domain. We observe parameter estimates on $\tilde{s}$ that are comparable in magnitude to those we obtain in the pooled sample. The estimates are larger in the political domain, and only statistically significant if we use the percentile rank of $\tilde{s}$ as a right-hand-side variable. Due to the smaller number of observations, a drop in statistical significance is expected. One possible explanation for the higher estimated coefficients in the political domain compared to the work domain is that participants possess more precise beliefs about the political domain, due to their greater familiarity with the Swiss political landscape than with the working conditions of US-based mTurk workers.


\begin{table}[htb!]
\caption{Relation between best-fitting and synthetic scoring parameters by domain \label{scoringCorrelatesDomain}}
\begin{normalsize}
\setlength{\tabcolsep}{3pt}
\begin{center}
\begin{tabular}{lcccccccc} \toprule
 & (1) & (2) & (3) & (4) & (5) & (6) & (7) & (8) \\
VARIABLE & \multicolumn{8}{c}{Estimated scoring parameter $s$} \\
\midrule
 & \multicolumn{4}{c}{Politics} & \multicolumn{4}{c}{Work} \\
\cmidrule(r){2-5} \cmidrule(r){6-9}
            &               &               &               &               &               &               &               &               \\
\textit{Scoring parameter $\tilde{s}$}&               &               &               &               &               &               &               &               \\
\quad Level &       0.123   &               &       0.117   &               &       0.098   &               &       0.081   &               \\
            &     (0.084)   &               &     (0.085)   &               &     (0.074)   &               &     (0.074)   &               \\
\quad \%-rank&               &       0.072   &               &       0.065   &               &       0.049   &               &       0.039   \\
            &               &     (0.036)   &               &     (0.036)   &               &     (0.031)   &               &     (0.032)   \\
\textit{Risk tolerance}&               &               &               &               &               &               &               &               \\
\quad Level &               &               &      -0.915   &               &               &               &      -0.536   &               \\
            &               &               &     (0.414)   &               &               &               &     (0.348)   &               \\
\quad \%-rank&               &               &               &      -0.070   &               &               &               &      -0.039   \\
            &               &               &               &     (0.036)   &               &               &               &     (0.031)   \\
\textit{Altruism}&               &               &               &               &               &               &               &               \\
\quad Level &               &               &       0.030   &               &               &               &       0.314   &               \\
            &               &               &     (0.286)   &               &               &               &     (0.251)   &               \\
\quad \%-rank&               &               &               &       0.041   &               &               &               &       0.052   \\
            &               &               &               &     (0.038)   &               &               &               &     (0.032)   \\
\hline
Observations & 398 & 398 & 398 & 398 & 394 & 394 & 394 & 394 \\
Subjects & 398 & 398 & 398 & 398 & 394 & 394 & 394 & 394 \\
\bottomrule
\end{tabular}
\end{center}
\end{normalsize}
\footnotesize
\textit{\textbf{Notes:}} Parameters estimated with OLS using midpoint values of best-fit score. Regressions include all subjects with monotonic beliefs about reservation prices. Risk tolerance is measured as the certainty equivalent of an equiprobable win 40 / win 10 lottery implied by individual-level CRRA estimates. Altruism is measured as the maximum amount of Swiss Francs the Planner is willing to give up to increase each stakeholder's payoff by \$2. For subjects with multiple switches in the risk preference or altruism elicitation, the corresponding variable is set to the mean value among the other subjects. We also include two indicator variables for the presence of multiple switching points, one for each characteristic. All regressions control for the type of preference profile presentation and for whether the political domain was displayed before or after the work domain. Standard errors are clustered by subject.
\end{table}


\newpage

\setcounter{table}{0}
\setcounter{figure}{0}

\section{Cardinal outcomes experiment} \label{appendixCardinal}

\setcounter{figure}{0}
\setcounter{table}{0}


\subsection{Design details}\label{appendixCardinalDesign}

Planners complete 20 decision tasks in random order, 12 concerning rescaled versions of the Swap treatment, and 8 concerning rescaled versions of the Addition treatment. The purpose of including re-scaled versions while disguising similarities is to augment sample size. We randomize the order in which options and Stakeholders are listed, which makes it difficult to spot the relationships between the decision tasks.

As in the main experiment, we ensure that our results are not an inadvertent artifact of a particular way of presenting outcome profiles by using \emph{all} three structurally distinct versions of conveying that information. Figures \ref{cardinalDisp0}, \ref{cardinalDisp1}, and \ref{cardinalDisp2} show a screenshot of each version. Each subject is randomly assigned to one of these display versions and completes the entire experiment in that version.

\paragraph{Parameters}


\textit{Swap treatments.} The following twelve tasks make up the Swap treatments. For two tasks, we use profiles $S1$ and $S1'$ with all amounts multiplied by a factor of two, and with $\epsilon_i=0$ for all $i$. Two other tasks are identical except that the strictly positive $\epsilon_i$ equal $0.25$. An additional group of four tasks parallel the first four, except that we use a multiplicative factor of 2.5, and we set the strictly positive values of $\epsilon_1$ equal to $1/5$. Yet another group of four tasks parallel the first two groups, except that we use a multiplication factor 3 and set the strictly positive values of $\epsilon_1$ equal to $1/6$.

The cardinal advantage we confer on option $A$ depends on the display format. With the format in Figure \ref{cardinalDisp0}, it is $\epsilon_1 = \epsilon_2 = \epsilon_3$ throughout, whereas with the remaining two formats, $\epsilon_1=\epsilon_2=0$ throughout.


\textit{Addition treatments.} The eight tasks that make up the Addition treatments belong to four pairs. Paired tasks are identical except that one includes option $X$ while the other does not. One pair is based on profile $A1$ with payoffs multiplied by 5 and $\delta = 0.4$. A second pair is based on $A1$ with payoffs multiplied by factor $4$ and $\delta = 0.5$. The remaining four rounds parallel the first four, except we use profile $A2$ instead of $A1$, and set $\nu=0.4$ and $\nu=0.5$ when the scaling factor is 5 and 4, respectively. With the display format for which no visual overlap occurs, we set $\nu= 0$ throughout.


\paragraph{Implementation}

Subjects read all instructions on-screen in English. We reproduce these instructions in Appendix \ref{instructionsCardinal}. Only those who passed two comprehension checks were allowed to complete the tasks. The first check ensures that subjects understand the payoff information displayed in the tasks. The second ensures that they understand the consequences of selecting any given payoff vector for the group of Stakeholders. We allow subjects who do not pass a check to revisit the instructions and try again.


\begin{figure}
\caption{Planner's decision interface, version 1 (axes stakeholders and payoffs) \label{cardinalDisp0}}
\begin{center}
\begin{tabular}{|c|}
\hline
\includegraphics[width=0.7\textwidth, angle=270,origin=c]{graphs/figureC1}
\\ \hline
\end{tabular}
\end{center}
\end{figure}


\begin{figure}
\caption{Planner's decision interface, version 2 (axes options and payoffs) \label{cardinalDisp1}}
\begin{center}
\begin{tabular}{|c|}
\hline
\includegraphics[width=0.7\textwidth]{graphs/figureC2}
\\ \hline
\end{tabular}
\end{center}
\end{figure}


\begin{figure}
\caption{Planner's decision interface, version 3 (axes individuals and options) \label{cardinalDisp2}}
\begin{center}
\begin{tabular}{|c|}
\hline
\includegraphics[width=0.7\textwidth, angle=270,origin=c]{graphs/figureC3}
\\ \hline
\end{tabular}
\end{center}
\end{figure}


\subsection{Supplementary results}\label{appendixCardinalResults}

In this section, we provide more detailed statistical analysis of the resulting choices. In particular, we provide separate results for each display format, and we perform robustness checks for treatments involving profiles $A1$ and $A2$.

Table \ref{choiceDistributionsCardinal} shows the choice distributions for each payoff profile category, both pooled across display formats and separately for each display format. Table \ref{statTestsCardinal} performs statistical tests of treatment effects. The first column pools across display formats and includes display format fixed effects. Columns 2, 3, and 4 then show results separately for each display format. We cluster all standard errors at the subject level.


\begin{table}
\caption{Choice distributions \label{choiceDistributionsCardinal}}
\begin{center}
\begin{tabular}{lcccccccccc}
\toprule
Profile          & $S1$           & $S1'$ & $S1$ & $S1'$ & $A1$    & $A1$ & $A2$    & $A2$ \\
 & $\epsilon_i = 0$ & $\epsilon_i = 0$ & $\epsilon_i > 0$ & $\epsilon_i > 0$ \\
Option $X$ available &&&&& Yes & No & Yes & No \\
\midrule
\multicolumn{9}{l}{Choice (pooled across displays)} \\
\quad A &     0.300 &     0.712 &     0.755 &     0.901 &     0.812 &     0.780 &     0.597 &     0.504 \\ \quad B &     0.687 &     0.275 &     0.233 &     0.085 &     0.175 &     0.220 &     0.396 &     0.496 \\ \quad C &     0.013 &     0.013 &     0.012 &     0.014 & - & - & - & - \\ \quad X & - & - & - & - &     0.013 & - &     0.007 & - \medskip \\ \multicolumn{9}{l}{Choice (display 1)} \\ \quad A &     0.440 &     0.644 &     0.842 &     0.874 &     0.842 &     0.879 &     0.729 &     0.687 \\ \quad B &     0.547 &     0.335 &     0.142 &     0.114 &     0.139 &     0.121 &     0.266 &     0.313 \\ \quad C &     0.012 &     0.021 &     0.016 &     0.012 & - & - & - & - \\\quad X & - & - & - & - &     0.018 & - &     0.005 & - \medskip \\ \multicolumn{9}{l}{Choice (display 2)} \\ \quad A &     0.293 &     0.673 &     0.750 &     0.910 &     0.855 &     0.795 &     0.565 &     0.458 \\ \quad B &     0.698 &     0.317 &     0.242 &     0.082 &     0.135 &     0.205 &     0.425 &     0.542 \\ \quad C &     0.008 &     0.010 &     0.008 &     0.008 & - & - & - & - \\\quad X & - & - & - & - &     0.010 & - &     0.010 & - \medskip \\ \multicolumn{9}{l}{Choice (display 3)} \\ \quad A &     0.170 &     0.818 &     0.675 &     0.919 &     0.740 &     0.668 &     0.500 &     0.374 \\ \quad B &     0.813 &     0.174 &     0.313 &     0.060 &     0.250 &     0.332 &     0.495 &     0.626 \\ \quad C &     0.017 &     0.009 &     0.012 &     0.021 & - & - & - & - \\\quad X & - & - & - & - &     0.010 & - &     0.005 & - \medskip \\
\bottomrule
\end{tabular}
\end{center}
\end{table}


Panel A of Table \ref{statTestsCardinal} compares the choice frequency of option $A$ for the case in which $\epsilon_i = 0$ for all $i$ across profiles $S1$ and $S1'$. While both baseline choice probability and effect size vary substantially across display formats, we observe large treatment effects for each of them. Panel B considers the parallel comparison for the case in which $\epsilon_i > 0$ for some $i$. Recall that for display format 1, we set $\epsilon_i >0$ for all $i$, whereas for display formats 2 and 3 we set $\epsilon_3 > 0$ but keep $\epsilon_1 = \epsilon_2 = 0$. While we find strong positive effects for display formats 2 and 3, we find a substantially smaller effect for display format 1. Possibly because we conferred a stronger cardinal advantage upon option $A$ in that display format, the baseline probability of selecting option $A$ is higher than for the other display formats. Panel C shows the effect of increasing $\epsilon_i$ separately for each display format with profile $S1$. Panel D shows the corresponding regressions for profile $S1'$. We see strong and highly statistically significant treatment effects for each display format and each of the two profiles.

\begin{table}
\caption{Statistical tests \label{statTestsCardinal}}
\begin{center}
\textbf{A. $S1$ vs. $S1'$ when $\epsilon_i = 0$ for all $i$}\\
\begin{tabular}{lcccc} \toprule
 & (1) & (2) & (3) & (4) \\
VARIABLES & \multicolumn{4}{c}{Option $A$ chosen} \\ \cmidrule(r){2-5}
Display format & All & 1 & 2 & 3 \\
\midrule
Profile S1' & 0.412 & 0.204 & 0.380 & 0.648 \\
 & (0.022) & (0.035) & (0.039) & (0.031) \\
Constant & 0.336 & 0.440 & 0.293 & 0.170 \\
 & (0.017) & (0.023) & (0.023) & (0.018) \\
 &  &  &  &  \\
Display format FE & Yes &  &  &  \\
Observations & 3,504 & 1,140 & 1,200 & 1,164 \\
 Subjects & 584 & 190 & 200 & 194 \\ \bottomrule
\end{tabular}
\medskip\\
\textbf{B. $S1$ vs. $S1'$ when $\epsilon_i > 0$ for some $i$}\\
\begin{tabular}{lcccc} \toprule
 & (1) & (2) & (3) & (4) \\
VARIABLES & \multicolumn{4}{c}{Option $A$ chosen} \\ \cmidrule(r){2-5}
Display format & All & 1 & 2 & 3 \\
\midrule
Profile S1' & 0.146 & 0.032 & 0.160 & 0.244 \\
 & (0.015) & (0.022) & (0.025) & (0.026) \\
Constant & 0.785 & 0.842 & 0.750 & 0.675 \\
 & (0.017) & (0.020) & (0.026) & (0.027) \\
 &  &  &  &  \\
Display format FE & Yes &  &  &  \\
Observations & 3,504 & 1,140 & 1,200 & 1,164 \\
 Subjects & 584 & 190 & 200 & 194 \\ \bottomrule
\end{tabular}
\medskip\\
\textbf{C. Effect of increasing $\epsilon_i$ in profile $S1$}\\
\begin{tabular}{lcccc} \toprule
 & (1) & (2) & (3) & (4) \\
VARIABLES & \multicolumn{4}{c}{Option $A$ chosen} \\ \cmidrule(r){2-5}
Display format & All & 1 & 2 & 3 \\
\midrule
Increased epsilon & 0.455 & 0.402 & 0.457 & 0.505 \\
 & (0.017) & (0.027) & (0.030) & (0.029) \\
Constant & 0.414 & 0.440 & 0.293 & 0.170 \\
 & (0.019) & (0.023) & (0.023) & (0.018) \\
 &  &  &  &  \\
Display format FE & Yes &  &  &  \\
Observations & 3,504 & 1,140 & 1,200 & 1,164 \\
 Subjects & 584 & 190 & 200 & 194 \\ \bottomrule
\end{tabular}
\medskip\\
\textbf{D. Effect of increasing $\epsilon_i$ in profile $S1'$}\\
\begin{tabular}{lcccc} \toprule
 & (1) & (2) & (3) & (4) \\
VARIABLES & \multicolumn{4}{c}{Option $A$ chosen} \\ \cmidrule(r){2-5}
Display format & All & 1 & 2 & 3 \\
\midrule
Increased epsilon & 0.189 & 0.230 & 0.237 & 0.101 \\
 & (0.013) & (0.023) & (0.025) & (0.020) \\
Constant & 0.664 & 0.644 & 0.673 & 0.818 \\
 & (0.017) & (0.020) & (0.023) & (0.020) \\
 &  &  &  &  \\
Display format FE & Yes &  &  &  \\
Observations & 3,504 & 1,140 & 1,200 & 1,164 \\
 Subjects & 584 & 190 & 200 & 194 \\ \bottomrule
\end{tabular}
\end{center}
\footnotesize
\end{table}

Panels A and B of Table \ref{statTestsCardinalSen}, respectively, show the effects of removing alternative $X$ from profiles $A1$ and $A2$. For profile $A1$, we see strong and significant effects with display formats 2 and 3. For display format 1, we observe a statistically insignificant effect with the opposite sign. Columns 5 to 8 repeat columns 1 to 4, but only include observations for which option $X$ was not chosen. The results are largely similar. For profile $A2$, we observe statistically significant effects (at the 5\% level) separately for each display format. Again, these results remain largely unchanged when we exclude observations for which the subjects selected $X$.

\begin{table}
\caption{Statistical tests \label{statTestsCardinalSen}}
\begin{center}
\textbf{A. $A1$ by availability of option $X$} \\
\setlength{\tabcolsep}{2pt}
\begin{tabular}{lcccccccc} \toprule
 & (1) & (2) & (3) & (4) & (5) & (6) & (7) & (8) \\
VARIABLES & \multicolumn{8}{c}{Option $A$ chosen} \\ \cmidrule(r){2-9}
Display format & All & 1 & 2 & 3 & All & 1 & 2 & 3 \\
\midrule
Option $X$ unavailable & -0.033 & 0.037 & -0.060 & -0.072 & -0.043 & 0.021 & -0.069 & -0.080 \\
 & (0.014) & (0.024) & (0.021) & (0.028) & (0.014) & (0.023) & (0.022) & (0.027) \\
Constant & 0.877 & 0.842 & 0.855 & 0.740 & 0.890 & 0.858 & 0.864 & 0.747 \\
 & (0.019) & (0.022) & (0.021) & (0.025) & (0.018) & (0.021) & (0.020) & (0.025) \\
 &  &  &  &  &  &  &  &  \\
Display format FE & Yes &  &  &  & Yes &  &  &  \\
Observations & 2,336 & 760 & 800 & 776 & 2,321 & 753 & 796 & 772 \\
 Subjects & 584 & 190 & 200 & 194 & 584 & 190 & 200 & 194 \\ \bottomrule
\end{tabular}
\medskip
\medskip\\
\textbf{B. $A2$ by availability of option $X$} \\
\setlength{\tabcolsep}{2pt}
\begin{tabular}{lcccccccc} \toprule
 & (1) & (2) & (3) & (4) & (5) & (6) & (7) & (8) \\
VARIABLES & \multicolumn{8}{c}{Option $A$ chosen} \\ \cmidrule(r){2-9}
Display format & All & 1 & 2 & 3 & All & 1 & 2 & 3 \\\midrule
Option $X$ unavailable & -0.092 & -0.042 & -0.108 & -0.126 & -0.097 & -0.046 & -0.113 & -0.129 \\
 & (0.015) & (0.021) & (0.025) & (0.029) & (0.015) & (0.021) & (0.025) & (0.028) \\
Constant & 0.754 & 0.729 & 0.565 & 0.500 & 0.758 & 0.733 & 0.571 & 0.503 \\
 & (0.029) & (0.030) & (0.032) & (0.032) & (0.029) & (0.029) & (0.032) & (0.032) \\
 &  &  &  &  &  &  &  &  \\
Display format FE & Yes &  &  &  & Yes &  &  &  \\
Observations & 2,336 & 760 & 800 & 776 & 2,328 & 758 & 796 & 774 \\
 Subjects & 584 & 190 & 200 & 194 & 584 & 190 & 200 & 194 \\ \bottomrule
\end{tabular}
\end{center}
\footnotesize
\end{table}


\newpage

\setcounter{table}{0}
\setcounter{figure}{0}

\section{General population experiment}

\setcounter{figure}{0}
\setcounter{table}{0}

\subsection{Design details and summary statistics}\label{app:genPopDesign}

Instructions are abridged versions from the laboratory experiment, but include the detailed presentation of the preference display. We only use versions 1 and 2 of presenting preference profiles (see Figures \ref{socialArchitectInterface0} and \ref{socialArchitectInterface1}, respectively), since version 3 is often perceived as less intuitive.

A native speaker of Swedish at a commercial translation agency translated the survey into Swedish. We aimed for 1000 respondents in each country. We began sampling with Dynata until no further subjects could be recruited. We then continued sampling the same survey with Lucid until no further subjects could be recruited (potential repeat participants were automatically filtered out by the Qualtrics survey). We retain subjects who participated through Dynata if they correctly answered the comprehension check about the preference display. For subjects recruited through Lucid we added a filter such that subjects could complete the survey only if they correctly answered these comprehension check questions. Because of these requirements, which are more stringent than typical for the subject population, we managed to obtain 711 subjects in Sweden and a comparable 805 subjects in the US. We recruited all Stakeholders with pollfish.

We exclude subjects who failed the comprehension check. Those recruited with Lucid were terminated upon failing the check. Those recruited with Dynata who failed the check could nonetheless finish the study, but we do not include them in our analysis.

Table \ref{demGenPop} presents the distribution of the demographic characteristics of the general population samples.

\begin{table}[htbp]
\caption{Demographic characteristics of the general population samples \label{demGenPop}}
\begin{small}
\begin{center}
\begin{tabular}{lcc}
\toprule
Variable & USA	 & Sweden \\
\midrule
\textbf{Gender}	&		&		\\
Male	&	0.509	&	0.560	\\
Female	&	0.489	&	0.433	\\
Non-binary	&	0.001	&	0.007	\\
\textbf{Age}	&		&		\\
18-25	&	0.098	&	0.184	\\
26-35	&	0.088	&	0.188	\\
36-45	&	0.062	&	0.141	\\
46-55	&	0.155	&	0.179	\\
56-65	&	0.160	&	0.191	\\
66-75	&	0.200	&	0.100	\\
76-85	&	0.214	&	0.015	\\
$>$85	&	0.022	&	0.001	\\
\textbf{Race}	&		&		\\
White (non-hispanic)	&	0.625	&		\\
Black	&	0.057	&		\\
White (hispanic)	&	0.255	&		\\
Other	&	0.063	&		\\
Native Swedish	&		&	0.778	\\
Other European background	&		&	0.124	\\
Other	&		&	0.098	\\
\textbf{Political party preference}	&		&		\\
Republican or other right-learning party	&	0.477	&		\\
Democrat or other left-learning party	&	0.523	&		\\
Left Party	&		&	0.149	\\
Social Democratic Party	&		&	0.249	\\
Green Party	&		&	0.065	\\
Centre Party	&		&	0.089	\\
Moderate Party	&		&	0.239	\\
Sweden Democrats	&		&	0.210	\\
\textbf{Education}	&		&		\\
Primary school	&	0.000	&	0.052	\\
Some high school	&	0.005	&	0.058	\\
High school or GED	&	0.145	&	0.266	\\
Some college	&	0.194	&	0.159	\\
Associate's or Bachelor's degree	&	0.419	&	0.412	\\
Master's degree	&	0.189	&	0.030	\\
Doctoral degree	&	0.048	&	0.024	\\
\textbf{Income bracket}	&		&		\\
$<$ USD 50k, $<$ SEK 500k	&	0.420	&	0.779	\\
between USD 50k and 100k or SEK 500k and 1,000k	&	0.349	&	0.208	\\
$>$ USD 100k, $>$ SEK 1,000k	&	0.231	&	0.013	\\
\textbf{Marital status}	&		&		\\
Married	&	0.511	&	0.370	\\
Widowed	&	0.097	&	0.038	\\
Divorced	&	0.142	&	0.090	\\
Separated	&	0.010	&	0.032	\\
Never married	&	0.241	&	0.470	\\
\\ \textbf{N}	&	805.000	&	711.000	\\
	&		&		\\
\bottomrule
\end{tabular}
\end{center}
\end{small}
\footnotesize
\end{table}


\subsection{Corroboration based on four-option profiles}\label{app:genPopFourOpt}

The general population experiment also includes the two four-option profiles of Table \ref{separatingProfiles} that separate between the class of proper scoring rules and the class of Condorcet extensions.

We find that only 8.0\% of U.S. subjects and 5.8\% of Swedish subjects consistently choose the Condorcet winner for both Condorcet-separating profiles (unweighted estimates). Moreover, 23.4\% (U.S.) and 14.6\% (Sweden) of those subjects express indifference between options for at least one of these profiles, which suggests that they follow plurality rule. That leaves roughly 5\% to 6\% of subjects in the Condorcet category. Likewise, small minorities (3.1\% of U.S. subjects and 1.7\% of Swedish subjects) choose the plurality runoff winner in both runoff-separating profiles. All of these results closely parallel our findings for the student sample.

\subsection{Robustness to survey weights}
\label{app:genPopRobustness}

Figure \ref{figClassificationGenPopUnweighted} replicates Figure \ref{figClassificationGenPop} without using survey weights. Classification frequencies remain very similar across the general population samples.

\begin{figure}[htp]
\caption{Classification of general population samples to pre-specified rules \label{figClassificationGenPopUnweighted}}
\vspace{-0.3cm}
\begin{center}
\includegraphics[width=0.8\textwidth]{graphs/figureD1}
\end{center}
\vspace{-0.8cm}
\footnotesize
\textbf{\textit{Notes:}} Bayes classifications based on consequential choices for the seven three-option preferences profiles indicated with stars in Table \ref{threeOptionProfiles}. Whiskers denote 95\% confidence intervals.
\end{figure}

We also explore how the frequencies change when we weigh observations to make the U.S. sample more representative with respect to race. (We leave the weights for the Swedish sample unchanged since \citet{StatisticsSweden} does not provide suitable data on race categories.) Using the 2018 General Social Survey \citep{GSS} we generate weights for 16 population categories, defined by (i) gender, (ii) political party preference, and (iii) race (white, hispanic, black, other). The differences between the U.S. and Swedish samples remain small and retain their direction, though the difference in the fraction of subjects assigned to strictly concave scoring rules now also becomes statistically significant. This result might be fragile, as it is driven by disproportionately large weights assigned to black participants, who represent only 5.7\% of the sample.


\subsection{Relation to attitudes about political representation}\label{app:externalValidity}

Here, we detail the relation between subjects' self-reported attitudes about political representation and our behavioral measure of aggregation preference. Next to rating, for each hypothetical political leader described in Section \ref{genPop}, the extent to which that person represents the will of the citizens, subjects also indicate their level of agreement or disagreement with each of the following two statements: ``\textit{The political system should strive for compromise solutions that everyone can live with even if the result is nobody's absolute favorite},'' and ``\textit{What the majority wants is right for a country, even if that makes some citizens suffer.}''


\begin{table}[htb!]
\setlength{\tabcolsep}{2pt}
\caption{Relation between behavioral measures of aggregation preference and attitudes about political representation \label{externalValidity}}
\begin{center}
\begin{tabular}{lcccc} \toprule
& (1) & (2) & (3) & (4) \\
VARIABLES & \multicolumn{2}{c}{Prefers compromise } & \multicolumn{2}{c}{Prefers compromise} \\
& \multicolumn{2}{c}{ candidate} & \multicolumn{2}{c}{ solutions} \medskip \\
Mean of the dep. var. & \multicolumn{2}{c}{0.801} & \multicolumn{2}{c}{0.687} \medskip \\
\cmidrule(r){2-3}\cmidrule(r){4-5}
            &            &            &            &            \\

Scoring parameter&       0.085&       0.081&       0.096&       0.078\\
            &     (0.042)&     (0.042)&     (0.056)&     (0.056)\\
& & & & \\
Demographic controls & & \checkmark & & \checkmark \\
N           &        1411&        1411&        1411&        1411\\
\bottomrule
\end{tabular}
\end{center}
\footnotesize
\textbf{\textit{Notes:}} OLS regressions. All regressions control for sample provider and display fixed effects. Demographic controls include nationality, gender, age, marital status, percentile rank of respondents' education within the sample in their respective country, indicators for being unemployed and for being part-time employed, as well as the percentile rank of income within the sample for the respondents' country.
\end{table}

Table \ref{externalValidity} shows the relation between these attitudes and our behavioral measure. For columns 1 and 2, the dependent variable is a binary indicator of preference for the compromise candidate. For columns 3 and 4, it is an index of preference for compromise policies constructed as follows: we assign values 0, $\frac{1}{3}$, $\frac{2}{3}$, and 1 to the responses `strongly disagree', `disagree', `agree', and `strongly agree' for the first of the two statement above, invert the scores for the second, and average responses across them. Each column reports an OLS regression pooling over U.S. and Swedish subjects. The main independent variable is the subject's scoring parameter, which we compute by restricting the Bayes classification to scoring rules and assigning interval midpoints. In all cases, we control for sample provider and display fixed effects. While the results are not strong, they are nevertheless suggestive. As expected, those who deploy more concave scoring rules tend to prefer greater compromise in political processes. The relationship is statistically significant at the 5\% level in column 1, at the 10\% level in columns 2 and 3, and statistically insignificant but directionally consistent in column 4.


\newpage
\section{Experiment instructions}


\subsection{Main experiment}\label{instructionsMain}
\includepdf[pages=-, landscape=true]{arrowProjectInstructions.pdf}

\subsection{Cardinal outcomes experiment}\label{instructionsCardinal}
\includepdf[pages=-, landscape=true]{instructionsCardinal.pdf}


\subsection{General population experiment}\label{instructionsGenPop}
\includepdf[pages=-, landscape=true]{arrowProjectInstructionsRepresentative.pdf}

\subsection{Elicitation of mTurk worker Stakeholder preferences}
\includepdf[pages=-, landscape=true]{arrowProjectInstructionsWorkers.pdf}

\subsection{Elicitation of Swiss citizen Stakeholder preferences}
\begin{center}
\includegraphics[width=\textwidth]{graphs/pollfishSurveyScreenshots.pdf}
\end{center}

\addcontentsline{toc}{section}{References}


\end{appendix}


\end{document}
