\documentclass[12pt, a4paper]{article}


\usepackage[margin=1.25in]{geometry}

\usepackage{tabularx,booktabs, makecell, varwidth, placeins}
\usepackage{textcomp}
\newcolumntype{Y}{>{\centering\arraybackslash}X}
\newcolumntype{L}[1]{>{\raggedright\arraybackslash}p{#1}}
\newcolumntype{C}[1]{>{\centering\arraybackslash}p{#1}}
\newcolumntype{R}[1]{>{\raggedleft\arraybackslash}p{#1}}

\usepackage{float}
\usepackage{longtable, caption}
\usepackage{changepage}


\usepackage[]{graphicx}
\usepackage{mathrsfs}
\usepackage{mdwlist}
\usepackage{amssymb}
\usepackage{eucal}
\usepackage{units}
\usepackage{rotating}
\usepackage{ulem}

\usepackage{soul}

\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{epstopdf}
\usepackage{dsfont}
\usepackage{bm}
\usepackage{enumerate}
\usepackage[usenames, table]{xcolor}
\usepackage{hhline}
\definecolor{blue}{rgb}{0,0,1}
\definecolor{purple}{rgb}{0.6,0,1}

\usepackage{pgfplotstable}

\usepackage{natbib}
\usepackage{multirow}
\usepackage{pdfpages}

\usepackage{setspace}
\usepackage{afterpage}

\usepackage{pdflscape}
\usepackage{marvosym}

\usepackage{adjustbox}

\usepackage{minitoc}
\usepackage[toc,page,header]{appendix}

\usepackage{multibib}
\newcites{appendix}{References}
\renewcommand \thepart{}
\renewcommand \partname{}

\newcommand{\quoteleft}{"}
\newcommand{\quoteright}{"}

\newcommand{\indep}{\perp \!\!\! \perp}
\newcommand{\nindep}{\not\!\perp \!\!\! \perp}

\usepackage[colorlinks = true, citecolor = blue, linkcolor = blue, urlcolor  = blue]{hyperref}
\usepackage[labelfont=bf]{caption}

\usepackage{array}
\newcolumntype{L}[1]{>{\raggedright\let\newline\\\arraybackslash\hspace{0pt}}m{#1}}
\newcolumntype{C}[1]{>{\centering\let\newline\\\arraybackslash\hspace{0pt}}m{#1}}
\newcolumntype{R}[1]{>{\raggedleft\let\newline\\\arraybackslash\hspace{0pt}}m{#1}}
\newcolumntype{Y}{>{\centering\arraybackslash}X}

\clubpenalty = 10000
\widowpenalty = 10000


\onehalfspacing
\usepackage[bottom]{footmisc}
\usepackage{etoolbox}
\AtBeginEnvironment{thebibliography}{\singlespacing}
\renewcommand{\footnotelayout}{\setstretch{1}}


\DeclareMathOperator{\plim}{plim}

\renewcommand{\theenumi}{(\roman{enumi})}
\renewcommand{\labelenumi}{\theenumi}
\newcommand{\bmath}{\begin{eqnarray}}
\newcommand{\emath}{\end{eqnarray}}
\newcommand{\bmathnn}{\begin{eqnarray*}}
\newcommand{\emathnn}{\end{eqnarray*}}
\newcommand{\bi}{\begin{itemize}}
\newcommand{\ei}{\end{itemize}}
\newcommand{\var}{\mathrm{var}}
\newcommand{\supp}{\mathrm{supp}}
\newcommand{\logit}{\mathrm{logit}}
\newcommand{\sech}{\mathrm{sech}}
\newcommand{\abs}{\mathrm{abs}}
\newcommand{\sgn}{\mathrm{sgn}}
\newcommand{\corr}{\mathrm{corr}}
\newcommand{\del}{\partial}
\newcommand{\w}{\omega}
\newcommand{\mat}[1]{\boldsymbol{#1}}

\newcommand{\red}{\color{red}}
\newcommand{\gray}{\color{gray}}

\newcommand{\blue}{\color{blue}}
\newcommand{\green}{\color{green}}
\newcommand{\black}{\color{black}}
\newcommand{\purple}{\color{purple}}

\usepackage{ bbold }
\newcommand{\one}{\mathbb{1}}


\setcounter{MaxMatrixCols}{10}

\theoremstyle{plain}
\newtheorem{observation}{Observation}
\newtheorem{proposition}{Proposition}
\newtheorem{prediction}{Prediction}
\newtheorem*{example}{Example}
\setcounter{section}{0}
\newtheorem{theorem}[proposition]{Theorem}
\newtheorem{lemma}{Lemma}
\newtheorem{corollary}[proposition]{Corollary}
\newtheorem{result}{Result}
\newtheorem{hypothesis}{Hypothesis}

\newtheorem{definition}{Definition}
\newtheorem{assumption}{Assumption}
\newtheorem*{remark}{Remark}
\theoremstyle{plain}
\DeclareMathOperator*{\argmax}{argmax}
\DeclareMathOperator*{\argmin}{argmin}
\DeclareMathOperator{\core}{Core}
\DeclareMathOperator{\cov}{cov}
\usepackage{setspace}
\setstretch{1.4}

\defcitealias{SEP.voting}{Stanford Encyclopedia of Philosophy (2016)}


\newcommand{\hide}[1]{}
\renewcommand{\hide}[1]{#1}


\newcommand{\alternative}[1]{{\blue{[#1]}}}


\newcommand{\insertbottomrule}{\\ \bottomrule}


\usepackage{tikz}
\usepackage{collcell}
\definecolor{airforceblue}{rgb}{0.29, 0.44, 0.54}


\defcitealias{equivalenceScale}{OECD, 2022}

\usetikzlibrary{fit,matrix}
\usetikzlibrary{positioning}
\usetikzlibrary{arrows.meta}


\renewcommand{\emph}[1]{\textit{#1}}

\newcommand{\papertitle}{\vspace*{-2cm}
 Choice with Competing Models: \\An Experimental Study}


\begin{document}

\title{\papertitle}
\author{Sandro Ambuehl \textregistered\; Heidi C. Thysen \footnote{Ambuehl: Department of Economics and UBS Center for Economics in Society, University of Zurich, Bl\"uemlisalpstrasse 10, 8006 Z\"urich, Switzerland, sandro.ambuehl@econ.uzh.ch. Thysen: Department of Economics, Norwegian School of Economics, Helleveien 30, 5045 Bergen, Norway, heidi.thysen@gmail.com. A previous version of this draft based on a different design and different experimental data circulated under the title `Competing Causal Interpretations: An Experimental Study.' We are grateful to Chiara Aina, Gonzalo Arrieta, Bj\"orn Bartling, Ernst Fehr, Gilat Levy, Chad Kendall, Nick Netzer, Rani Spiegler, Jakub Steiner, Severine Toussaert, Bertil Tungodden, Roberto Weber, and Georg Weizs\"acker for helpful comments and suggestions, to Timo Huber for his excellent programming of the experiment, and to Eva K\"upper and Nandita Gupta for excellent research assistance. This project was funded by a generous grant from FAIR -- Centre for Experimental Research on Fairness, Inequality and Rationality at the Norwegian School of Economics, the  Department of Economics at the University of Zurich, and the SNSF Starting Grant \#211222. All experiments were approved by the Ethics Review Board of the Department of Economics at the University of Zurich and the Norwegian School of Economics.}}

\maketitle
\thispagestyle{empty}

\begin{abstract}
People often make choices with imperfect knowledge of how the variables in their decision problem are related. We study such choices when individuals face menus of conflicting and possibly misspecified models that link these variables. Do they discard inaccurate models, what types of inaccuracies do they detect, and how? Or do they instead follow models that sound appealing at face value, and what determines that appeal? Our experiment yields two main findings. First, many individuals readily intuit the models’ predicted correlations and reject models that contradict the data. Performance is high because the required inference is qualitative rather than quantitative. Second, when unable to identify the correct model, most choose cautiously by focusing on worst-case outcomes. This behavior contradicts the Narrative Competition literature’s assumption of best-case maximization, but a failure of contingent reasoning when interpreting models’ payoff implications can mimic that assumption. Our results are robust to tripled stakes.\\ \textit{JEL codes: C91, D01, D83}
\end{abstract}


\setcounter{tocdepth}{3}
\setcounter{secnumdepth}{3}


\doparttoc
\newpage
\faketableofcontents


\part{}

\setcounter{tocdepth}{3}
\tableofcontents

\parttoc
\newpage


\vspace{-1.3cm}
\section{Introduction}\label{intro}
\label{intro} \setcounter{page}{1}


A vast literature on belief updating in economics studies how individuals use information for choice when the structure of the data-generating process (DGP) is known \citep[see, e.g.][]{Benjamin.Review}. Yet, many decision problems lack structural information, such as which variables in the decision problem are related and in what way.  How do individuals learn and choose when they face uncertainty about the structure of the data-generating process?

We consider a setting in which individuals face a menu of conflicting and possibly misspecified models fitted to the DGP. These models make claims about the relationships between variables and differ in their implications for choice and utility. Individuals decide which model to follow. This framework captures, for instance, political referenda in which voters choose among a small number of proposals, each based on a different structural understanding of the world. It likewise applies to managers choosing among a few action plans prepared by teams with differing structural views of the competitive environment.


We study how individuals choose between such models. On the one hand, individuals may recognize and discard models that are inconsistent with the data (\emph{Fact-Based} decision making). If so, how do they make these decisions? Do they look for qualitative inconsistencies or do they focus on broad quantitative fit? What  model implications can they intuit and connect to the data, and what types of inconsistencies do they fail to recognize? On the other hand, individuals may follow models whose implications for utility and choice seem appealing at face value (\emph{Utility-Based} decision making). If so, what determines that appeal? Is it the utility a model promises under the assumption that it is correct---which is the assumption that drives the literature on Narrative Competition \citep[][]{EliazSpiegler.Narratives, EliazGalpertiSpiegler, Levyetal.}? Or do they instead favor caution, as suggested by the literature on choice under uncertainty \citep[see, e.g.,][]{ambiguityReview, Gilboa.Review}?


We study these questions in a laboratory experiment. This method provides full control over the data-generating process and it allows us to abstract from confounding factors such as prior beliefs or attachment to political groups.


Our setting involves four variables: the investment amount (which the subject chooses), the payoff amount, and two additional variables we call \emph{covariates}. These variables are causally linked through a data-generating process (\emph{DGP}) that the subject does not know. Instead, the subject observes two competing models that we convey in natural language, using terms such as `$X$ directly affects $Y$,' `$X$ influences $Y$ indirectly through $Z$,' or  `$X$ is a symptom of $Z$,' along with graphical illustrations. Models differ in the causal roles they assign to the four variables, possibly erroneously. While each model treats the investment as exogenous, the payoff and the covariates might be endogenous or exogenous. They might be causes or symptoms of other variables, or mediators between them. We fit each model to data produced by the DGP. Subjects observe the fitted model's implications for choice and utility. They have access to (simulated) empirical data from the DGP which they may use to rule out misspecified models. After studying the models, their implications, and the empirical data, subjects select between two investment levels, each implied as optimal by one of the models (but possibly suboptimal according to the DGP), and have their payoff determined according to their choice and the DGP.

Subjects make a choice in each of several rounds. Each round features a different DGP and a different pair of models fit to it, along with different implications for the optimal investment levels, as well as different claims about best- and worst-case outcomes. We formulate our hypotheses as a list of decision criteria subjects might use. We systematically vary the available models and their implications across the rounds to ensure that each combination of decision criteria corresponds to a unique pattern of choices across all rounds. This revealed preference information lets us identify subjects' decision making. Formally, we estimate a finite mixture model to identify the frequency of each decision criterion.


We document two main results. First, subjects display a remarkable ability to discard misspecified models by connecting models and data. Three fifths make purely Fact-Based choices. They not only intuit the relevant correlational implications but also find which of 18 possible charts displaying empirical data will help them check these implications. These subjects show no tendency to detect some types of misspecification more often than others. An additional fifth detect only misspecifications revealed by unconditional, but not by conditional, correlational data. Half of the latter succeed only when the model-inconsistent correlation is implied by a proposed direct causal effect, but not when it results from an indirect effect or a common cause affecting two variables. The final fifth of subjects do not make any consistent use of empirical information. A treatment that raises the stakes by a factor of three (up to \$90 in some cases) does not affect these results. Hence, failures to detect certain inconsistencies reflect limited ability rather than limited motivation.


Our second main result is that subjects unable or unwilling to rule out misspecified models choose cautiously. In our main treatment, roughly one fifth of subjects ever deploy Utility-Based criteria. Caution describes the vast majority of these choices, whereas best-case maximizing choices---the assumption of the Narrative Competition literature---are extremely rare. An additional treatment restricts access to model structures and empirical data to reveal the Utility-Based criteria of the remaining four fifths of subjects. In this treatment, two thirds choose cautiously, while the remaining third maximizes best-case outcomes. This result arises when we communicate a model’s implications as its prediction about the maximally achievable utility, along with its prediction about the utility expected if the competitor’s recommendation is followed. While this presentation is natural for models fit to data, it differs from the presentation of payoff information in typical risky-choice experiments (which is unnatural in our setting). In treatments that present payoff implications in the latter way, the fraction of best-case maximizing subjects drops to a tenth. A treatment involving dominated choices shows that the natural way of presenting implications of models fit to data induces errors, likely due to a failure of contingent thinking \citep{NiederleVespa.AnnualReviews}, and hence does not reflect actual preferences.


Our subjects’ ability to rule out incorrect models may seem surprising given the literature on belief updating \citep{Benjamin.Review}, especially since we provide no assistance. Our data suggests that their performance is due to the possibility to rely on qualitative rather than quantitative inference.  In principle, subjects could assess a model’s correctness through quantitative reasoning, as each round provides data on the relation between investment and outcome according to the DGP, from which they could infer the optimal investment. Yet few subjects view that data, and virtually none view it exclusively. Instead, they intuit the models’ correlational implications and check the empirical correlations on which the implications differ. They rarely view empirical correlations on which the models’ qualitative implications coincide. Rounds in which both models are misspecified provide further evidence for our hypothesis of qualitative rather than quantitative inference: subjects tend to pick the model with fewer inconsistencies between model-implied and empirical correlations rather than the one whose implied investment is closer to the empirically optimal level (which they could eyeball from the empirical data).


Out-of-sample predictions demonstrate that our estimates capture stable behavioral tendencies. They also reveal that a three-type model largely suffices to explain the bulk of predictable variation in the data. That model includes (i) a type that rules out all misspecified models, (ii) a type that excludes misspecified models only if a proposed direct causal effect appears absent in the empirical data (and otherwise randomizes), and (iii) a type that does not seek to or fails to discard misspecified models but makes choices to maximize worst-case payoffs.

Concerns that we may have enlisted unrepresentatively sophisticated subjects are unwarranted. While we observe the expected relationships between subjects’ choices, their educational background, and their Cognitive Reflection Test scores \citep{CRT, CRT3}, our subjects' statistical background is limited---only about half can complete the adage `Correlation does not \ldots'
Moreover, their CRT performance mirrors that of other university subject pools and of financial professionals. Other subject characteristics correlate only weakly or not at all with choices in our setting. Belief in pseudoscience \citep{Torresetal.} is directionally but statistically insigificantly associated with worse decision making, and neither political position nor political centrism have significant predictive power. This result contrasts with the common view that disagreements with one's own political views stem from others' objective inferential errors, but that view itself may be mistaken \citep[\emph{na\"ive realism}, ][]{GriffinRoss}

Broadly, our results provide empirical foundations for the emergent literature on mental models whose applications span fields as diverse as behavioral economics \citep{Spiegler.QJE}, macroeconomics \citep{Molavi2019}, finance \citep{Molavietal.,Shiller.Narratives}, strategic management \citep{FelinZenger, Camuffoetal.}, institutional economics \citep{denzauNorth}, and contract theory \citep{SchumacherThysen}. It makes three specific main contributions.\footnote{
A previous version of this paper reports the results from an earlier experiment. (Subjects who had participated in that experiment could not participate in the present experiment.) That experiment also found a remarkable ability of subjects to discard misspecified models, and a preference for cautious choice when subjects could not exclude misspecified models. It also found around 15\% of subjects making choices consistent with the Best-Case Promise criterion. Its design has several shortcomings relative to the current experiment. First, it did not present information about the payoffs from following a competing model, which constitutes a possible reason for Best-Case Promise choice. Second, it did not include anything like our Utility-Focused part. Third, it artificially simplified the problem for subjects by highlighting the data charts for which the two models made differing predictions and by providing some explanation about the correlational implications of causal structures.
}

First, it advances the literature on how individuals draw inferences from data \citep[reviewed in][]{Benjamin.Review} by studying the case in which individuals lack a-priori information about the structure of the data-generating process. While a large literature in cognitive science \citep[see, e.g., ][for reviews]{OxfordHandbook, SlomanBook, BayesianModelsOfCognitionBook} also addresses that question, that literature focuses on learning from scratch.\footnote{A narrow exception, \citet{Steyversetal.} test whether subjects can distinguish the collider $X\rightarrow Y \leftarrow Z$ from the fork  $X\leftarrow Y \rightarrow Z$ using a sequence of individual observations. In contrast to our study, \citet{Steyversetal.} show subjects trial-by-trial data. Hence, in their work, subpar performance may reflect an inability to extract correlational information from such data rather than an inability to recognize inconsistencies between predicted and empirical correlations.} It finds that humans are generally much more adept in learning about structure than in updating proabilistic beliefs in the types of experiments reviewed in \citet{Benjamin.Review}. Our paper's focus---choice between competing candidate models---permits decision strategies that differ fundamentally from those suited to learning from scratch. In particular, it allows for \emph{constraint-based learning} \citep{spirtes2000causation}, which tests whether the data satisfy the correlational constraints implied by a given model.

Second, our results inform the literature that interprets misspecified models as `lens through which people view the world' \citep[e.g., ][]{SchwartzsteinSunderam,  CharlesKendall, Andreetal.MacroNarratives}. We show that when models are provided externally, individuals are unlikely to blindly accept any such lens. Instead, they are adept at dismissing misspecified models when the requisite empirical data is available and the inconsistencies between models and data are qualitative rather than quantitative.


The third main contribution of our work is to test the main behavioral assumption of the Narrative Competition literature: decision makers adopt whichever model promises the highest utility \emph{if correct}. That literature derives fascinating implications of such behavior---mutually inconsistent narratives will necessarily coexist; it is possible to predict which narratives will survive competition and which ones will not \citep{EliazSpiegler.Narratives};\footnote{
For example, consider the question of whether mask-wearing causally reduces COVID-19 transmission. Focus on two causal models. Model 1 accurately describes reality and states that increased masking reduces COVID-19 transmission. Model 2 surmises that masking has no effect on disease transmission. If most individuals adopt the first model, they will wear masks, and case counts will be low. According to Model 1, this situation can be maintained only by continued masking, which has small hassle costs. Model 2 is more attractive because it predicts that ending mask-wearing will eliminate hassle costs without affecting case counts. The literature assumes that due to its greater attractiveness, individuals will flock to Model 2. Contrarily, if Model 2 is more popular, individuals will not wear masks, and case counts will be high. According to Model 2, masking cannot change this situation. Model 1 makes the more attractive prediction that case counts can be lowered at a small hassle cost. Individuals will thus flock to Model 1. Overall, the more popular one model, the more attractive the other. Accordingly, no model, including the correct one, can survive alone; multiplicity is an equilibrium.}
and cycles of populism will emerge \citep{Levyetal.}.\footnote{The formal assumption in \citet{Levyetal.} is that the difference in the maximal utilities promised by two parties motivates turnout.} Our data show that a substantial minority of individuals will engage in such Best-Case maximization due an interaction of the natural mode of communicating model implications with a failure of contingent reasoning \citep{NiederleVespa.AnnualReviews}, but only as long as they cannot access data to discard misspecified models.


Several recent studies in economics relate to our work. First, about three fifths of the subjects in \citet{Frechetteetal.} can learn and reproduce the structure of three-variable Bayesian networks from a list of observations. While their subjects' remarkable performance parallels ours', that work does not involve externally proposed, potentially misspecified models, and hence fundamentally differs from our tests of whether people can recognize inconsistencies between models and data. Second, \citet{CharlesKendall} study the effect of providing chain or collider narratives to subjects required to interpret a line-by-line dataset of three binary variables. In a setting with stakes of a few cents,\footnote{A subject in their main sample who makes the worst choices loses 33 cents relative to optimal choice.} they find that providing subjects with a single narrative affects their choices in the direction consistent with that narrative, as do \citet{BarronFries} in a different setting. They also find that subjects presented with two conflicting narratives choose intermediate actions. Despite the ostensible similarity to our work, their research questions fundamentally differ from ours. We are interested in settings such as voting on political referenda or selecting between leaders with differing views of the interactions between economic variables. In these settings, agents must choose between models, and intermediate choices are unavailable. More importantly, our work generally characterizes the limits of subjects' inferential abilities in a way that is not constrained to  chain or collider narratives, but rather concerns a large set of models and DGPs. Additionally, we study the prevalence and nature of Utility-Based criteria. Third, \citet{Angrisanietal.} find that the evolution of beliefs during the Covid19 pandemic is consistent with the model of \citet{EliazSpiegler.Narratives}. While their structural model fits the data well, their field data naturally limits the extent to which they can rule out alternative explanations. More distantly related, \citet{aina2025weighting} ask how subjects quantitatively update beliefs in a balls-and-urns setting when each of two competing information structures could have generated a given signal. Their setting does not allow discarding information structures based on inconsistencies with the data.


The remainder of this paper proceeds as follows. Section \ref{sec:theory} outlines the choice setting and defines the choice criteria we study. Section \ref{sec:Identification} explains our identification strategies along with details concerning the experimental design. Section \ref{sec:analysis} showcases our main empirical results. Finally, Section \ref{sec:conclusion} concludes.


\section{Setting and choice criteria}\label{sec:theory}

\subsection{Choice problem}\label{sec:choiceProblem}


In each round of our experiment, subjects choose between two investment levels $I$. They know that the investment maps into an outcome $Y$, and that their payoff will be $\pi(I) = Y - I$, but they do not know the data-generating process (\emph{DGP}) that determines whether and how the investment affects the outcome. DGPs involve four variables,\footnote{This is the minimum number of variables that allows us to answer our research questions.} the investment $I$, the outcome $Y$, and two covariates $C_1$ and $C_2$, represented to subjects, respectively, as  \includegraphics[height=0.8\baselineskip]{pics/moneyDrop.png}, \includegraphics[height=0.9\baselineskip]{pics/iconBonus}, and two counters of different colors (\includegraphics[height=0.7\baselineskip]{pics/greyCounter.png}). These variables are related through a recursive system of Gaussian (`regression') equations that are linear in $C_1$, $C_2$, $Y$, and in the square root of $I$. Investment $I$ is always exogenous, all other variables may be endogenous or exogenous. Subjects simply learn that the variables interact through some `mechanism.'

\begin{figure}
\thispagestyle{empty}
\caption{Decision screen \label{fig:interface}}
\vspace{-0.4cm}
\begin{center}
\includegraphics[height = 0.9\textheight]{pics/interfaceNew2}\\
\end{center}
\footnotesize
\textbf{\textit{Notes:}} Any element in the data dashboard at the top of the screen is clickable and displays the information illustrated in Figure \ref{fig:dataCharts}.
\end{figure}


Subjects choose an investment level by deciding which of two advisors to follow, as shown in Figure \ref{fig:interface}. Each advisor has a potentially misspecified model in the form a system of regression equations that is linear in $C_1$, $C_2$, $Y$, and in the square root of $I$. The model is fitted to a large sample of data generated by the DGP (formally: the population moments implied by the DGP). We communicate models to subjects in natural language using terms such as `$X$ directly affects $Y$,' `$X$ influences $Y$ indirectly through $Z$,' or  `$X$ is a symptom of $Y$,' both in video and written format, and illustrated graphically. Each fitted model implies a \emph{recommendation} about the payoff-maximizing investment amount and predicts the implied expected payout. It also predicts the payout the subject can expect from following the recommendation of the competing advisor. We do not explain the details of model-fitting, but we inform subjects that if an advisor's model is correct, the recommended investment is truly best for the subject, and that advisor's predictions about the payment amounts from following the own and the competing advisor's recommendation are accurate. We also tell subjects that all these numbers are meaningless if an advisor's model is incorrect.

When the subject selects an advisor, she automatically invests the amount recommended by the chosen advisor. Her payoff is determined by that amount and the DGP (with noise realizations set to zero). If the model that led to that recommendation is misspecified, the subject will thus obtain a less-than-maximal payoff.


\begin{figure}
\caption{Example data charts shown to subjects\label{fig:dataCharts}}
\begin{center}
\begin{tabular}{cc}
\textbf{A. Unconditional correlation} & \textbf{B. Conditional correlation} \\ \\
\includegraphics[width=0.47\textwidth]{pics/dataChartUncond}&
\includegraphics[width=0.47\textwidth]{pics/dataChartCond}\\
\end{tabular}
\end{center}
\footnotesize
\textbf{\textit{Notes:}} Subjects can retrieve each chart by clicking on the corresponding link in their `data dashboard.' Panel A shows an example of a chart displaying an unconditional correlation. Panel B shows an example of a chart displaying a conditional correlation.
\end{figure}


Crucially, subjects can access data generated by the DGP to detect misspecified models. As Figure \ref{fig:dataCharts} illustrates, subjects can view the unconditional correlation between any pair of variables in the form of a bar chart (Panel A), and can condition each  correlation on any third variable (Panel B).\footnote{We do not show correlations that simultaneously condition on two variables, for two reasons. First, such correlations would be needed only to test whether a diamond-shaped DAG ($I \rightarrow J \rightarrow K, I \rightarrow L \rightarrow K$, with $J$ and $L$ unconnected) fits the data, which we do not include in the experiment as it is not necessary to identify our decision criteria. Second, such data is rarely available public debates.} We do not provide any hints regarding the connection between models and empirical correlations. It is up to the subjects to decide which of the 18 charts to examine.

It is also up to them to intuit any model's correlational implications to connect it to the data. While we illustrate models in the format of \emph{directed acyclic graphs} (\emph{DAG}) that represent the underlying regression specifications, we do not provide any explanation of that depiction.\footnote{For instance, if $C_1$ denotes the yellow counter, the model of the advisor on the left in Figure \ref{fig:interface}, corresponds to the three-equation system
\bmathnn

Y &=& \beta_Y + \beta_{C_1 Y} C_1 + \epsilon_Y \\
C_1 &=& \beta_{C_1} + \beta_{I C_1} \sqrt{I} + \epsilon_{C_1} \\
C_2 &=& \beta_{C_2} + \beta_{I C_2} \sqrt{I} + \beta_{C_1 C_2} C_1 + \epsilon_{C_2}

\emathnn
where variables $\beta$ denote real-valued parameters to be estimated, and variables $\epsilon$ denote indepenent mean zero Gaussian errors whose variances may differ from each other. All our graphical illustrations of models are DAG-representations of the underlying system of regression equations. There is one equation for each variable that has one or more links pointing to it, and all variables whose links directly point into that variable (but not those that are merely indirectly connected) appear on the right-hand side of the regression equation. The interpretation is causal; a variable that appears as on the left hand side of an equation is endogenous, other variables are exogenous.\label{fn:systemsDAGs}
}

\subsection{Choice criteria}\label{sec:criteriaAndTypes}

We now outline the choice criteria that describe decision approaches in our setting and whose prevalence we will estimate. We first describe how correlational information can be used to rule out misspecified models and define various levels of partial inferential ability. These constitute our Fact-Based criteria. We then present the Utility-Based criteria implied by the literatures on Narrative Competition and choice under uncertainty. Because subjects may combine Fact-Based and Utility-Based criteria by first excluding misspecified models when able to and applying Utility-Based criteria otherwise, our estimation will consider all possible pairs of criteria from the two classes.


\paragraph{Fact-Based criteria}


A handful of archetypical causal models provide the key insights required for excluding misspecified models in our setting. While defined on two or three nodes, the corresponding insights also apply to the four-node DAGs we use in our experiment. Appendix \ref{app: DAG implications} provides the corresponding  formal statement.


\begin{observation}\label{obs: DAG implications}
Consider a system of linear Gaussian equations that is consistent with a DAG $G$ defined over nodes $N =\{I,J,K\}$.
\begin{enumerate}
	\item If $G: I\rightarrow J$, then generically $\cov(I,J)\neq 0$
	\item
	\begin{enumerate}
		\item If $G: I \rightarrow K \rightarrow J$ or $G: I \leftarrow K \rightarrow J$, then generically $\cov(I,J)\neq 0$
		\item If $G: I\rightarrow K\leftarrow J$, then $\cov(I,J)=0$.
	\end{enumerate}
	\item
	\begin{enumerate}
		\item If $G: I\rightarrow K \rightarrow J$ or $G: I \leftarrow K \rightarrow J$, then $\cov(I,J|K)=0$
		\item If $G: I\rightarrow K \leftarrow J$, then generically $\cov(I,J|K) \neq 0$.
	\end{enumerate}
\end{enumerate}
\end{observation}


While causation does not necessarily imply correlation, case (i) conveys that it does so outside a knife-edge set of parameters. This fact also holds when causation is indirect (as in the \emph{chain} $I\rightarrow K \rightarrow J$) or when a common cause affects two variables (as in the \emph{fork} $I \leftarrow K \rightarrow J$), as stated in case (ii)(a). The independence implication in case (ii)(b) holds by definition; non-independence of $I$ and $J$ would require some causal path between these nodes.

To understand the conditional correlational implications in case (iii), first consider the \emph{chain}, $I\rightarrow K \rightarrow J$. When $I$ indirectly causes $J$ but the mediator $K$ is held fixed, changes in $I$ cannot translate into changes in $J$, and hence $\cov(I,J|K) = 0$. In the case of the  \emph{fork}, $I \leftarrow K \rightarrow J$, the common cause $K$ is the sole reason why $I$ and $J$ are (unconditionally) correlated. Hence, holding $K$ fixed eliminates that correlation, and thus $\cov(I,J|K) = 0$. The implication of the \emph{$v$-collider} $I\rightarrow K \leftarrow J$ becomes apparent in the example in which $K$ is defined as $K=I+J$. Then, once we fix $K$, larger $I$ must coincide with smaller $J$.


To see how a subject can draw inferences about model misspecification, consider the example of Figure \ref{fig:interface}. The right-hand side advisor's model contains the chain \mbox{\raisebox{-0.5ex}{\includegraphics[height = 0.7\baselineskip]{pics/moneyDrop.png}} $\rightarrow$ \raisebox{-0.7ex}{\includegraphics[height = 0.7\baselineskip]{pics/counterPurple}} $\rightarrow$ \raisebox{-0.5ex}{\includegraphics[height = 0.9\baselineskip]{pics/iconBonus}}}. If the data show a correlation between \mbox{\raisebox{-0.5ex}{\includegraphics[height = 0.7\baselineskip]{pics/moneyDrop.png}} and \raisebox{-0.5ex}{\includegraphics[height = 0.9\baselineskip]{pics/iconBonus}}} even if  \raisebox{-0.7ex}{\includegraphics[height = 0.7\baselineskip]{pics/counterPurple}} is held fixed, the subject can infer that that model must be mistaken, following case (iii)(a) above.


Some correlational implications may be easier to intuit and understand than others. We use this variation to characterize the limits to subjects' inferential abilities:


\begin{definition} \textbf{Fact-Based criteria}\
\label{def:dataBased}
\begin{enumerate}
	\item Direct Links: Subjects discard a model if it posits a direct link between two nodes $I$ and $J$ but $I$ and $J$ are not correlated in the data. (Case (i) in Observation \ref{obs: DAG implications})
	\item Unconditional Correlations: Subjects discard a model if any of its implied unconditional correlations (or absence thereof) are inconsistent with the data. (Cases (i) and (ii) in Observation \ref{obs: DAG implications})
	\item Conditional Correlations:  Subjects discard a model if any of its implied conditional correlations (or absence thereof) are inconsistent with the data. (Case (iii) in Observation \ref{obs: DAG implications})
	\item All correlations: Subjects discard any model that is based on a misspecified model. (All cases in Observation \ref{obs: DAG implications})
\end{enumerate}
\end{definition}


\paragraph{Utility-Based criteria}\label{sec:utilityCriteria}

Utility-based criteria come to bear if individuals are unable or unwilling make a Fact-Based choice. These criteria depend solely on the collection of the models' implications about payoffs and recommended investments. To define them formally, let $M_1$ and $M_2$ denote the models in the subjects' choice set. Let $M_{\emptyset}$ denote a third, unavailable, model in which the investment does not affect the outcome. Define a \emph{state} of the world $s$ that encodes whether the DGP is consistent with $M_1$, $M_2$, or $M_{\emptyset}$, denoted as $s_1, s_2$, and $s_\emptyset$, respectively. An \emph{act} $M$ maps states into payoffs $u_{M, s}$. The choice of model is an act.

The Narrative Equilibrium literature assumes that individuals choose the model that solves $\max_{M \in \{M_1, M_2\}} \allowbreak \max_{s \in \{s_1, s_2\}} u_{M, s}$---it promises the highest utility if true. Such max-max decision-making is diametrically opposed to tendency for cautious choice documented in the empirical literature of choice under ambiguity \citep{ambiguityReview}. That literature suggests that a decision maker will instead select the model that promises the highest payoff if one of the non-selected models is correct. If the decision maker only considers the models available for choice, $M_1$ and $M_2$, she will solve $\max_{M \in \{M_1, M_2\}} \min_{s \in \{s_1, s_2\} } u_{M, s}$. Hence, she will first identify which model predicts the higher expected payoff from choosing its competitor, and then choose that competitor.\footnote{When the two models are linear, this criterion is equivalent to selecting the model that promises the \emph{lower} payoff if correct, as we formally show in Appendix \ref{app:theoryMain}.} If the decision maker also accounts for the possibility that neither model is correct, and that the DGP may not feature an effect of the investment on the outcome at all, she solves $\max_{M \in \{M_1, M_2\}} \min_{s \in \{s_1, s_2, s_{\emptyset}\} } u_{M, s}$. In this case, the possibility that the investment will be wasted causes her to select the model that recommends the lowest investment. For completeness, we also consider the possibility that decision makers will select the model that recommends the highest investment, for instance due to the illusion of control \citep[see, e.g.,][]{StefanDavid, Klusowskietal.}.\footnote{Choosing the model with the highest recommended investment does not correspond to a max-max criterion. The reason is that a high investment might lead to low payoffs if it substantially exceeds the optimal investment.}
Overall, we thus consider four Utility-Based criteria:
\begin{definition}\
\label{defMotivational}
\label{def:adviceBased}
\begin{enumerate}
\item The {\normalfont Best-Case Promise} criterion selects the model that promises the highest payout if correct.
\item The {\normalfont Worst-Case Promise} criterion selects the model that promises the highest payout if the competing model is correct.
\item The {\normalfont Minimize Investment} criterion selects the model that implies the lowest investment.
\item The {\normalfont Maximize Investment} criterion selects the model that implies the highest investment.
\end{enumerate}
\end{definition}


\paragraph{Types} We define a \emph{type} as a pair consisting of a Fact-Based criterion and a Utility-Based criterion. A decision maker may apply a criterion from only one class. Either order of application is possible, but since each Utility-Based criterion uniquely determines choice, any subject who starts with a Utility-Based criterion necessarily ignores all Fact-Based criteria. We assume that individuals randomize uniformly across any options that remain after applying their decision criteria.

\begin{definition}
A {\normalfont Type} is a pair of a (possibly empty) fact-based criterion and a (possibly empty) utility-based criterion. Indeterminacies are resolved through uniform randomization.
\end{definition}


\section{Identification and design specifics}\label{sec:Identification}

Our experiment has two parts. The \emph{Comprehensive Part} (Subsection \ref{sec:designMainExperiment}) is our main focus. It identifies the frequency of types when subjects have access to correlational data. The \emph{Utility-Focused Part} (Subsection \ref{sec:designPart2}) focuses on Utility-Based criteria by withholding the information required to apply Fact-Based criteria. It  includes treatments to assess whether subjects who follow the Best-Case Promise criterion do so intentionally or by mistake. Subsection \ref{sec:designDetails} provides details of the experimental implementation.


\subsection{Comprehensive Part} \label{sec:designMainExperiment}


The design of the Comprehensive Part follows the standard revealed-preference logic: each subject makes a choice from a sequence of menus constructed so that the overall choice distribution reveals the distribution of criteria used. Formally, we will estimate type frequencies using a finite-mixture approach closely related to the Strategy Frequency Estimation Method of \cite{DalBo.Evolution, DalBoetal.StrategyChoice}.\footnote{Unlike \cite{DalBo.Evolution, DalBoetal.StrategyChoice}, some types in our setting predict indifference on certain choice sets, and we estimate our model using GMM rather than MLE.} Out-of-sample prediction analysis addresses concerns about overfitting.


While the approach is conceptually simple, constructing a sequence of menus that identifies the frequency of all types is challenging. Identification requires variation in observed and implied correlations as well as in recommendations and predictions. To identify the share of subjects whose choices follow the Best-Case Promise criterion, for example, we would ideally vary only which model makes the higher promise while keeping everything else constant. The fraction of subjects whose choices change in response would then indicate the prevalence of that criterion. However, we cannot vary such properties independently, since we can choose only model structures and the DGP, not the resulting identifying properties. For instance, varying the set of correlations that can inform choice requires changing a model’s structure, which in turn necessarily alters its recommendations and predictions.


Our construction of a sequence of menus for identification relies on two insights. First, assessing whether a DAG fits the data requires checking only whether the predicted conditional (in)dependence relations hold, not the magnitudes of these relations. Thus, inferences based on any Fact-Based criterion are unaffected by the DGP’s parameters as long as its structure remains unchanged. Once a DGP and a misspecified model are fixed, we can therefore freely choose the DGP’s parameters to identify Utility-Based criteria. Second, as we show in Appendix \ref{app:theoryMain}, for any pair of models we can vary independently whether the model recommending the higher investment also makes the higher or lower promise by adjusting the distribution of investments in the simulated empirical data used for model fitting. This strategy allows us to distinguish, for example, the Worst-Case Promise criterion from the Minimize Investment criterion.


\begingroup
\newgeometry{textwidth=\dimexpr\textwidth+2in\relax}
\small


\setlength{\tabcolsep}{3pt}
\begin{longtable}{c c c p{0.20\textwidth} c c c c c}
\caption{Menus \label{tab:menusMain}}\\
(1)&(2)&(3)&\multicolumn{1}{c}{(4)}&\multicolumn{1}{c}{(5)}&\multicolumn{1}{c}{(6)}&\multicolumn{1}{c}{(7)}&\multicolumn{1}{c}{(8)}&\multicolumn{1}{c}{(9)} \\ \toprule
Menu& DGP & Competitor & \multicolumn{1}{c}{Description} & \multicolumn{3}{c}{Criterion identifies} & \multicolumn{2}{c}{Model chosen by} \\
& & & & \multicolumn{3}{c}{misspecified model} & & \\
\cmidrule(r){5-7}\cmidrule(r){8-9}
& & & & Direct & Uncond. & Cond. & Best-Case & Min. \\
& & & & Links & corr. & corr. & Promise & Invest.\\
\midrule
\endfirsthead
\caption[]{Menus (continued)}\\
(1)&(2)&(3)&\multicolumn{1}{c}{(4)}&\multicolumn{1}{c}{(5)}&\multicolumn{1}{c}{(6)}&\multicolumn{1}{c}{(7)}&\multicolumn{1}{c}{(8)}&\multicolumn{1}{c}{(9)} \\ \toprule
Menu& DGP & Competitor & \multicolumn{1}{c}{Description} & \multicolumn{3}{c}{Criterion identifies} & \multicolumn{2}{c}{Model chosen by} \\
& & & & \multicolumn{3}{c}{misspecified model} & & \\
\cmidrule(r){5-7}\cmidrule(r){8-9}
& & & & Direct & Uncond. & Cond. & Best-Case & Min. \\
& & & & Links & corr. & corr. & Promise & Invest.\\
\midrule
\endhead
\rule[0.00000\textheight]{0pt}{0.02000\textheight}$M_{1}$ & \multirow{2}{*}{\begin{tikzpicture}[>=Stealth, scale=1]
\scriptsize
\node (A) at (0,2) {$I$};
\node (X) at (0,1) {$C_1$};
\node (Y) at (0,0) {$Y$};
\node (Z) at (0.5,1.5) {$C_2$};
\draw[->] (A) to[out=180, in=180, looseness=1.3] (X);
\draw[->] (A) to[out=0, in=90, looseness=1] (Z);
\draw[->] (Z) to[out=-90, in=0, looseness=1] (X);
\draw[->] (X) -- (Y);
\end{tikzpicture}} & \multirow{2}{*}{\begin{tikzpicture}[>=Stealth, scale=1]
\scriptsize
\node (A) at (0,2) {$I$};
\node (X) at (0,1) {$C_2$};
\node (Y) at (0,0) {$Y$};
\node (Z) at (0.5,1.5) {$C_1$};
\draw[->] (A) to[out=180, in=180, looseness=1.3] (X);
\draw[->] (A) to[out=0, in=90, looseness=1] (Z);
\draw[->] (Z) to[out=-90, in=0, looseness=1] (X);
\draw[->] (X) -- (Y);
\end{tikzpicture}} & \multirow[t]{2}{*}{\scriptsize\parbox[t]{0.20\textwidth}{Your Bonus only depends on one of the Counters. Your Action influences that Counter both directly and through the other Counter.}} & No & No & Yes & DGP & DGP \\
\rule[-0.03000\textheight]{0pt}{0.10000\textheight}$M_{2}$ & & & & No & No & Yes & Comp. & DGP \\
\rule[-0.11000\textheight]{0pt}{0.13000\textheight}$M_{3}$ & \multirow{1}{*}{\begin{tikzpicture}[>=Stealth, node distance=0.5cm]
\scriptsize
\node (A) {$I$};
\node[below=of A] (X) {$C_1$};
\node[below=of X] (Z) {$C_2$};
\node[below=of Z] (Y) {$Y$};
\draw[->] (A) -- (X);
\draw[->] (X) -- (Z);
\draw[->] (Z) -- (Y);
\end{tikzpicture}} & \multirow{1}{*}{\begin{tikzpicture}[>=Stealth, node distance=0.5cm]
\scriptsize
\node (A) {$I$};
\node[below=of A] (X) {$C_2$};
\node[below=of X] (Z) {$C_1$};
\node[below=of Z] (Y) {$Y$};
\draw[->] (A) -- (X);
\draw[->] (X) -- (Z);
\draw[->] (Z) -- (Y);
\end{tikzpicture}} & \multirow[t]{1}{*}{\scriptsize\parbox[t]{0.20\textwidth}{Your Action indirectly influences your Bonus. It influences the first Counter which, in turn, influences the second Counter, which then influences your Bonus.}} & No & No & Yes & DGP & Comp. \\
\rule[0.00000\textheight]{0pt}{0.04000\textheight}$M_{4}$ & \multirow{2}{*}{\begin{tikzpicture}[>=Stealth, scale=1]
\scriptsize
\node (A) at (-0.5,2) {$I$};
\node (Y) at (0,1) {$Y$};
\node (Z) at (0,0) {$C_2$};
\node (X) at (0.5,2) {$C_1$};
\draw[->] (A) to[out=-90, in=180, looseness=1] (Y);
\draw[->] (X) to[out=-90, in=0, looseness=1] (Y);
\draw[->] (X) to[out=-90, in=0, looseness=0.6] (Z);
\draw[->] (Y) -- (Z);
\end{tikzpicture}} & \multirow{2}{*}{\begin{tikzpicture}[>=Stealth, scale=1]
\scriptsize
\node (A) at (-0.5,2) {$I$};
\node (Y) at (0,1) {$Y$};
\node (Z) at (0,0) {$C_1$};
\node (X) at (0.5,2) {$C_2$};
\draw[->] (A) to[out=-90, in=180, looseness=1] (Y);
\draw[->] (X) to[out=-90, in=0, looseness=1] (Y);
\draw[->] (X) to[out=-90, in=0, looseness=0.6] (Z);
\draw[->] (Y) -- (Z);
\end{tikzpicture}} & \multirow[t]{2}{*}{\scriptsize\parbox[t]{0.20\textwidth}{Your Action influences your Bonus directly, as does one of the Counters. That counter is not influenced by anything. The other Counter is a symptom of both the first Counter and your bonus.}} & No & Yes & No & DGP & Comp. \\
\rule[-0.05000\textheight]{0pt}{0.12000\textheight}$M_{5}$ & & & & No & Yes & No & Comp. & Comp. \\
\rule[0.00000\textheight]{0pt}{0.03000\textheight}$M_{6}$ & \multirow{2}{*}{\begin{tikzpicture}[>=Stealth, scale=1]
\scriptsize
\node (A) at (-0.5,2) {$I$};
\node (Y) at (0,1) {$Y$};
\node (Z) at (0,0) {$C_2$};
\node (X) at (0.5,2) {$C_1$};
\draw[->] (X) to[out=-90, in=0, looseness=1] (Y);
\draw[->] (A) to[out=-90, in=180, looseness=1] (Y);
\draw[->] (A) to[out=-90, in=180, looseness=0.6] (Z);
\draw[->] (Y) -- (Z);
\end{tikzpicture}} & \multirow{2}{*}{\begin{tikzpicture}[>=Stealth, scale=1]
\scriptsize
\node (A) at (-0.5,2) {$I$};
\node (Y) at (0,1) {$Y$};
\node (Z) at (0,0) {$C_1$};
\node (X) at (0.5,2) {$C_2$};
\draw[->] (X) to[out=-90, in=0, looseness=1] (Y);
\draw[->] (A) to[out=-90, in=180, looseness=1] (Y);
\draw[->] (A) to[out=-90, in=180, looseness=0.6] (Z);
\draw[->] (Y) -- (Z);
\end{tikzpicture}} & \multirow[t]{2}{*}{\scriptsize\parbox[t]{0.20\textwidth}{Your Action influences your Bonus directly, as does one of the Counters. That counter is not influenced by anything. The other Counter is a symptom of both your Action and your Bonus.}} & Yes & Yes & No & DGP & Comp. \\
\rule[-0.05000\textheight]{0pt}{0.12000\textheight}$M_{7}$ & & & & Yes & Yes & No & Comp. & Comp. \\
\rule[0.00000\textheight]{0pt}{0.04000\textheight}$M_{8}$ & \multirow{2}{*}{\begin{tikzpicture}[>=Stealth, scale=1]
\scriptsize
\node (A) at (0,2) {$I$};
\node (X) at (0,1) {$C_1$};
\node (Y) at (0,0) {$Y$};
\node (Z) at (-0.5,0.5) {$C_2$};
\draw[->] (A) -- (X);
\draw[->] (Z) to[out=-90, in=180, looseness=1] (Y);
\draw[->] (X) to[out=0, in=0, looseness=1.3] (Y);
\draw[->] (X) to[out=180, in=90, looseness=1] (Z);
\end{tikzpicture}} & \multirow{2}{*}{\begin{tikzpicture}[>=Stealth, scale=1]
\scriptsize
\node (A) at (0,2) {$I$};
\node (X) at (0,1) {$C_2$};
\node (Y) at (0,0) {$Y$};
\node (Z) at (-0.5,0.5) {$C_1$};
\draw[->] (A) -- (X);
\draw[->] (Z) to[out=-90, in=180, looseness=1] (Y);
\draw[->] (X) to[out=0, in=0, looseness=1.3] (Y);
\draw[->] (X) to[out=180, in=90, looseness=1] (Z);
\end{tikzpicture}} & \multirow[t]{2}{*}{\scriptsize\parbox[t]{0.20\textwidth}{Your Action directly influences one of the Counters. That Counter influences your Bonus both directly and through the other counter.}} & No & No & Yes & Comp. & DGP \\
\rule[-0.04000\textheight]{0pt}{0.11000\textheight}$M_{9}$ & & & & No & No & Yes & DGP & DGP \\
\rule[-0.08000\textheight]{0pt}{0.07000\textheight}$M_{10}$ & \multirow{1}{*}{\begin{tikzpicture}[>=Stealth, scale=1]
\scriptsize
\node (A) at (0,2) {$I$};
\node (X) at (0,1) {$C_1$};
\node (Y) at (0,0) {$Y$};
\node (Z) at (1,0) {$C_2$};
\draw[->] (A) -- (X);
\draw[->] (Y) -- (Z);
\draw[->] (X) -- (Y);
\end{tikzpicture}} & \multirow{1}{*}{\begin{tikzpicture}[>=Stealth, scale=1]
\scriptsize
\node (A) at (0,2) {$I$};
\node (X) at (0,1) {$C_2$};
\node (Y) at (0,0) {$Y$};
\node (Z) at (1,0) {$C_1$};
\draw[->] (A) -- (X);
\draw[->] (Y) -- (Z);
\draw[->] (X) -- (Y);
\end{tikzpicture}} & \multirow[t]{1}{*}{\scriptsize\parbox[t]{0.20\textwidth}{Your Action indirectly influences your Bonus through one of the Counters. The second Counter is a symptom of the Bonus.}} & No & No & Yes & DGP & Comp. \\
\rule[0.00000\textheight]{0pt}{0.05000\textheight}$M_{11}$ & \multirow{2}{*}{\begin{tikzpicture}[>=Stealth, scale=1]
\scriptsize
\node (A) at (-0.5,2) {$I$};
\node (Y) at (0,1) {$Y$};
\node (Z) at (0,0) {$C_2$};
\node (X) at (0.5,2) {$C_1$};
\draw[->] (A) to[out=-90, in=180, looseness=1] (Y);
\draw[->] (X) to[out=-90, in=0, looseness=1] (Y);
\draw[->] (Y) -- (Z);
\end{tikzpicture}} & \multirow{2}{*}{\begin{tikzpicture}[>=Stealth, scale=1]
\scriptsize
\node (A) at (-0.5,2) {$I$};
\node (Y) at (0,1) {$Y$};
\node (Z) at (0,0) {$C_1$};
\node (X) at (0.5,2) {$C_2$};
\draw[->] (A) to[out=-90, in=180, looseness=1] (Y);
\draw[->] (X) to[out=-90, in=0, looseness=1] (Y);
\draw[->] (Y) -- (Z);
\end{tikzpicture}} & \multirow[t]{2}{*}{\scriptsize\parbox[t]{0.20\textwidth}{Your Action influences your Bonus directly, as does one of the Counters. That counter is not influenced by anything. The other Counter is a symptom of your Bonus.}} & No & Yes & Yes & DGP & Comp. \\
\rule[-0.02000\textheight]{0pt}{0.09000\textheight}$M_{12}$ & & & & No & Yes & Yes & Comp. & Comp. \\
\rule[0.00000\textheight]{0pt}{0.05000\textheight}$M_{13}$ & \multirow{2}{*}{\begin{tikzpicture}[>=Stealth, scale=1]
\scriptsize
\node (A) at (-0.5,2) {$I$};
\node (X) at (0,1) {$C_1$};
\node (Y) at (0,0) {$Y$};
\node (Z) at (0.5,2) {$C_2$};
\draw[->] (Z) to[out=-90, in=0, looseness=1] (X);
\draw[->] (A) to[out=-90, in=180, looseness=1] (X);
\draw[->] (Z) to[out=-90, in=0, looseness=0.6] (Y);
\draw[->] (X) -- (Y);
\end{tikzpicture}} & \multirow{2}{*}{\begin{tikzpicture}[>=Stealth, scale=1]
\scriptsize
\node (A) at (-0.5,2) {$I$};
\node (X) at (0,1) {$C_2$};
\node (Y) at (0,0) {$Y$};
\node (Z) at (0.5,2) {$C_1$};
\draw[->] (Z) to[out=-90, in=0, looseness=1] (X);
\draw[->] (A) to[out=-90, in=180, looseness=1] (X);
\draw[->] (Z) to[out=-90, in=0, looseness=0.6] (Y);
\draw[->] (X) -- (Y);
\end{tikzpicture}} & \multirow[t]{2}{*}{\scriptsize\parbox[t]{0.20\textwidth}{Your Bonus directly depends on both Counters. Your Action influences one of these Counters, as does the second Counter. The second counter is not influenced by anything.}} & Yes & Yes & No & DGP & Comp. \\
\rule[-0.02000\textheight]{0pt}{0.09000\textheight}$M_{14}$ & & & & Yes & Yes & No & Comp. & Comp. \\
\\
\bottomrule
\end{longtable}
\vspace{-0.3cm}
\begin{center}
\begin{minipage}{0.83\linewidth}
\setstretch{1}
\footnotesize \textit{\textbf{Notes}}: $I$ denotes the investment, referred to as Action in the videos. Column 4 shows the text spoken in the video. In the screens that correspond to Figure \ref{fig:interface}, the counters are referred to by color. In the case of $M_1$, for instance, the text for one of the models is ``Your Bonus only depends on the red counter. Your Action influences that Counter both directly and through the blue counter.'  \normalsize
\setstretch{}
\end{minipage}
\end{center}


\restoregeometry
\endgroup

Three requirements lead us to impose further restrictions on the sequence of menus we construct.
First, Fact-Based criteria should be applicable to distinguish between the models in each menu. Hence, the two models in each menu must stem from two different Markov-equivalence classes.\footnote{Two DAGs are Markov-equivalent if they are indistinguishable based on their conditional independence relationships; see \citet{VermaPearl1990} for a characterization.}
Second, subjects should be unable to choose based on preferences over model structures, such as favoring simpler or more complex models.\footnote{An earlier version of this paper included a laboratory experiment  that also measured preferences concerning the complexity or simplicity of causal structures. In that sample, 32.2\% of subjects chose based on such criteria in some rounds. Overall, subjects favored more complex structures (22.9\%) over simpler ones (9.3\%), possibly reflecting the heuristic that models with more links seem better able to fit any data pattern. Extending the experiment to identify such \emph{Structure-Based} criteria greatly increases the complexity of the design.}  To achieve this, the two models  in any given menu differ only in that the position of the two covariates are interchanged.
Third, no two models may recommend the same investment, for otherwise subjects would have no instrumental reason to distinguish between them. Appendix \ref{app:theoryMain} shows that the  set of all possible DAGs in our setting can be partitioned into 15 action-equivalence classes, with any two DAGs within the same class implying the same optimal investment and promises for any DGP. To meet the requirement, each menu contains models from two different action-equivalence classes.
We also impose the technical constraints that none of our models contain isolated subsets of variables and that all have generic (non-knife-edge) parameters.\footnote{A subset of variables is \emph{isolated} if it neither influences nor is influenced by variables outside that subset. In our setting, a property of the parameter vector is \emph{generic} if it is violated only on a subset of the parameter space with Lebesgue measure zero.}


After heuristically constructing a sequence of fourteen menus, we formally verify that it identifies the full vector of type probabilities, as described in Subsection \ref{structModel}. To illustrate the strength of identification achieved by this sequence, Appendix \ref{appendix:typeDistanceHeatmap} reports the distance between the choice patterns of all pairs of types.


Table \ref{tab:menusMain} presents the resulting sequence $\mathcal{M} = (M_1, \ldots, M_{14})$ used to identify the distribution of types ($M$ stands for \emph{m}ain). Each menu in $\mathcal{M}$ includes one correct and one misspecified model. The table reports, for each menu, whether a given fact-based criterion can exclude the misspecified model and shows the choices implied by the Best-Case Promise and Minimize Investment criteria. The Worst-Case Promise and Maximize Investment criteria select the opposite model in each case. Given the restrictions imposed on the menus, we can distinguish 17 types.\footnote{There are 4 Utility-Based and 4 Fact-Based criteria. Including randomization within each class increases these numbers to 5 and 5, respectively. A type who always identifies the correct model never reveals a Utility-Based criterion, implying a maximum of 21 types. Our model cannot separately identify a fully random type from a noisy population that follows nontrivial criteria, so we assume no subject randomizes throughout, leaving 20 types. Moreover, for subjects using the Maximize Investment criterion, we cannot distinguish whether they apply it alone or in combination with an imperfect Fact-Based criterion (Direct Links, Unconditional Correlations, Conditional Correlations). We attribute such choices to the type using the Maximize Investment criterion without any Fact-Based component, yielding 17 types in total.}


In addition to the 14 main menus, we include 4 menus $\mathcal{W} = (W_1, \ldots, W_4)$ (\emph{w} for “\emph{w}rong”) in which both models are inconsistent with the DGP. Randomly interspersed with the main menus, these ensure that ruling out one misspecified model does not imply that the remaining model is correct. They will also provide insight into the nature of subjects’ decision making; see Section \ref{sec:bothWrong}. Two additional practice menus, $\mathcal{P} = (P_1, P_2)$, familiarize subjects with the decision interface and are always presented first. As preregistered, we exclude choice data from the practice menus from analysis.


\subsection{Utility-Focused Part}\label{sec:designPart2}

\begin{figure}[ht]
\thispagestyle{empty}
\caption{Decision screen in the Utility-Focused part \label{fig:interfaceAmbiguity}}
\begin{center}
\begin{tabular}{cc}
\multicolumn{2}{c}{Model frame} \\
\midrule
\textbf{A. State constant} & \textbf{B. Action constant} \\
\includegraphics[width=0.47\textwidth]{pics/interfaceAmbiguityAdviceStateConst} &
\includegraphics[width=0.47\textwidth]{pics/interfaceAmbiguityAdviceActionConst} \\
\multicolumn{2}{c}{Gamble frame} \\
\midrule
\textbf{C. State constant} & \textbf{D. Action constant} \\
\includegraphics[width=0.47\textwidth]{pics/interfaceAmbiguityBallsStateConst} &
\includegraphics[width=0.47\textwidth]{pics/interfaceAmbiguityBallsActionConst} \\
\end{tabular}
\end{center}
\footnotesize
\end{figure}


The Utility-Focused Part withholds the model descriptions and empirical data necessary for applying Fact-Based criteria. To avoid the need to account for the support of subjects' beliefs, which distinguishes the Worst-Case Promise and Minimize Investment criteria, we inform subjects that one model in each round is correct, and we withhold information about the recommended investment amounts. As in the Comprehensive Part, variation in the menus across multiple rounds reveals choice criteria.


If the Best-Case Promise criterion finds empirical support, the question arises why our results differ from research on decision making under uncertainty, which typically finds a preference for cautious choice. We consider two hypotheses. First, Best-Case Promise choices may occur when uncertainty arises from incomplete knowledge about a knowable process (epistemic uncertainty), even if they do not occur in settings characterized by intrinsic randomness (aleatory uncertainty), where cautious choice is most often observed.\footnote{Prior work, if anything, finds that people are more \emph{reluctant} to bet under epistemic than under aleatory uncertainty \citep{FoxUlkumen}.} To test this hypothesis, subjects make choices in two frames. The \emph{Model frame} mirrors Part 1 of the experiment (epistemic uncertainty; Panel A of Figure \ref{fig:interfaceAmbiguity}). The \emph{Gamble frame} presents structurally identical choices in a balls-and-urns setting (aleatory uncertainty; Panel D). Subjects learn that either a blue or an orange ball will be drawn from an urn of unknown composition and choose between a bet on blue or and a bet on orange.


The second hypothesis is that choices consistent with the Best-Case Promise criterion reflect a failure of contingent reasoning \citep{NiederleVespa.AnnualReviews}. This failure may arise when model predictions are communicated in a way natural to the task but different from how uncertainty is typically presented in the literature. Using the notation of Subsection \ref{sec:utilityCriteria}, the predictions of model $i$ fit to data are naturally expressed as what the subject can expect, according to that model, from following its recommendation ($u(a^*_i, s_i)$) and from choosing a different action ($u(a^*_{-i}, s_i)$), as shown in Panel A of Figure \ref{fig:interfaceAmbiguity}. We refer to this as the \emph{State-Constant} presentation. Crucially, these two values do not describe the payoff distribution the subject faces when choosing model $i$: if she chooses model $i$ but the competing model is correct, her payoff is $u(a^*_{i}, s_{-i})$, which is a prediction of model $-i$, not of model $i$. Determining the possible payoffs from choosing a model thus requires contingent reasoning. Hence, subjects may misinterpret or neglect worst-case payoffs. To test this hypothesis, the \emph{Action-Constant} presentation (Panel B) directly communicates the payoffs from choosing model $i$, namely $\big(u(a^{i}, s_{i}), u(a^{i}, s_{-i})\big)$. For completeness, we also include a State-Constant presentation for choices in the Gamble frame (Panel D).

A difference in choices between the State-Constant and Action-Constant presentations alone does not reveal which presentation reflects subjects’ true preferences. To address this question, we vary the payout structure. In the \emph{Spread} condition, payouts mirror those in Part 1 of the experiment: one option offers a better upside but a worse downside than the other, as necessarily implied by fitting linear models to data (see Appendix \ref{app:theoryMain}). In the \emph{Dominance} condition, one option offers both a better upside and a more favorable downside and is therefore first-order stochastically dominant. We will attribute the choice of a dominated option to a failure of contingent reasoning induced by the presentation mode.\footnote{Choosing a dominated option indicates misunderstanding of worst-case payoffs rather than mere neglect of them. Neglect would still lead subjects to select the dominant option.}


Overall, the Utility-Focused part of our experiment thus follows a  $2\times 2 \times 2$ design, as we vary the frame (Model frame vs. Gamble frame), the presentation mode (State Constant vs. Action Constant) and the payment vectors (Spread vs. Dominance). Subjects encouter each menu twice, with minor variation in payoffs, and thus make decisions in a total of 16 rounds. In one version of each menu, we flip the location of the payoff information on the screen in one but not the other speech bubble.


\subsection{Experiment implementation}\label{sec:designDetails}


All subjects first complete the Comprehensive Part, which is our main focus and cognitively more demanding than the Utility-Focused Part, which subjects complete second. The experiment then proceeds with additional elicitations. Here, we explain key design specifics. We defer details to Appendix \ref{appendix:designDetails}. Appendix \ref{app:instructions} provides screenshots of the complete study interface.

In the Comprehensive Part, we ensure that subjects pay attention to the models by starting each round with a video. For each menu, the video plays a spoken version of the explanation in column 4 of Table \ref{tab:menusMain} and gradually builds the graphical representation shown in columns 2 and 3.\footnote{The video constructs a single DAG, with two indistinguishable grey counters indicating that the models differ only in which counter occupies which position.} The instructions emphasize
that in some rounds both advisors are wrong,
that the data do not affect advisors’ model specifications,
that some advisors’ models may conflict with the correlational data generated by the DGP,
that recommendations and promises result from fitting the model to the data,
and that the model fitting contains no errors,
yet that recommendations and promises from misspecified models are nevertheless incorrect.
They also require subjects to open at least one chart chart showing unconditional correlations and one showing conditional correlations. To ensure comprehension of the instructions and the payoff structure, subjects can only continue once they pass  two comprehension checks that are difficult to answer correctly by chance. The comprehension checks contain no reference to the connection between models and data.\footnote{In each check, subjects classify each of eight statements as correct or incorrect. If they err, they are told only that a mistake occurred, not which or how many statements are wrong, and must revisit the instructions until passing.}


To maximize clarity, we adopt four design choices. First, the data charts omit statistical uncertainty, which could distract from our main research questions and is seldom shown in data visualizations aimed for the general public. Second, to abstract from overfitting, the charts display the true correlations implied by the DGP rather than finite-sample estimates. Third, we choose DGP parameters that yield few negative correlations, as such correlations may be harder for subjects to process, especially when chained in sequence. Fourth, we ensure that recommendations and promises differ clearly within each menu but are as similar as possible across menus. In the High Stakes condition, one model generally promises around Fr. 72 from following it and around Fr. 18 from following the competitor, while the other promises around Fr. 48 from following it and around Fr. 42 from following the competitor. Amounts are reduced by two-thirds in the Low Stakes condition.\footnote{Appendix Table \ref{tab:utilities} lists the precise amounts for each menu. In some cases, it was not possible to choose DGP parameters to produce precisely these amounts as model implications.}


The Utility-Focused Part begins with instructions and is block-randomized: some subjects first make all decisions in the Model frame, others first make all decisions in the Gamble frame. Within each frame, subjects first complete decisions either in the State-Constant or in the Action-Constant presentation mode, and then switch to the respective other presentation mode. Each of these four blocks begins with a screen explaining the payoff format. Subjects must correctly indicate how much they would earn from choosing the option on the left if the option on the right turned out to be ex-post payoff-maximizing. The possible payouts mirror those in the Comprehensive Part. In the \emph{Spread} condition, the payoff vectors are (72, 18) and (48, 42). In the \emph{Dominance} condition, they are (72, 42) and (48, 18). To avoid repetition across rounds, we uniformly randomly vary payment amounts within Fr. 1.20 of these values. All amounts are reduced by two-thirds in the Low Stakes condition.


\label{additionalElicitations}

Toward the end of the study, we elicit several individual characteristics to relate to the use of decision criteria. These include each subject’s field of study (classified as STEM, economics and business, or other) and measures of familiarity with concepts in probabilistic causal inference: completing the aphorism “correlation does not…,” writing out the name of the mathematical object $P(A|B)$ in words, spelling out the acronym “DAG,” and reporting whether they have taken a class on causal statistical inference.\footnote{We score the first three items by whether responses include the strings “caus,” “conditional” or “given,” and “acyc,” respectively.}

We also elicit risk preferences following \citet{EckelGrossman} and ambiguity preferences following \citet{Dimmocketal.}, both incentivized. To reduce noise and allow OR-IV estimation \citep{ORIV}, subjects complete two versions of each elicitation with slightly different parameters. We also administer an extended Cognitive Reflection Test \citep{CRT, CRT3} and measure belief in pseudoscience \citep{Torresetal.}. Subjects report their gender and the Swiss political party closest to their views, which we locate on the political spectrum using \citet{chapelHillExpertSurvey}.\footnote{The survey lacks a score for the Swiss Communist Party (“Partei der Arbeit”); we assign it a value of 0, the leftmost position.} Finally, subjects describe in their own words how they typically made decisions in the main rounds of the experiment.


Subjects learn that a single decision from the entire study will be randomly selected for payment. To eliminate the influence of standard risk preferences, subjects are paid the expected value of their chosen option according to the DGP. The model’s stochasticity serves solely to generate variation for data fitting.


\section{Analysis}\label{sec:analysis}\label{sec:dataCollection}


We conducted the experiment with 414 subjects in June and July 2025 at the Laboratory for Experimental and Behavioral Economics at the University of Zurich. The mean payment was Fr. 47.63 (USD 60.00), with averages of Fr. 31.96 and Fr. 63.78 in the Low and High Stakes conditions, respectively.\footnote{We preregistered a target sample size of 400 subjects, \url{https://www.socialscienceregistry.org/trials/16173}.} Appendix \ref{app:summaryStats} provides summary statistics of our sample's composition. Subjects were required to spend at least 75 minutes in the lab. Some took up to 2 hours, with a median completion time of around 80 minutes. The median time spent on a main round was 50 seconds.\footnote{The median time spent on a practice round was 119 seconds. Appendix \ref{app:orderEffects} examines order effects. Minor order effects in terms of time spent per round appear to reflect learning rather than fatigue, as subjects viewed data charts at a constant rate throughout the experiment.}


We organize the analysis into five subsections. The first three concern the Comprehensive Part.
Subsection \ref{sec:main} documents subjects’ remarkable ability to discard misspecified models and presents the main estimates of all decision criteria.
Subsection \ref{sec:bothWrong} focuses on menus in which all available models are misspecified to show that subjects draw inferences by detecting qualitative inconsistencies between model implications and empirical data rather than through quantitative reasoning.
Subsection \ref{sec:outOfSample} uses out-of-sample prediction analysis to demonstrate the robustness of our results. It also provides parsimonious models whose predictive accuracy rivals that of our unrestricted model.
Subsection \ref{sec:utilityBased} studies the Best-Case Promise criterion by analyzing choices from the Utility-Focused Part of the experiment.
Finally, Subsection \ref{sec:demographics} relates decision-making to subjects' educational background, political preferences, and demographic and psychological traits.


\subsection{Comprehensive Part}\label{sec:main}\label{identificationExperiment1}\label{structModel}


Our main evidence consists of the incentivized choices across the 14 menus in set $\mathcal{M}$. To help interpret these choices, Panel A of Figure \ref{fig:fingerprints} shows the predicted choice distributions for selected types across the menus. A subject able to link each model’s correlational implications to the data will always select the correct model. In contrast, a subject who processes only unconditional correlations will select the correct model in menus $M_4$ to $M_7$ and $M_{11}$ to $M_{14}$. For the remaining menus, light blue shading indicates uniform randomization, reflecting the randomization assumption made in Section \ref{sec:criteriaAndTypes}.

Panel B shows the aggregate empirical choice distributions for the Low and High Stakes conditions. Three patterns emerge. First, subjects often choose the correct model. The aggregate data most closely match the type that consistently selects the correct model. In fact, in each round, at least two thirds of subjects choose the correct model, and the average subject does so in 10 of the 14 rounds---well above the random-choice benchmark of 7. Second, choices of misspecified models are also common (about one fifth to one third of cases). These choices vary systematically across menus but do not resemble any single type’s pattern, indicating heterogeneity across subjects. Third, choice distributions are similar across stakes conditions: higher stakes do not lead subjects to select the correct model more often, if at all. Hence, failures to identify the correct model reflect limited ability rather than lack of motivation.\footnote{Pilot experiments on Prolific also showed no stake effects. However, more than 80\% of those subjects refused to view even a single data chart, apparently attempting to rush through the study.}


\begin{figure}[htp]
\caption{Fingerprints of selected choice criteria \label{fig:fingerprints}}
\begin{center}
\vspace{-0.6cm}

\begin{tikzpicture}\matrix [matrix of nodes, row sep=-1\pgflinewidth, column sep=-1\pgflinewidth, nodes={minimum width=0.85cm, minimum height=0.5cm, inner sep=0pt, anchor=center, text=black, font=\normalfont}, column 1/.style={nodes={text width=2.25cm, align=left, anchor=west}}] {
|[fill=none,draw=none,text=black]| Menu &|[fill=none,draw=black,minimum width=0.85cm,inner sep=0pt]| $M_{1}$&|[fill=none,draw=black,minimum width=0.85cm,inner sep=0pt]| $M_{2}$&|[fill=none,draw=black,minimum width=0.85cm,inner sep=0pt]| $M_{3}$&|[fill=none,draw=black,minimum width=0.85cm,inner sep=0pt]| $M_{4}$&|[fill=none,draw=black,minimum width=0.85cm,inner sep=0pt]| $M_{5}$&|[fill=none,draw=black,minimum width=0.85cm,inner sep=0pt]| $M_{6}$&|[fill=none,draw=black,minimum width=0.85cm,inner sep=0pt]| $M_{7}$&|[fill=none,draw=black,minimum width=0.85cm,inner sep=0pt]| $M_{8}$&|[fill=none,draw=black,minimum width=0.85cm,inner sep=0pt]| $M_{9}$&|[fill=none,draw=black,minimum width=0.85cm,inner sep=0pt]| $M_{10}$&|[fill=none,draw=black,minimum width=0.85cm,inner sep=0pt]| $M_{11}$&|[fill=none,draw=black,minimum width=0.85cm,inner sep=0pt]| $M_{12}$&|[fill=none,draw=black,minimum width=0.85cm,inner sep=0pt]| $M_{13}$&|[fill=none,draw=black,minimum width=0.85cm,inner sep=0pt]| $M_{14}$\\};
\end{tikzpicture}\

\\
\textbf{A. Theoretical predictions of selected types} \\

{\centering \raisebox{0.6\baselineskip}[0pt][0pt]{\rule{\textwidth}{0.4pt}}\par}


\centerline{\textbf{All Correlations}}\vspace{-0.5cm}
\begin{tikzpicture}\matrix [name=matrix, matrix of nodes, row sep=-1\pgflinewidth, column sep=-1\pgflinewidth, nodes={minimum width=0.85cm, minimum height=0.5cm, inner sep=0pt, anchor=center, text=black, font=\normalfont}, column 1/.style={nodes={text width=2.25cm, align=left, anchor=west}}] {
|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|\\|[fill=none,draw=none,text=black]|DGP &|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|\\|[fill=none,draw=none,text=black]|Competitor&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|\\|[fill=none,draw=none,text=black]| &|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|\\};
\end{tikzpicture}


\centerline{\textbf{Only Unconditional Correlations}}\vspace{-0.5cm}
\begin{tikzpicture}\matrix [name=matrix, matrix of nodes, row sep=-1\pgflinewidth, column sep=-1\pgflinewidth, nodes={minimum width=0.85cm, minimum height=0.5cm, inner sep=0pt, anchor=center, text=black, font=\normalfont}, column 1/.style={nodes={text width=2.25cm, align=left, anchor=west}}] {
|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|\\|[fill=none,draw=none,text=black]|DGP &|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|\\|[fill=none,draw=none,text=black]|Competitor&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|\\|[fill=none,draw=none,text=black]| &|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|\\};
\end{tikzpicture}


\centerline{\textbf{Only Conditional Correlations}}\vspace{-0.5cm}
\begin{tikzpicture}\matrix [name=matrix, matrix of nodes, row sep=-1\pgflinewidth, column sep=-1\pgflinewidth, nodes={minimum width=0.85cm, minimum height=0.5cm, inner sep=0pt, anchor=center, text=black, font=\normalfont}, column 1/.style={nodes={text width=2.25cm, align=left, anchor=west}}] {
|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|\\|[fill=none,draw=none,text=black]|DGP &|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|\\|[fill=none,draw=none,text=black]|Competitor&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|\\|[fill=none,draw=none,text=black]| &|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|\\};
\end{tikzpicture}


\centerline{\textbf{Direct Links}}\vspace{-0.5cm}
\begin{tikzpicture}\matrix [name=matrix, matrix of nodes, row sep=-1\pgflinewidth, column sep=-1\pgflinewidth, nodes={minimum width=0.85cm, minimum height=0.5cm, inner sep=0pt, anchor=center, text=black, font=\normalfont}, column 1/.style={nodes={text width=2.25cm, align=left, anchor=west}}] {
|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|\\|[fill=none,draw=none,text=black]|DGP &|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|\\|[fill=none,draw=none,text=black]|Competitor&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!50,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|\\|[fill=none,draw=none,text=black]| &|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|\\};
\end{tikzpicture}


\centerline{\textbf{High Promise}}\vspace{-0.5cm}
\begin{tikzpicture}\matrix [name=matrix, matrix of nodes, row sep=-1\pgflinewidth, column sep=-1\pgflinewidth, nodes={minimum width=0.85cm, minimum height=0.5cm, inner sep=0pt, anchor=center, text=black, font=\normalfont}, column 1/.style={nodes={text width=2.25cm, align=left, anchor=west}}] {
|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|\\|[fill=none,draw=none,text=black]|DGP &|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|\\|[fill=none,draw=none,text=black]|Competitor&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|\\|[fill=none,draw=none,text=black]| &|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|\\};
\end{tikzpicture}


\centerline{\textbf{Low Spending}}\vspace{-0.5cm}
\begin{tikzpicture}\matrix [name=matrix, matrix of nodes, row sep=-1\pgflinewidth, column sep=-1\pgflinewidth, nodes={minimum width=0.85cm, minimum height=0.5cm, inner sep=0pt, anchor=center, text=black, font=\normalfont}, column 1/.style={nodes={text width=2.25cm, align=left, anchor=west}}] {
|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|\\|[fill=none,draw=none,text=black]|DGP &|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|\\|[fill=none,draw=none,text=black]|Competitor&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!0,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|&|[fill=airforceblue!100,draw=black,minimum width=0.85cm,inner sep=0pt]|\\|[fill=none,draw=none,text=black]| &|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|\\};
\end{tikzpicture}


\\
\textbf{B. Empirical choices}  \\

{\centering \raisebox{0.6\baselineskip}[0pt][0pt]{\rule{\textwidth}{0.4pt}}\par}


\centerline{\textbf{Low stakes}}
\begin{tikzpicture}\matrix [matrix of nodes, row sep=-1\pgflinewidth, column sep=-1\pgflinewidth,nodes={minimum width=0.85cm, minimum height=0.5cm, inner sep=0pt, anchor=center, text=black, font=\normalfont}, column 1/.style={nodes={text width=2.25cm, align=left, anchor=west}}] {
|[fill=none,draw=none,text=black]|DGP &|[fill=airforceblue!100,draw=black,text=white]| 74&|[fill=airforceblue!100,draw=black,text=white]| 77&|[fill=airforceblue!95,draw=black,text=white]| 70&|[fill=airforceblue!95,draw=black,text=white]| 70&|[fill=airforceblue!88,draw=black,text=white]| 67&|[fill=airforceblue!100,draw=black,text=white]| 72&|[fill=airforceblue!100,draw=black,text=white]| 75&|[fill=airforceblue!93,draw=black,text=white]| 69&|[fill=airforceblue!95,draw=black,text=white]| 70&|[fill=airforceblue!100,draw=black,text=white]| 76&|[fill=airforceblue!100,draw=black,text=white]| 75&|[fill=airforceblue!95,draw=black,text=white]| 70&|[fill=airforceblue!95,draw=black,text=white]| 70&|[fill=airforceblue!100,draw=black,text=white]| 79\\|[fill=none,draw=none,text=black]|Competitor &|[fill=airforceblue!28,draw=black,text=black]| 26&|[fill=airforceblue!23,draw=black,text=black]| 23&|[fill=airforceblue!40,draw=black,text=black]| 30&|[fill=airforceblue!40,draw=black,text=black]| 30&|[fill=airforceblue!49,draw=black,text=black]| 33&|[fill=airforceblue!34,draw=black,text=black]| 28&|[fill=airforceblue!25,draw=black,text=black]| 25&|[fill=airforceblue!43,draw=black,text=black]| 31&|[fill=airforceblue!40,draw=black,text=black]| 30&|[fill=airforceblue!24,draw=black,text=black]| 24&|[fill=airforceblue!25,draw=black,text=black]| 25&|[fill=airforceblue!40,draw=black,text=black]| 30&|[fill=airforceblue!40,draw=black,text=black]| 30&|[fill=airforceblue!21,draw=black,text=black]| 21\\|[fill=none,draw=none,text=black]|  &|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|\\};
\end{tikzpicture}
\centerline{\textbf{High stakes}}
\begin{tikzpicture}\matrix [matrix of nodes, row sep=-1\pgflinewidth, column sep=-1\pgflinewidth,nodes={minimum width=0.85cm, minimum height=0.5cm, inner sep=0pt, anchor=center, text=black, font=\normalfont}, column 1/.style={nodes={text width=2.25cm, align=left, anchor=west}}] {
|[fill=none,draw=none,text=black]|DGP &|[fill=airforceblue!85,draw=black,text=white]| 66&|[fill=airforceblue!100,draw=black,text=white]| 73&|[fill=airforceblue!93,draw=black,text=white]| 69&|[fill=airforceblue!83,draw=black,text=white]| 65&|[fill=airforceblue!95,draw=black,text=white]| 70&|[fill=airforceblue!95,draw=black,text=white]| 70&|[fill=airforceblue!100,draw=black,text=white]| 77&|[fill=airforceblue!90,draw=black,text=white]| 68&|[fill=airforceblue!100,draw=black,text=white]| 72&|[fill=airforceblue!100,draw=black,text=white]| 76&|[fill=airforceblue!100,draw=black,text=white]| 74&|[fill=airforceblue!100,draw=black,text=white]| 74&|[fill=airforceblue!90,draw=black,text=white]| 68&|[fill=airforceblue!100,draw=black,text=white]| 72\\|[fill=none,draw=none,text=black]|Competitor &|[fill=airforceblue!52,draw=black,text=white]| 34&|[fill=airforceblue!31,draw=black,text=black]| 27&|[fill=airforceblue!43,draw=black,text=black]| 31&|[fill=airforceblue!55,draw=black,text=white]| 35&|[fill=airforceblue!40,draw=black,text=black]| 30&|[fill=airforceblue!40,draw=black,text=black]| 30&|[fill=airforceblue!23,draw=black,text=black]| 23&|[fill=airforceblue!46,draw=black,text=black]| 32&|[fill=airforceblue!34,draw=black,text=black]| 28&|[fill=airforceblue!24,draw=black,text=black]| 24&|[fill=airforceblue!28,draw=black,text=black]| 26&|[fill=airforceblue!28,draw=black,text=black]| 26&|[fill=airforceblue!46,draw=black,text=black]| 32&|[fill=airforceblue!34,draw=black,text=black]| 28\\|[fill=none,draw=none,text=black]|  &|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|&|[fill=none,draw=none]|\\};
\end{tikzpicture}


\end{center}
\vspace{-0.6cm}
\footnotesize
\textbf{\textit{Notes:}} Each column corresponds to a menu. We use dark shading if the criterion chooses the corresponding option, and no shading if the rule does not choose the option. Intermediate shades indicate the number of tied options. In Panel B, numbers list the percentage of subjects choosing a given option. The extent of shading reflects percentages.
\end{figure}


\paragraph{Mixture model}
To draw detailed inferences about the use of each decision criterion, we fit a finite mixture model that yields an estimate $(\hat{t}_1, \ldots, \hat{t}_n)$ of the frequency $t_i$ of each type $i$ in our data. Conceptually, summing the theoretically predicted choice distributions in Panel A of Figure \ref{fig:fingerprints}, weighted by a candidate type distribution $\tilde{t} = (\tilde{t}_1, \ldots, \tilde{t}_n)$, gives the aggregate choice distribution we would expect to observe. The estimation procedure searches for the distribution $\tilde{t}$ that minimizes the distance between predicted and observed aggregate choice distributions. To allow for stochastic choice, we assume that in each round a subject either selects a type-consistent option with probability $(1-q)$ or randomizes uniformly and independently across all options with probability $q$, following \citet{CostaGomesCrawford, AmbuehlBernheim}.\footnote{We designed the experiment to make this assumption plausible. Subjects saw menus in individually randomized order, ensuring that any inattention is evenly distributed across menus. The screen position of models was also randomized, so tendencies such as always choosing the right-hand option appear as uniform randomization across menus.}


We define the distance between the predicted and observed choice distributions as the weighted sum of all first and second moments. The first moments are the differences between predicted and observed probabilities of choosing the correct model for each menu $m$. The second moments are the corresponding differences conditional on choices in another menu $m'$, for all menu pairs $(m, m')$.\footnote{Second moments are needed for identification. Suppose there are two menus, each with two options (A, B), and three types: type 1 always chooses A, type 2 always chooses B, and type 3 randomizes. If A is chosen 50\% of the time in both menus, first moments alone cannot distinguish between a population of 50\% type 1 and 50\% type 2, a population of only type 3, or a mixture of the two. Including second moments resolves this, as only type 3 produces the pattern of choosing A in one menu and B in the other.}  We estimate the model using the generalized method of moments (GMM), applying the standard two-stage feasible GMM procedure to obtain optimal moment weights. Appendix \ref{appendix:GMM} provides details.


Our mixture model has three attractive features. First, it guarantees that the estimated type distribution converges to the true population distribution as the number of subjects increases. Individual-level approaches such as Bayesian classifiers offer no such guarantee when the number of decisions per subject is fixed. They may therefore perform poorly when choices are noisy, as our simulations confirm. Second, holding the stochastic choice probability $q$ fixed, the predicted aggregate choice distribution is a linear function of the type frequencies. This linearity allows analytical proof of identification: a sequence of menus identifies the vector of type weights if this linear function has full rank.\footnote{We prove identification with the noise probability held fixed and verify through simulations that the estimator correctly recovers the noise probability in synthetic data.} Third, because of linearity, the GMM objective function is quadratic for any fixed noise probability. Numerical optimization is therefore rapid and will not get stuck in local optima, even for large type sets.\footnote{Because these properties hold only when the noise probability is fixed, we define a grid of starting values for it, run the minimization for each, and select the global optimum.} The model has two limitations. First, it assumes that the mean noise probability is identical across types. Second, the stochastic choice probability and the share of subjects who randomize uniformly throughout are not separately identifiable. We interpret the data under the assumption that the latter share is zero.


 \paragraph{Model estimates} Our estimation results confirm the visual impression from Figure \ref{fig:fingerprints}: in both the Low and High Stakes conditions, the most common type draws correct inferences from both conditional and unconditional correlations. This type accounts for 65.4\% (s.e. 4.9\%) and 56.9\% (s.e. 4.8\%) of subjects in the Low and High Stakes conditions, respectively. These shares do not imply that more than half the subjects identify the correct model in every round. The estimated noise parameters are 24.2\% (s.e. 2.0\%) and 21.4\% (s.e. 2.0\%) in the Low and High Stakes conditions, respectively. Thus, even subjects who generally identify the correct advisor randomize in roughly one out of five rounds and thus select the wrong model in about one out of ten, with such errors distributed randomly across menus and options. Each remaining type accounts for less than 10\% of subjects, though many receive nontrivial weight. Appendix \ref{appendix:rawTypes} reports the complete estimated type vector.


\begin{figure}[htb]
\caption{Distribution of decision criteria
\label{fig:criteriaDistributionPooledInclTest}}
\begin{center}
\begin{tabular}{cc}
\textbf{A. Fact-based criteria} &
\textbf{B. Utility-based criteria} \\
\includegraphics[width=0.48\textwidth]{graphs/freqAccStructPooledInclTest} &
\includegraphics[width=0.48\textwidth]{graphs/freqMotStructPooledInclTest} \\
\end{tabular}
\end{center}
\vspace{-0.5cm}
\footnotesize
\textit{\textbf{Notes}}: Pooled across stakes, and training and test sets. Whiskers show 95\%-confidence intervals, truncated at 0. \textit{Panel A:}   \textit{Panel B:} Estimates of advice-based criteria in Experiment 1 are shown conditional on not using the Conditional Correlations criterion.
\end{figure}


To summarize the remaining subjects, recall that each type combines a Fact-Based and a Utility-Based criterion. We estimate the prevalence of each Fact-Based criterion by summing over all types that apply it. Likewise, marginalizing over Fact-Based criteria yields the distribution of Utility-Based criteria. Aggregating across types also helps average out noise that could affect estimates when many types are fitted to limited data.


Panel A of Figure \ref{fig:criteriaDistributionPooledInclTest} plots the distribution of Fact-Based criteria using all 14 rounds and pooling across the stakes conditions.\footnote{Appendix \ref{app:distCritPrereg} shows these distributions estimated only using the training set and split by stakes condition, as preregistered. The qualitative results are unchanged, although the use of a much smaller sample greatly increases the standard errors of the estimates.} Next to the majority of subjects who consistently identify the correct model, 24.1\% make partly fact-based decisions. While similar numbers of subjects use each of the partial Fact-Based criteria, two of these three criteria only require an understanding of unconditional but not conditional correlations. The remaining 17.5\% (s.e. 1.7\%) use no Fact-Based criterion at all.

Panel B displays the distribution of Utility-Based criteria, focusing on subjects who do not consistently select the correct model. Support for the Best-Case Promise criterion is minimal (3.8\%, s.e. 1.6\%). Instead, subjects who deploy Utility-Based criteria optimize worst-case outcomes (18.3\%, s.e. 3.1\%), either through the Worst-Case Promise criterion (11.6\%, s.e. 1.7\%) or through the Minimal Investment criterion (6.8\%, s.e. 2.4\%). Nonetheless, among subjects who fail to identify the correct model through Fact-Based criteria in some rounds, the modal choice (17.5\%, s.e. 1.7\%) is uniform randomization, consistent with the principle of insufficient reason.


\paragraph{Decision strategies}
What strategies help subjects identify the correct model? We argue that subjects rely on qualitative inference, looking for inconsistencies between model implications and the data. A competing hypothesis is that subjects rely on quantitative inference by visually estimating the return to investment from the chart showing the relation between investment and bonus. While a few subjects report doing so when asked to describe their decision-making in words, viewing data show this behavior is rare. Subjects view the investment/bonus-chart exclusively in only 2.1\% of cases. Moreover, they view it at all in just 26.2\% of cases, compared with 42.4\% and 41.7\% for the charts relating investment to each covariate, respectively. For subjects seeking qualitative inconsistencies between model implications and data, the latter charts are useful in many rounds, whereas the investment/bonus-chart never serves that purpose.

The fact that subjects frequently choose the correct model despite receiving no assistance suggests that the qualitative, model-based inference in our experiment comes naturally to many people, unlike the purely quantitative reasoning required in standard balls-and-urns belief-updating tasks.


How do subjects draw these inferences? Do they reason from data to models, viewing charts until a pattern stands out to them, or from models to data, intuiting the models’ correlational implications and checking the data for those that distinguish the models? Several observations support the latter hypothesis. Of the 18 charts available per menu, the median subject views only four, consistent with the idea that they know what to look for. Of the 12 charts showing conditional correlations, the average subject views 2.84, of which only 0.68 do not distinguish the models. Viewing non-distinguishing charts can still be useful, as it helps detect when both causal models in a round are misspecified. Targeting is less precise for unconditional correlations, likely due to individual heterogeneity, but still consistent with purposeful selection: of the 6 charts showing unconditional correlations, subjects view on average 1.85, roughly half of which (0.87) distinguish the models. This far exceeds what would be expected if chart selection were random.\footnote{In an average round, 86\% of subjects view at least one data chart. Overall, subjects view at least one chart that distinguishes the models in 60.3\% of rounds, consistent with our estimated Fact-Based criteria in Figure \ref{fig:criteriaDistributionPooledInclTest}, given the estimated random-choice probability of 24.9\%.}


\FloatBarrier


 \paragraph{Subjects' general sophistication}
A potential criticism of our interpretation is that our participants’ performance reflects an unusual level of sophistication and therefore may not generalize to other subject pools. Our data cast doubt on this view. Although a substantial share of subjects major in STEM (64\%), their statistical sophistication is limited. Only 54\% can complete the aphorism “Correlation does not\ldots,” 22.5\% report having taken a class on statistical causal inference, one quarter can name the expression `$P(A|B)$,' and 8\% can spell out “DAG.” A comparison of our subjects' mean CRT score of 71.4\% to that of other subject pools suggests a similar conclusion. Students at TU and LMU Munich achieve 75\%, \citep[][]{Coutts2025}, and those at Mannheim University and Osaka University score 68\% and 83\%, respectively \citep[][]{glaser2019thinking, Hanaki2021}. While these scores are higher than those found in representative U.S. samples \citep[58.3\%,][]{Caplin2023}, they fall short of scores observed among professionals in the financial \citep{thoma2015good, angrisani2022strategic, weitzel2020bubbles} and oil-producing \citep{welsh2017cognitive} industries.\footnote{Different studies use different extensions of the CRT test. While we use \citet{CRT3}, the studies cited above predominantly use \citet{CRT2}, which limits comparability. (\citet{Hanaki2021} also includes questions from \citet{finucane2010developing}). Performance on the three original CRT questions \citep{CRT} is available for three of these studies. Subjects in \citet{glaser2019thinking},  \citet{thoma2015good}, and \citet{welsh2017cognitive} answer 68\%,  91\%, and 81\% correctly, respectively. Our subjects' score of 77.3\% on those three questions falls well within this range.}


Overall, we find that subjects display a remarkable ability to intuit models’ correlational implications and verify them against empirical data to rule out misspecified models. Utility-Based criteria that receive empirical support maximize worst-case rather than best-case payoffs.\footnote{When asked to describe their decision process, many subjects report trying to infer the correct model from the data and defaulting to payoff safety when uncertain.} Support for the central assumption of the Narrative Competition literature, the Best-Case Promise criterion, is minimal. These conclusions remain robust to a threefold increase in stakes.


\begin{table}[htb]
\small
\caption{Menus with two misspecified models}
\setlength{\tabcolsep}{3pt}
\label{tab:menusWrong}


\resizebox{\textwidth}{!}{
\begin{tabular}{l c c c c}
\toprule
 & $W_{1}$ & $W_{2}$ & $W_{3}$ & $W_{4}$ \\
\midrule
A. DGP and model structures \\
\noalign{\vskip -0.5ex}
\quad DGP & \begin{tikzpicture}[>=Stealth, scale=1, baseline=(current bounding box.north)]
\scriptsize
\node (Z) at (-0.5,2) {$C_2$};
\node (A) at (-0.5,1) {$I$};
\node (Y) at (0,0) {$Y$};
\node (X) at (0.5,2) {$C_1$};
\draw[->] (A) to[out=-90, in=180, looseness=1] (Y);
\draw[->] (X) to[out=-90, in=0, looseness=0.7] (Y);
\draw[->] (A) -- (Z);
\end{tikzpicture}  & \begin{tikzpicture}[>=Stealth, scale=1, baseline=(current bounding box.north)]
\scriptsize
\node (A) at (0,2) {$I$};
\node (X) at (0,1) {$C_1$};
\node (Y) at (0,0) {$Y$};
\node (Z) at (-0.5,0.5) {$C_2$};
\draw[->] (A) -- (X);
\draw[->] (Z) to[out=-90, in=180, looseness=1] (Y);
\draw[->] (X) to[out=0, in=0, looseness=1.3] (Y);
\draw[->] (X) to[out=180, in=90, looseness=1] (Z);
\end{tikzpicture} & \begin{tikzpicture}[>=Stealth, scale=1, baseline=(current bounding box.north)]
\scriptsize
\node (A) at (0,2) {$I$};
\node (X) at (0,1) {$C_1$};
\node (Y) at (0,0) {$Y$};
\node (Z) at (1,0) {$C_2$};
\draw[->] (A) -- (X);
\draw[->] (Y) -- (Z);
\draw[->] (X) -- (Y);
\end{tikzpicture} & \begin{tikzpicture}[>=Stealth, scale=1, baseline=(current bounding box.north)]
\scriptsize
\node (A) at (0,2) {$I$};
\node (X) at (0,1) {$C_1$};
\node (Y) at (0,0) {$Y$};
\node (Z) at (1,1) {$C_2$};
\draw[->] (A) -- (X);
\draw[->] (X) -- (Z);
\draw[->] (X) -- (Y);
\end{tikzpicture} \\
\quad Model 1 & \begin{tikzpicture}[>=Stealth, scale=1, baseline=(current bounding box.north)]
\scriptsize
\node (A) at (0,2) {$I$};
\node (X) at (0,1) {$C_1$};
\node (Y) at (0,0) {$Y$};
\node (Z) at (0.5,1.5) {$C_2$};
\draw[->] (A) to[out=180, in=180, looseness=1.3] (X);
\draw[->] (A) to[out=0, in=90, looseness=1] (Z);
\draw[->] (X) to[out=0, in=-90, looseness=1] (Z);
\draw[->] (X) -- (Y);
\end{tikzpicture} & \begin{tikzpicture}[>=Stealth, scale=1, baseline=(current bounding box.north)]
\scriptsize
\node (A) at (-0.5,2) {$I$};
\node (X) at (-0.5,1) {$C_1$};
\node (Y) at (0,0) {$Y$};
\node (Z) at (0.5,2) {$C_2$};
\draw[->] (X) to[out=-90, in=180, looseness=1] (Y);
\draw[->] (Z) to[out=-90, in=0, looseness=0.7] (Y);
\draw[->] (A) -- (X);
\end{tikzpicture} & \begin{tikzpicture}[>=Stealth, node distance=0.5cm, baseline=(current bounding box.north)]
\scriptsize
\node (A) {$I$};
\node[below=of A] (X) {$C_1$};
\node[below=of X] (Z) {$C_2$};
\node[below=of Z] (Y) {$Y$};
\draw[->] (A) -- (X);
\draw[->] (X) -- (Z);
\draw[->] (Z) -- (Y);
\end{tikzpicture} & \begin{tikzpicture}[>=Stealth, scale=1, baseline=(current bounding box.north)]
\scriptsize
\node (A) at (0,2) {$I$};
\node (X) at (0,1) {$C_1$};
\node (Y) at (0,0) {$Y$};
\node (Z) at (1,0) {$C_2$};
\draw[->] (A) -- (X);
\draw[->] (Y) -- (Z);
\draw[->] (X) -- (Y);
\end{tikzpicture} \\
\quad Model 2 & \begin{tikzpicture}[>=Stealth, scale=1, baseline=(current bounding box.north)]
\scriptsize
\node (A) at (0,2) {$I$};
\node (X) at (0,1) {$C_2$};
\node (Y) at (0,0) {$Y$};
\node (Z) at (0.5,1.5) {$C_1$};
\draw[->] (A) to[out=180, in=180, looseness=1.3] (X);
\draw[->] (A) to[out=0, in=90, looseness=1] (Z);
\draw[->] (X) to[out=0, in=-90, looseness=1] (Z);
\draw[->] (X) -- (Y);
\end{tikzpicture} & \begin{tikzpicture}[>=Stealth, scale=1, baseline=(current bounding box.north)]
\scriptsize
\node (A) at (-0.5,2) {$I$};
\node (X) at (-0.5,1) {$C_2$};
\node (Y) at (0,0) {$Y$};
\node (Z) at (0.5,2) {$C_1$};
\draw[->] (X) to[out=-90, in=180, looseness=1] (Y);
\draw[->] (Z) to[out=-90, in=0, looseness=0.7] (Y);
\draw[->] (A) -- (X);
\end{tikzpicture} & \begin{tikzpicture}[>=Stealth, node distance=0.5cm, baseline=(current bounding box.north)]
\scriptsize
\node (A) {$I$};
\node[below=of A] (X) {$C_2$};
\node[below=of X] (Z) {$C_1$};
\node[below=of Z] (Y) {$Y$};
\draw[->] (A) -- (X);
\draw[->] (X) -- (Z);
\draw[->] (Z) -- (Y);
\end{tikzpicture} & \begin{tikzpicture}[>=Stealth, scale=1, baseline=(current bounding box.north)]
\scriptsize
\node (A) at (0,2) {$I$};
\node (X) at (0,1) {$C_2$};
\node (Y) at (0,0) {$Y$};
\node (Z) at (1,0) {$C_1$};
\draw[->] (A) -- (X);
\draw[->] (Y) -- (Z);
\draw[->] (X) -- (Y);
\end{tikzpicture} \\
\quad \textit{Description} & \begin{tabular}[t]{@{}p{3.3cm}@{}}\scriptsize Your Action indirectly influences your Bonus through one of the Counters. The second Counter is a symptom of both your Action and the first Counter.\end{tabular} & \begin{tabular}[t]{@{}p{3.3cm}@{}}\scriptsize Your Action indirectly influences your Bonus through one of the counters. The other counter is not influenced by anything, but it has an additional, direct influence on your Bonus.\end{tabular} & \begin{tabular}[t]{@{}p{3.3cm}@{}}\scriptsize Your Action indirectly influences your Bonus. It influences the first Counter which, in turn, influences the second Counter, which then influences your Bonus.\end{tabular} & \begin{tabular}[t]{@{}p{3.3cm}@{}}\scriptsize Your Action indirectly influences your Bonus through one of the Counters. The second Counter is a symptom of the Bonus.\end{tabular} \\
\midrule
B. Inconsistencies with DGP \\ \quad \textit{Unconditional correlations} \\\qquad Model 1 & 4 & 4 & 0 & 0 \\
\qquad Model 2 & 4 & 4 & 0 & 0 \\
\quad \textit{Conditional correlations} \\\qquad Model 1 & 10 & 2 & 8 & 6 \\
\qquad Model 2 & 10 & 10 & 12 & 14 \\
\midrule
\multicolumn{5}{l}{C. Implied optimal investment}\\
\qquad DGP & 12.40      & 78.16      & 28.88      & 18.07      \\
\qquad Model 1 & 0.00       & 8.03       & 24.68      & 18.07      \\
\qquad Model 2 & 10.04      & 36.08      & 3.26       & 1.17       \\
\midrule
\multicolumn{5}{l}{D. Choice frequencies}\\
\quad \textit{Low stakes} \\\qquad Model 1 & {\setlength{\fboxsep}{0pt}\fcolorbox{black}{airforceblue!76}{\makebox[\dimexpr\linewidth-1000\fboxrule\relax][c]{\strut\textcolor{white}{51}}}} & {\setlength{\fboxsep}{0pt}\fcolorbox{black}{airforceblue!81}{\makebox[\dimexpr\linewidth-1000\fboxrule\relax][c]{\strut\textcolor{white}{61}}}} & {\setlength{\fboxsep}{0pt}\fcolorbox{black}{airforceblue!93}{\makebox[\dimexpr\linewidth-1000\fboxrule\relax][c]{\strut\textcolor{white}{69}}}} & {\setlength{\fboxsep}{0pt}\fcolorbox{black}{airforceblue!90}{\makebox[\dimexpr\linewidth-1000\fboxrule\relax][c]{\strut\textcolor{white}{68}}}} \\
\qquad Model 2 & {\setlength{\fboxsep}{0pt}\fcolorbox{black}{airforceblue!75}{\makebox[\dimexpr\linewidth-1000\fboxrule\relax][c]{\strut\textcolor{white}{49}}}} & {\setlength{\fboxsep}{0pt}\fcolorbox{black}{airforceblue!67}{\makebox[\dimexpr\linewidth-1000\fboxrule\relax][c]{\strut\textcolor{white}{39}}}} & {\setlength{\fboxsep}{0pt}\fcolorbox{black}{airforceblue!43}{\makebox[\dimexpr\linewidth-1000\fboxrule\relax][c]{\strut\textcolor{black}{31}}}} & {\setlength{\fboxsep}{0pt}\fcolorbox{black}{airforceblue!46}{\makebox[\dimexpr\linewidth-1000\fboxrule\relax][c]{\strut\textcolor{black}{32}}}} \\
\quad \textit{High stakes} \\\qquad Model 1 & {\setlength{\fboxsep}{0pt}\fcolorbox{black}{airforceblue!75}{\makebox[\dimexpr\linewidth-1000\fboxrule\relax][c]{\strut\textcolor{white}{50}}}} & {\setlength{\fboxsep}{0pt}\fcolorbox{black}{airforceblue!79}{\makebox[\dimexpr\linewidth-1000\fboxrule\relax][c]{\strut\textcolor{white}{57}}}} & {\setlength{\fboxsep}{0pt}\fcolorbox{black}{airforceblue!81}{\makebox[\dimexpr\linewidth-1000\fboxrule\relax][c]{\strut\textcolor{white}{62}}}} & {\setlength{\fboxsep}{0pt}\fcolorbox{black}{airforceblue!90}{\makebox[\dimexpr\linewidth-1000\fboxrule\relax][c]{\strut\textcolor{white}{68}}}} \\
\qquad Model 2 & {\setlength{\fboxsep}{0pt}\fcolorbox{black}{airforceblue!75}{\makebox[\dimexpr\linewidth-1000\fboxrule\relax][c]{\strut\textcolor{white}{50}}}} & {\setlength{\fboxsep}{0pt}\fcolorbox{black}{airforceblue!72}{\makebox[\dimexpr\linewidth-1000\fboxrule\relax][c]{\strut\textcolor{white}{43}}}} & {\setlength{\fboxsep}{0pt}\fcolorbox{black}{airforceblue!64}{\makebox[\dimexpr\linewidth-1000\fboxrule\relax][c]{\strut\textcolor{white}{38}}}} & {\setlength{\fboxsep}{0pt}\fcolorbox{black}{airforceblue!46}{\makebox[\dimexpr\linewidth-1000\fboxrule\relax][c]{\strut\textcolor{black}{32}}}} \\
\bottomrule
\end{tabular}}


\footnotesize
\vspace{0.2cm}
\textit{\textbf{Notes}}: Panel A shows the structure of the DGP, and the two available models in each menu in set $\mathcal{W}$. Panel B shows the number of inconsistencies between each model and the DGP. Panel C shows the investment amounts implied as optimal by each model when fit to the DGP and by the DGP itself in the Low Stakes condition. Panel D shows empirical choice frequencies in percent.


\end{table}


\subsection{Choices when both advisors are wrong}\label{sec:bothWrong}


We next examine choices on menus in $\mathcal{W}$, where both models are inconsistent with the data.\footnote{The analysis in this subsection is exploratory and was not preregistered.}  A first finding is that the average subject does not merely select the remaining option after ruling out one misspecified model. Instead, subjects also evaluate whether the remaining models fit the data: when neither model is correct, the average number of charts viewed rises from 4.7 to 5.2 ($p<0.01$), and average time per round increases from 50 to 68 seconds.


More importantly, choices from menus in $\mathcal{W}$ provide further evidence that subjects reason qualitatively rather than quantitatively, as follows. Once a subject determines that both available models are misspecified, several decision approaches are possible. First, she may continue reasoning qualitatively and select the model with fewer inconsistencies between the data and its correlational implications. As shown in Panel B of Table \ref{tab:menusWrong}, this criterion applies to all menus in $\mathcal{W}$ except $W_1$ and requires understanding conditional correlational implications. Second, she may reason quantitatively by estimating the strength of the relationship between investment and bonus from the corresponding chart and choosing the model whose recommended investment is closer to the optimal level implied by that chart, as listed in Panel C. Third, she may revert to a Utility-Based criterion when the available Fact-Based criteria of Definition \ref{def:dataBased} do not yield a clear choice.


Panel D displays subjects’ choices. In menus where one model has fewer correlational inconsistencies with the DGP, about 60 to 70\% of subjects choose that model. When both models have the same number of inconsistencies (menu $W_1$), choices are evenly split. This pattern indicates that minimizing the number of inconsistencies is subjects’ preferred criterion. In contrast, quantitative reasoning predicts that subjects predominantly choose model 2 in menus $W_1$ and $W_2$, yet they do so in only 40 to 50\% of cases.


\FloatBarrier


To formally estimate the prevalence of the decision approaches, we fit a version of our mixture model on choices from the menus in $\mathcal{W}$. We exclude all Fact-Based criteria from Definition \ref{def:dataBased} since $\mathcal{W}$ is constructed such that these criteria do not apply. Instead, we include a type that minimizes the number of inconsistencies between the chosen model and the DGP (randomizing when the numbers are equal),\footnote{We assume randomization because subjects rarely combine Utility-Based and Data-Based criteria in Section \ref{sec:main}.} and a type that selects the model whose implied optimal investment is closest to the true value. The latter type’s behavior coincides with the Maximize Investment criterion. Given its negligible incidence in Section \ref{sec:main}, we set the fraction of subjects following that criterion to zero and attribute the corresponding choices to quantitative fact-based reasoning. We also include all types that apply one of the Utility-Based criteria (except the Maximize Investment criterion) from Definition \ref{def:adviceBased} without combining it with any other criterion.


\begin{table}[htb]
\caption{Distribution of types when both models are inconsistent with the DGP}
\label{tab:estimatesBothWrong}
\begin{center}

\begin{tabular}{lccc}
& (1) & (2) & (3) \\
\toprule
Stakes & Low & High & $p$-value \\
\midrule
\multicolumn{4}{l}{\textit{Utility-Based Criteria}} \\
\quad Best-Case Promise & 0.059 & 0.121 & 0.202 \\
 & (0.034) & (0.035) & \\
\quad Worst-Case Promise & 0.086 & 0.222 & 0.005 \\
 & (0.034) & (0.035) & \\
\quad Minimize Investment & 0.108 & 0.067 & 0.359 \\
 & (0.031) & (0.033) & \\
\multicolumn{4}{l}{\textit{Data-Based Criteria}} \\
\quad Closest recommendation & 0.134 & 0.138 & 0.941 \\
 & (0.037) & (0.038) & \\
\quad Fewest Inconsistencies & 0.614 & 0.453 & 0.000 \\
 & (0.028) & (0.035) & \\
\noalign{\medskip}
Noise & 0.450 & 0.418 & 0.682 \\
 & (0.055) & (0.054) & \\
Subjects & 210 & 204 & \\
\bottomrule
\end{tabular}


\end{center}
\footnotesize
\textit{\textbf{Notes:}} Column 3 shows the $p$-values of a $z$-test of the null hypothesis that a criterion is equally common across the Low and High Stakes conditions.
\end{table}


Table \ref{tab:estimatesBothWrong} reports the results. Consistent with the visual evidence from Table \ref{tab:menusWrong}, subjects who reason qualitatively by minimizing the number of inconsistencies are far more prevalent than those who reason quantitatively by selecting the model whose recommended investment is closest to the truth.


\FloatBarrier


\subsection{Out-of-sample predictions and parsimonious models}
\label{sec:parsimoniousModels}\label{sec:outOfSample}

We now perform out-of-sample predictions to test whether our main classification reflects stable behavioral tendencies rather than overfitting. Relatedly, we examine whether more parsimonious models can predict choices similarly well as our unrestricted, 17-types model.


To answer our first question, we define the training set $\mathcal{M}^{train} = \{M_1, \ldots, M_7\}$ and the test set $\mathcal{M}^{test} = \{M_8, \ldots, M_{14}\}$, as preregistered. We estimate the model on the training set and then compute the Euclidean distance between the predicted and observed joint choice distributions across all menus in the test set. Importantly, each model appears only in either the training or the test set, so out-of-sample predictive power reflects general choice procedures rather than simple repetition of choices for identical models. The training set allows identification of all 17 types, while the test set identifies a 16-dimensional subset.
We pool the Low and High Stakes conditions.


We find that the distance between the predicted and empirical distributions on the test set is 0.092. For comparison, the distance between the uniform and empirical distributions is 0.844. Thus, the full model reduces this distance to about one-tenth (0.108). This strong out-of-sample predictive performance indicates that the model captures stable behavioral tendencies.


\begin{figure}[htb]
\caption{Out-of-sample predictive power as function of the number of types \label{fig:outOfSample}}
\vspace{-0.8cm}
\begin{center}
\includegraphics[width=0.9\textwidth]{graphs/outOfSample.pdf}
\end{center}
\footnotesize
\textit{\textbf{Notes}}: This chart plots the Euclidian distance between the predicted and observed marginal distributions of choices on the test set, normalized by the distance between the uniform and observed distributions. The solid line averages only across bootstrap samples in which the listed typeset predicts best, that is samples $b$ with $S_k^b=S_k$. The dashed line averages the normalized distance across all bootstrap draws, including those for which the most predictive type set differs from $S_k$.
\end{figure}


To address the second question of whether a more parsimonious model can match the full model’s predictive power, we evaluate all possible subsets of the 17 types. We repeat the out-of-sample prediction procedure for each subset. Because random variation in the test set could inflate the apparent performance of specific type combinations, overfitting is a concern. To mitigate it, we apply a bootstrap approach. Instead of using the original sample, we run the procedure on each of 100 bootstrap resamples of subjects. For each $k \in {1, \ldots, 17}$ and each bootstrap sample $b$, we identify the subset $S_k^b$ among all $k$-element subsets that yields the highest predictive accuracy for that bootstrap sample. We then aggregate across bootstrap samples by selecting the $k$-element subset $S_k$ that performs best in the largest number of bootstrap draws.


We find that the sets of best-predicting types are nested when we include 8 or fewer types. For each $k$, the best-predicting $k$-element set $S_k$ consists of the best-predicting $(k-1)$-element set $S_{k-1}$ plus one additional type. Figure \ref{fig:outOfSample} shows the sequence of added types for each $k$ up to 8 and displays the distance between the predicted and empirical distributions, normalized by the distance between the uniform and empirical distributions on the test set.

We find that the largest gain in out-of-sample predictive power comes from including a single type in the type set. As our analysis in section \ref{sec:main} already suggests, this is the type that consistently selects the correct model. It alone reduces the distance between predicted and actual distributions by more than 85\%. The best two-type combination adds the type that exclusively follows the Direct Links criterion. Utility-Based criteria appear once we include three or more types. The first additions capture cautious choice, with the Minimize Investment criterion in the three-element set and the Worst-Case Promise criterion in the four-element set. The best three-element set rivals the predictive power of the full model, and the best four-element set exceeds it. Predictive accuracy continues to improve up to eight types and then declines.\footnote{Optimal model complexity depends on sample size, see, for instance, \citet{Ortolevaetal.CompetingModels}.} This decline is expected, as larger sets increase the risk of overfitting, which produces more variable and thus weaker out-of-sample predictions.


We conclude that a three- or four-type model consisting of a fully accurate type, a type that learns only from correlations corresponding to direct links in the models, and a type that maximizes worst-case outcomes without consulting the data captures most of the predictable variation in the data.


\FloatBarrier


\subsection{Utility-Focused Part}\label{sec:utilityBased}

We now examine how individuals choose absent access to the information required to apply Fact-Based criteria. This analysis provides insight into the Best-Case Promise criterion and thus speaks to the Narrative Competition literature.


\begin{figure}[H]
\caption{Utility-based choices}
\label{fig:utilityBased}
\label{fig:aggregateDistAmbiguity}
\begin{center}
\begin{tabular}{cc}
\textbf{A. Spread condition }
&
\textbf{B. Dominance condition }\\
\includegraphics[width=0.5\textwidth]{graphs/ambiguityFramesMainByStakes.pdf} &
\includegraphics[width=0.5\textwidth]{graphs/ambiguityFramesFOSDbyStakes.pdf} \\
\end{tabular}
\end{center}

\footnotesize
\textbf{\textit{Notes:}} Light bars represent the Low Stakes condition, dark bars the High Stakes condition. Whiskers represent 95\% confidence intervals with standard errors clustered by subject.
\end{figure}


Figure \ref{fig:utilityBased} summarizes subjects’ choices in the Utility-Focused Part. Panel A shows how often subjects choose the Best-Case Promise alternative in the Spread condition (light and dark bars indicate the Low and High Stakes conditions, respectively). When information is presented in the format natural to models fit to data---the State-Constant presentation---about one third of subjects choose the Best-Case Promise alternative, providing some empirical support for the Narrative Competition literature.\footnote{This share is sizable relative to findings in the literature on cautious choice and in the separate literature reviewed by \citet{vanDerWeeleetal.}, which rarely observes behavior consistent with anticipatory utility in laboratory settings.} However, Best-Case Promise choices fall below ten percent when payoffs are presented in the format typical of experiments on choice under uncertainty---the Action-Constant presentation.

Which presentation reflects subjects’ actual preferences? Panel B addresses this question by showing the frequency of choosing the dominant alternative in the Dominance condition. Subjects almost always choose the dominant option in the Action-Constant presentation, but about one quarter fail to do so in the State-Constant presentation. Hence, choices consistent with the Best-Case Promise criterion in the Spread condition largely reflect mistakes.

While the presentation mode greatly affects choices, a change from the Model Frame to the Gamble frame has no discernible effect.\footnote{Appendix \ref{app:part2} shows that all comparisons across presentation modes are highly statistically significant, that all comparisons across the Model and Gamble frames are insignificant at the 5\% level, and that stake-effects are significant in some conditions.  It also complements these aggregate statistics with individual-level analysis to show that the aggregates do not mask individual-level heterogeneity.} A threefold increase in stakes causes a minor decrease in Best-Case Promise choices. Assuming higher stakes increase effort and reduce confusion, this finding supports our interpretation that Best-Case Promise choices do not reflect preferences.

We conclude that the key assumption of the Narrative Competition literature describes a sizable minority of subjects when model implications are presented in the way natural to the setting, but these choices reflect a cognitive limitation rather than preferences.

\FloatBarrier


\begin{table}[htbp]
\caption{Effects of subjects' background  \label{tab:predictorsAccuracy}}
\begin{center}
\setlength{\tabcolsep}{3pt}
\resizebox{\textwidth}{!}{
\begin{tabular}{lccccccccccc}
 & (1) & (2) & (3) & (4) & (5)  \\
 \toprule
 & Part 1 &  \multicolumn{4}{c}{Part 2} \\
 \cmidrule(r){2-2} \cmidrule(r){3-6}
Dependent variable & \multicolumn{1}{c}{Chooses correct} & \multicolumn{2}{c}{Chooses }   & \multicolumn{2}{c}{Chooses } \\
 & \multicolumn{1}{c}{model} & \multicolumn{2}{c}{maxmax}  & \multicolumn{2}{c}{dominated}  \\
\cmidrule(r){2-2}\cmidrule(r){3-4}\cmidrule(r){5-6}
Treatment  & \multicolumn{1}{c}{} & \multicolumn{2}{c}{Spread}   & \multicolumn{2}{c}{Dominance} \\
Presentation mode \\
\quad State constant & \checkmark & \checkmark && \checkmark \\
\quad Action constant & & & \checkmark && \checkmark \\
\midrule
\vspace{-0.5cm}


                    &               &               &               &               &               \\

\textit{Preference measures (OR-IV)}\\ \quad Risk aversion perc. rank (0 to 1)&      -0.153** &       0.004   &      -0.230***&       0.176   &      -0.072*  \\
                    &     (0.057)   &     (0.106)   &     (0.061)   &     (0.098)   &     (0.034)   \\
\quad Ambiguity aversion perc. rank (0 to 1)&       0.078   &      -0.110   &      -0.118*  &      -0.034   &      -0.003   \\
                    &     (0.045)   &     (0.087)   &     (0.052)   &     (0.081)   &     (0.024)   \\
\textit{Demographics}\\ \quad Female&       0.030   &      -0.008   &       0.011   &       0.022   &      -0.006   \\
                    &     (0.021)   &     (0.040)   &     (0.022)   &     (0.037)   &     (0.013)   \\
\quad Political position (0 to 1) \\ \qquad Linear&       0.050   &       0.124   &      -0.156   &      -0.040   &      -0.117   \\
                    &     (0.120)   &     (0.220)   &     (0.142)   &     (0.219)   &     (0.078)   \\
\qquad Squared      &      -0.001   &      -0.001   &       0.003   &       0.002   &       0.002   \\
                    &     (0.002)   &     (0.003)   &     (0.002)   &     (0.003)   &     (0.001)   \\
\textit{Educational background} \\ \quad Knowledge index (0 to 1)&       0.130***&      -0.139   &      -0.009   &      -0.100   &      -0.026   \\
                    &     (0.037)   &     (0.071)   &     (0.045)   &     (0.067)   &     (0.024)   \\
\quad Field: STEM   &       0.066*  &       0.015   &      -0.029   &       0.009   &      -0.021   \\
                    &     (0.026)   &     (0.048)   &     (0.025)   &     (0.045)   &     (0.017)   \\
\quad Field: Econ. or business&       0.057   &      -0.058   &      -0.035   &      -0.018   &      -0.018   \\
                    &     (0.037)   &     (0.061)   &     (0.031)   &     (0.061)   &     (0.020)   \\
\textit{Psychological measures} \\ \quad CRT score (0 to 1)&       0.267***&      -0.444***&      -0.067   &      -0.304***&      -0.082*  \\
                    &     (0.045)   &     (0.082)   &     (0.048)   &     (0.084)   &     (0.039)   \\
\quad Pseudoscience score (0 to 1)&      -0.049   &       0.244*  &      -0.058   &       0.240*  &      -0.018   \\
                    &     (0.056)   &     (0.108)   &     (0.061)   &     (0.103)   &     (0.029)   \\
High stakes         &       0.004   &      -0.082*  &      -0.029   &      -0.011   &      -0.026*  \\
                    &     (0.018)   &     (0.036)   &     (0.021)   &     (0.033)   &     (0.012)   \\
\medskip
Observations        &       11228   &        3208   &        3208   &        3208   &        3208   \\
Subjects            &         401   &         401   &         401   &         401   &         401   \\
\bottomrule


\end{tabular}}
\end{center}
\footnotesize
\vspace{-0.8cm}
\textit{\textbf{Notes:}} Column 1 includes three design control dummies (for whether the correct model is associated with the high promise, with the minimal action, and the interaction of the two). Column 1 uses 14 choices for each subject (training and test set); Columns 2 to 5 use 4 choices each. The ORIV-stacked regression doubles these numbers. All regressions exclude 10 subjects who identify as neither male nor female. Knowledge index is the fraction of the following questions a subject can answer correctly: 1. Name of $P(A|B)$, 2. Complete ``Correlation does not...'' 3. Spell out `DAG'. Omitted category for gender is male. The omitted category for field of study is `other.' Political position is the position of the preferred political party according to \cite{chapelHillExpertSurvey}, with higher values indicating a more right-wing orientation. Pseudoscience score  \citep{Torresetal.} is higher the more an individual believes in pseudoscience.
Standard errors in parenthesis, clustered by subject.
* $p<0.1$, ** $p<0.05$, *** $p<0.01$.
\end{table}

\subsection{Individual characteristics }\label{sec:demographics}

Finally, we examine the effects of subjects’ background characteristics. Column 1 of Table \ref{tab:predictorsAccuracy} reports how these characteristics relate to the probability of choosing the correct model for a given menu in set $\mathcal{M}$. We use OR-IV regression \citep{ORIV} to correct for measurement error in risk and ambiguity preferences, each elicited twice. The specification includes controls for whether the correct model makes the higher or lower promise, whether it recommends the higher or lower investment, and the interaction between these variables. Because stakes have limited effects, we pool across stakes conditions and include an indicator for the High Stakes condition.\footnote{We preregistered complementing this analysis with a multinomial logit-type extension of our main estimates, assuming that the three most common types would receive substantial probability weight. However, the results in Sections \ref{sec:main} and \ref{sec:outOfSample} show a predominant influence of the All Correlations criterion and scattered use of other criteria. The low incidence of types other than those using the All Correlations criterion makes the multinomial logit specification unsuitable.}

Reassuringly, we find that greater background knowledge of statistics and causal inference positively predicts choice of the correct model ($p<0.01$ in both cases), as does studying a STEM field ($p<0.05$), and, although insignificantly so, studying economics or business. The effects are sizeable compared to the random choice benchmark of 50\%. Other variables have no explanatory power. In particular, and contrary to common expectations, we do not find an effect of political position or political extremity, measured by the square term.
This result appears less surprising when considering that both sides of the political spectrum tend to doubt their political opponents' capacity to make rational inferences from observations \citep[\emph{cf.} \emph{na\"ive realism};][]{GriffinRoss}.

Ex ante, the effects of risk and ambiguity preferences in our setting are ambiguous because of countervailing forces. The risk of selecting an incorrect model can be reduced by investing more effort in checking data charts, yet that effort is itself risky if it is uncertain whether a correct model will be found.\footnote{Risk and ambiguity aversion have no statistically significant effects on time spent per round. The most ambiguity-averse subjects view 1.25 more data charts than the least ambiguity-averse subjects ($p<0.05$), but risk aversion has no such effect.} Empirically, we find that more risk-averse individuals identify the correct model less often ($p<0.01$), while ambiguity aversion has no statistically significant effect.


Similar regressions for the Utility-Focused Part show that subjects with higher CRT scores ($p<0.05$) and those in the High Stakes condition ($p<0.1$) choose the Best-Case Promise alternative significantly less often, while subjects with stronger beliefs in pseudoscience choose it more often ($p<0.1$). These results align with our interpretation that Best-Case Promise choices in the Spread condition under the State-Constant presentation are unlikely to reflect preferences (column 2). Column 3 supports the interpretation that choices under the Action-Constant presentation do reflect preferences: correlations with cognitive ability measures disappear, while correlations with risk and ambiguity preferences emerge. Results from the Dominance condition (columns 4 and 5) are likewise consistent with this interpretation, indicating that choices in the Action-Constant but not in the State-Constant presentation reflect genuine preferences.


\FloatBarrier

\section{Conclusion}\label{sec:conclusion}

In this paper, we have experimentally studied how subjects learn and choose in decision problems for which they lack structural information, such as which variables are related and in what way.


We document two main results. First, subjects display a remarkable ability to discard misspecified models based on qualitative inference obtained by comparing models' correlational implications to the data. Second, when unable or unwilling to choose based on facts, subjects opt for cautious alternatives rather than for the Best-Case Promise alternatives assumed in the Narrative Competition literature. To the extent Best-Case Promise choices occur, they largely reflect a failure of contingent reasoning. That failure is induced by the natural way of presenting the implications of models fitted to data, and hence might also occur outside our laboratory setting. Our results are robust to a threefold increase in the magnitudes of the monetary stakes.

Future research should extend our work in several directions.
First, since we focused on settings where subjects can distinguish models only through observational data, we did not include Markov-equivalent models. Future work could examine choices between models that differ only in their interventional, but not observational, implications.
Second, empirical settings in which individuals learn from small samples merit attention, as uncertainty about the data becomes relevant. This is particularly pertinent to the literature on Model Persuasion that is driven by the mechanics of overfitting to small samples.


\addcontentsline{toc}{section}{References}


\ifx\undefined\bysame
\newcommand{\bysame}{\leavevmode\hbox to\leftmargin{\hrulefill\,\,}}
\fi
\begin{thebibliography}{xx}

\harvarditem[Aina and Schneider]{Aina and Schneider}{2025}{aina2025weighting}
{\bf Aina, Chiara and Florian Schneider}, ``Weighting competing models,'' 2025.

\harvarditem[Ambuehl and Bernheim]{Ambuehl and
  Bernheim}{{forthcoming}}{AmbuehlBernheim}
{\bf Ambuehl, Sandro and B~Douglas Bernheim}, ``Social Preferences over Ordinal
  Outcomes,'' {\it American Economic Review}, {forthcoming}.

\harvarditem[Andre et al.]{Andre, Haaland, Roth and
  Wohlfart}{2023}{Andreetal.MacroNarratives}
{\bf Andre, Peter, Ingar Haaland, Christopher Roth, and Johannes Wohlfart},
  ``Narratives about the Macroeconomy,'' 2023.

\harvarditem[Angrisani et al.]{Angrisani, Samek and
  Serrano-Padial}{2023}{Angrisanietal.}
{\bf Angrisani, Marco, Anya Samek, and Ricardo Serrano-Padial}, ``Competing
  Narratives in Action: An Empirical Analysis of Model Adoption Dynamics,''
  {\it Unpublished}, 2023.

\harvarditem[Angrisani et al.]{Angrisani, Cipriani and
  Guarino}{2022}{angrisani2022strategic}
{\bf \bysame{}, Marco Cipriani, and Antonio Guarino}, ``Strategic
  sophistication and trading profits: An experiment with professional
  traders,'' {\it Unpublished}, 2022.

\harvarditem[Barron and Fries]{Barron and Fries}{2023}{BarronFries}
{\bf Barron, Kai and Tilman Fries}, ``Narrative persuasion,'' {\it WZB
  Discussion Paper}, 2023.

\harvarditem[Benjamin]{Benjamin}{2019}{Benjamin.Review}
{\bf Benjamin, Daniel~J}, ``Errors in probabilistic reasoning and judgment
  biases,'' {\it Handbook of Behavioral Economics: Applications and Foundations
  1}, 2019, {\it 2}, 69--186.

\harvarditem[Dal~B{\'o} and Fr{\'e}chette]{Dal~B{\'o} and
  Fr{\'e}chette}{2011}{DalBo.Evolution}
{\bf B{\'o}, Pedro~Dal and Guillaume~R Fr{\'e}chette}, ``The evolution of
  cooperation in infinitely repeated games: Experimental evidence,'' {\it
  American Economic Review}, 2011, {\it 101} (1), 411--429.

\harvarditem[Dal~B{\'o} and Fr{\'e}chette]{Dal~B{\'o} and
  Fr{\'e}chette}{2019}{DalBoetal.StrategyChoice}
{\bf \bysame{} and \bysame{}}, ``Strategy choice in the infinitely repeated
  prisoner's dilemma,'' {\it American Economic Review}, 2019, {\it 109} (11),
  3929--3952.

\harvarditem[Camuffo et al.]{Camuffo, Gambardella and
  Pignataro}{2023}{Camuffoetal.}
{\bf Camuffo, Arnaldo, Alfonso Gambardella, and Andrea Pignataro},
  ``Theory-driven strategic management decisions,'' {\it CEPR DP 17664v2},
  2023.

\harvarditem[Caplin et al.]{Caplin, Deming, Leth-Petersen and
  Weidmann}{2023}{Caplin2023}
{\bf Caplin, Andrew, David~J. Deming, S{\o}ren Leth-Petersen, and Ben
  Weidmann}, ``Economic Decision-Making Skill Predicts Income in Two
  Countries,'' {\it NBER Working Paper}, 2023, (31674).

\harvarditem[Costa-Gomes and Crawford]{Costa-Gomes and
  Crawford}{2006}{CostaGomesCrawford}
{\bf Costa-Gomes, Miguel~A and Vincent~P Crawford}, ``Cognition and behavior in
  two-person guessing games: An experimental study,'' {\it American Economic
  Review}, 2006, {\it 96} (5), 1737--1768.

\harvarditem[Coutts et al.]{Coutts, Drobner, Koh and
  Woolnough}{2025}{Coutts2025}
{\bf Coutts, Alexander, Christoph Drobner, Boon~Han Koh, and Chris Woolnough},
  ``High Stakes, More Mistakes? Belief Elicitation and Incentives,'' January
  2025.
\newblock Working paper.

\harvarditem[Denzau and North]{Denzau and North}{1994}{denzauNorth}
{\bf Denzau, Arthur~T and Douglass~C North}, ``Shared mental models: ideologies
  and institutions,'' {\it Kyklos}, 1994, {\it 47}, 3--31.

\harvarditem[Dimmock et al.]{Dimmock, Kouwenberg, Mitchell and
  Peijnenburg}{2015}{Dimmocketal.}
{\bf Dimmock, Stephen~G, Roy Kouwenberg, Olivia~S Mitchell, and Kim
  Peijnenburg}, ``Estimating ambiguity preferences and perceptions in multiple
  prior models: Evidence from the field,'' {\it Journal of Risk and
  Uncertainty}, 2015, {\it 51}, 219--244.

\harvarditem[Eckel and Grossman]{Eckel and Grossman}{2008}{EckelGrossman}
{\bf Eckel, Catherine~C and Philip~J Grossman}, ``Men, women and risk aversion:
  Experimental evidence,'' {\it Handbook of experimental economics results},
  2008, {\it 1}, 1061--1073.

\harvarditem[Eliaz and Spiegler]{Eliaz and
  Spiegler}{2020}{EliazSpiegler.Narratives}
{\bf Eliaz, Kfir and Ran Spiegler}, ``A model of competing narratives,'' {\it
  American Economic Review}, 2020, {\it 110} (12), 3786--3816.

\harvarditem[Eliaz et al.]{Eliaz, Galperti and
  Spiegler}{2022}{EliazGalpertiSpiegler}
{\bf \bysame{}, Simone Galperti, and Ran Spiegler}, ``False Narratives and
  Political Mobilization,'' {\it arXiv preprint arXiv:2206.12621}, 2022.

\harvarditem[Engelmann et al.]{Engelmann, Lebreton, Schwardmann, van~der Weele
  and Chang}{forthcoming}{vanDerWeeleetal.}
{\bf Engelmann, Jan, Ma\"el Lebreton, Peter Schwardmann, Joel~J van~der Weele,
  and Li-Ang Chang}, ``Anticipatory anxiety and wishful thinking,'' {\it
  American Economic Review}, forthcoming.

\harvarditem[Felin and Zenger]{Felin and Zenger}{2017}{FelinZenger}
{\bf Felin, Teppo and Todd~R Zenger}, ``The theory-based view: Economic actors
  as theorists,'' {\it Strategy Science}, 2017, {\it 2} (4), 258--271.

\harvarditem[Finucane and Gullion]{Finucane and
  Gullion}{2010}{finucane2010developing}
{\bf Finucane, Melissa~L and Christina~M Gullion}, ``Developing a tool for
  measuring the decision-making competence of older adults.,'' {\it Psychology
  and aging}, 2010, {\it 25} (2), 271.

\harvarditem[Fox and {\"U}lk{\"u}men]{Fox and
  {\"U}lk{\"u}men}{2011}{FoxUlkumen}
{\bf Fox, Craig~R and G{\"u}lden {\"U}lk{\"u}men}, ``Distinguishing two
  dimensions of uncertainty,'' in W.~Brun, G.~Kirkeb{\o}en, and H.~Montgomery,
  eds., {\it Essays in Judgment and Decision Making}, Oslo:
  Universitetsforlaget, 2011.

\harvarditem[Frechette et al.]{Frechette, Vespa and
  Yuksel}{2023}{Frechetteetal.}
{\bf Frechette, Guillaume, Emanuel Vespa, and Sevgi Yuksel}, ``Extracting
  Models From Data Sets: An Experiment Using Notes-to-Self,'' {\it
  Unpublished}, 2023.

\harvarditem[Frederick]{Frederick}{2005}{CRT}
{\bf Frederick, Shane}, ``Cognitive reflection and decision making,'' {\it
  Journal of Economic Perspectives}, 2005, {\it 19} (4), 25--42.

\harvarditem[Gilboa]{Gilboa}{2025}{Gilboa.Review}
{\bf Gilboa, Itzhak}, ``Decision under uncertainty: State of the science,''
  {\it Annual Review of Economics}, 2025, {\it 17.}

\harvarditem[Gillen et al.]{Gillen, Snowberg and Yariv}{2019}{ORIV}
{\bf Gillen, Ben, Erik Snowberg, and Leeat Yariv}, ``Experimenting with
  measurement error: Techniques with applications to the caltech cohort
  study,'' {\it Journal of Political Economy}, 2019, {\it 127} (4), 1826--1863.

\harvarditem[Glaser et al.]{Glaser, Iliewa and Weber}{2019}{glaser2019thinking}
{\bf Glaser, Markus, Zwetelina Iliewa, and Martin Weber}, ``Thinking about
  prices versus thinking about returns in financial markets,'' {\it The Journal
  of Finance}, 2019, {\it 74} (6), 2997--3039.

\harvarditem[Griffin and Ross]{Griffin and Ross}{1991}{GriffinRoss}
{\bf Griffin, Dale~W and Lee Ross}, ``Subjective construal, social inference,
  and human misunderstanding,'' in ``Advances in experimental social
  psychology,'' Vol.~24, Elsevier, 1991, pp.~319--359.

\harvarditem[Griffiths et al.]{Griffiths, Chater and
  Tenenbaum}{2024}{BayesianModelsOfCognitionBook}
{\bf Griffiths, Thomas~L., Nick Chater, and Joshua Tenenbaum}, {\it Bayesian
  Models of Cognition: Reverse Engineering the Mind}, MIT Press, 2024.

\harvarditem[Hanaki et al.]{Hanaki, Inukai, Masuda and
  Shimodaira}{2021}{Hanaki2021}
{\bf Hanaki, Nobuyuki, Keigo Inukai, Takehito Masuda, and Yuta Shimodaira},
  ``Participants' Characteristics at ISER-Lab in 2020,'' ISER Discussion Paper
  1141, Osaka University, Institute of Social and Economic Research (ISER)
  September 2021.

\harvarditem[Jolly et al.]{Jolly, Bakker, Hooghe, Marks, Polk, Rovny,
  Steenbergen and Vachudova}{2022}{chapelHillExpertSurvey}
{\bf Jolly, Seth, Ryan Bakker, Liesbet Hooghe, Gary Marks, Jonathan Polk, Jan
  Rovny, Marco Steenbergen, and Milada~Anna Vachudova}, ``Chapel Hill expert
  survey trend file, 1999--2019,'' {\it Electoral studies}, 2022, {\it 75},
  102420.

\harvarditem[Kendall and Charles]{Kendall and Charles}{2022}{CharlesKendall}
{\bf Kendall, Chad~W and Constantin Charles}, ``Causal Narratives,'' {\it
  Unpublished}, 2022.

\harvarditem[Klusowski et al.]{Klusowski, Small and
  Simmons}{2021}{Klusowskietal.}
{\bf Klusowski, Joowon, Deborah~A Small, and Joseph~P Simmons}, ``Does choice
  cause an illusion of control?,'' {\it Psychological Science}, 2021, {\it 32}
  (2), 159--172.

\harvarditem[Levy et al.]{Levy, Razin and Young}{2022}{Levyetal.}
{\bf Levy, Gilat, Ronny Razin, and Alwyn Young}, ``Misspecified politics and
  the recurrence of populism,'' {\it American Economic Review}, 2022, {\it 112}
  (3), 928--962.

\harvarditem[Molavi]{Molavi}{2019}{Molavi2019}
{\bf Molavi, Pooya}, ``Macroeconomics with Learning and Misspecification: A
  General Theory and Applications,'' 2019.

\harvarditem[Molavi et al.]{Molavi, Tahbaz-Salehi and
  Vedolin}{2021}{Molavietal.}
{\bf \bysame{}, Alireza Tahbaz-Salehi, and Andrea Vedolin}, ``Model Complexity,
  Expectations, and Asset Prices,'' {\it NBER working paper}, 2021.

\harvarditem[Niederle and Vespa]{Niederle and
  Vespa}{2023}{NiederleVespa.AnnualReviews}
{\bf Niederle, Muriel and Emanuel Vespa}, ``Cognitive Limitations: Failures of
  Contingent Thinking,'' {\it Annual Review of Economics}, 2023, {\it 15},
  307--328.

\harvarditem[Montiel~Olea et al.]{Montiel~Olea, Ortoleva, Pai and
  Prat}{2022}{Ortolevaetal.CompetingModels}
{\bf Olea, Jos{\'e}~Luis~Montiel, Pietro Ortoleva, Mallesh~M Pai, and Andrea
  Prat}, ``Competing models,'' {\it The Quarterly Journal of Economics}, 2022,
  {\it 137} (4), 2419--2457.

\harvarditem[Schumacher and Thysen]{Schumacher and
  Thysen}{2022}{SchumacherThysen}
{\bf Schumacher, Heiner and Heidi~Christina Thysen}, ``Equilibrium contracts
  and boundedly rational expectations,'' {\it Theoretical Economics}, 2022,
  {\it 17} (1), 371--414.

\harvarditem[Schwartzstein and Sunderam]{Schwartzstein and
  Sunderam}{2021}{SchwartzsteinSunderam}
{\bf Schwartzstein, Joshua and Adi Sunderam}, ``Using models to persuade,''
  {\it American Economic Review}, 2021, {\it 111} (1), 276--323.

\harvarditem[Shiller]{Shiller}{2017}{Shiller.Narratives}
{\bf Shiller, Robert~J}, ``Narrative economics,'' {\it American Economic
  Review}, 2017, {\it 107} (4), 967--1004.

\harvarditem[Sloman]{Sloman}{2005}{SlomanBook}
{\bf Sloman, Steven}, {\it Causal models: How people think about the world and
  its alternatives}, Oxford University Press, 2005.

\harvarditem[Spiegler]{Spiegler}{2016}{Spiegler.QJE}
{\bf Spiegler, Ran}, ``Bayesian networks and boundedly rational expectations,''
  {\it The Quarterly Journal of Economics}, 2016, {\it 131} (3), 1243--1290.

\harvarditem[Spirtes et al.]{Spirtes, Glymour and
  Scheines}{2000}{spirtes2000causation}
{\bf Spirtes, Peter, Clark~N Glymour, and Richard Scheines}, {\it Causation,
  prediction, and search}, MIT press, 2000.

\harvarditem[Stefan and David]{Stefan and David}{2013}{StefanDavid}
{\bf Stefan, Simona and Daniel David}, ``Recent developments in the
  experimental investigation of the illusion of control. A meta-analytic
  review,'' {\it Journal of Applied Social Psychology}, 2013, {\it 43} (2),
  377--386.

\harvarditem[Steyvers et al.]{Steyvers, Tenenbaum, Wagenmakers and
  Blum}{2003}{Steyversetal.}
{\bf Steyvers, Mark, Joshua~B Tenenbaum, Eric-Jan Wagenmakers, and Ben Blum},
  ``Inferring causal networks from observations and interventions,'' {\it
  Cognitive science}, 2003, {\it 27} (3), 453--489.

\harvarditem[Thoma et al.]{Thoma, White, Panigrahi, Strowger and
  Anderson}{2015}{thoma2015good}
{\bf Thoma, Volker, Elliott White, Asha Panigrahi, Vanessa Strowger, and Irina
  Anderson}, ``Good thinking or gut feeling? Cognitive reflection and intuition
  in traders, bankers and financial non-experts,'' {\it PloS one}, 2015, {\it
  10} (4), e0123202.

\harvarditem[Thomson and Oppenheimer]{Thomson and Oppenheimer}{2016}{CRT3}
{\bf Thomson, Keela~S and Daniel~M Oppenheimer}, ``Investigating an alternate
  form of the cognitive reflection test,'' {\it Judgment and Decision making},
  2016, {\it 11} (1), 99--113.

\harvarditem[Toplak et al.]{Toplak, West and Stanovich}{2014}{CRT2}
{\bf Toplak, Maggie~E, Richard~F West, and Keith~E Stanovich}, ``Assessing
  miserly information processing: An expansion of the Cognitive Reflection
  Test,'' {\it Thinking \& Reasoning}, 2014, {\it 20} (2), 147--168.

\harvarditem[Torres et al.]{Torres, Barberia and
  Rodr{\'\i}guez-Ferreiro}{2020}{Torresetal.}
{\bf Torres, Marta~N, Itxaso Barberia, and Javier Rodr{\'\i}guez-Ferreiro},
  ``Causal illusion as a cognitive basis of pseudoscientific beliefs,'' {\it
  British Journal of Psychology}, 2020, {\it 111} (4), 840--852.

\harvarditem[Trautmann and Van De~Kuilen]{Trautmann and Van
  De~Kuilen}{2015}{ambiguityReview}
{\bf Trautmann, Stefan~T and Gijs Van~De Kuilen}, ``Ambiguity attitudes,'' {\it
  The Wiley Blackwell handbook of judgment and decision making}, 2015, {\it 2},
  89--116.

\harvarditem[Verma and Pearl]{Verma and Pearl}{1990}{VermaPearl1990}
{\bf Verma, Thomas~S. and Judea Pearl}, ``Equivalence and Synthesis of Causal
  Models,'' in ``Proceedings of the Sixth Conference on Uncertainty in
  Artificial Intelligence'' Association for Uncertainty in Artificial
  Intelligence Mountain View, CA 1990, pp.~220--227.

\harvarditem[Waldmann]{Waldmann}{2017}{OxfordHandbook}
{\bf Waldmann, Michael}, {\it The Oxford handbook of causal reasoning}, Oxford
  University Press, 2017.

\harvarditem[Weitzel et al.]{Weitzel, Huber, Huber, Kirchler, Lindner and
  Rose}{2020}{weitzel2020bubbles}
{\bf Weitzel, Utz, Christoph Huber, J{\"u}rgen Huber, Michael Kirchler, Florian
  Lindner, and Julia Rose}, ``Bubbles and financial professionals,'' {\it The
  Review of Financial Studies}, 2020, {\it 33} (6), 2659--2696.

\harvarditem[Welsh and Begg]{Welsh and Begg}{2017}{welsh2017cognitive}
{\bf Welsh, Matthew~Brian and Steve~H Begg}, ``The Cognitive Reflection Test:
  familiarity and predictive power in professionals,'' in ``Proceedings of the
  Annual Meeting of the Cognitive Science Society,'' Vol.~39 2017.

\end{thebibliography}


\newpage
\begin{appendix}

\renewcommand\thetable{\thesection.\arabic{table}}
\renewcommand\thefigure{\thesection.\arabic{figure}}
\setcounter{page}{0}

\begin{center}
{\Huge \textbf{ONLINE APPENDIX}}\\

{\Large   \textbf{\LARGE \vspace*{2cm} \papertitle}  \\  \bigskip \bigskip Sandro Ambuehl, Heidi C. Thysen}
\end{center}


\pagenumbering{gobble}
\setstretch{1.2}
\addcontentsline{toc}{section}{Appendix}
\part{}
\parttoc
\setstretch{1.4}

\newpage
\pagenumbering{arabic}


\setcounter{table}{0}
\setcounter{figure}{0}


\section{Theory}\label{app: theory}


\subsection{Characteristic correlational implications of selected DAGs} \label{app: DAG implications}


Using the notation and language of \citeappendix{Pearl.2009}, the formal statement of Observation \ref{obs: DAG implications} in the case of an arbitrary number of nodes can be stated as follows. This observation follows directly from Theorem 5.2.1 in \citeappendix{Pearl.2009}.

\begin{observation}
Consider a DAG $G = (N,E)$, with $I, J, K \in N$ and $Z\subset N$.
\begin{enumerate}
\item If $I\rightarrow J$, then generically $\cov(I,J)\neq 0$.
\item
\begin{enumerate}
    \item If $Z=\emptyset$ does not $d$-separate $I$ and $J$, then generically $\cov(I,J)\neq 0$.
    \item If $Z=\emptyset$ $d$-separates $I$ and $J$, then $\cov(I,J)=0$.
\end{enumerate}
\item
\begin{enumerate}
    \item If $Z=\{K\}$ $d$-separates $I$ and $J$, then $\cov(I,J|K)=0$.
    \item If $Z=\{K\}$ does not $d$-separate $I$ and $J$, then generically $\cov(I,J|K)\neq 0$.
\end{enumerate}
\end{enumerate}
\end{observation}


\subsection{Tools to construct menus that identify criteria distributions}\label{app:theoryMain}

Here, we first construct the action-equivalence classes such that given any DGP, any two DAGs within the same equivalence class recommend the same investment (Subsection \ref{app: equivalence classes}). Second, we show that for any pair of models we can vary independently whether the model recommending the higher investment also makes the higher or lower promise by adjusting the distribution of investments in the simulated empirical data used for model fitting (Subsection \ref{app: highest promise}). Third, we show that in menus of two linear models, the Worst-Case Promise criterion is equivalent to selecting the model that promises the \emph{lower} payoff if correct (Subsection \ref{app:maxmin}). Throughout we use the notation $A=\sqrt{I}$ (recall that our systems are linear in the square root of $I$), $X=C_1$, and $Z=C_2$.


Throughout, we will use the following definition.
\begin{definition}
Consider a DAG $G=(N,E)$.
\begin{enumerate}
\item For $I\in N$, $G(I) = \{J\in N\mid (J,I)\in E\}$ is the set of \textup{Parents} of node $I$.
\item For $I,J,K\in N$ we call the triple $(I,J,K)$ a \textup{$v$-collider} if $I,J\in G(K)$, but $I\not\in G(J)$, and $J\not\in G(I)$.
\end{enumerate}
\end{definition}


Furthermore, we let $V_G(a) = \mathbb{E}_G[Y\mid A = a] - \frac{c}{2}a^2$ denote the expected payoff according to $G$ when action $a$ is implemented.

Throughout, we will make use of the following observation.

\begin{observation}\label{app: action recommendation}
If a linear system of equations with DAG representation $G$ and $G(A)=\emptyset$ is fitted to the data using ordinary least squares, the mean of $Y$ conditional on $A$ is given by

\begin{align*}
\mathbb{E}_G[Y|A] = \hat{\alpha}^G + \hat{\alpha}^G_A A.
\end{align*}
\end{observation}
When $G$ is consistent with the DGP we let $\alpha^*$, and $\alpha^*_A$ denote the intercept and slope coefficients. The optimal action recommendation associated with DAG $G$ is the solution to the maximization problem
$
a^G = \arg\max_{a} \hat{\alpha}^G + \hat{\alpha}^G_A a - \frac{c}{2}a^2.
$
Hence, DAG $G$ recommends the action $a^G = \frac{\hat{\alpha}^G}{c}$.

\subsubsection{Action equivalence classes}\label{app: equivalence classes}

We now define the 15 action-equivalence classes. We characterize the set of DAGs in each equivalence class and report the estimated effect of the action on the bonus for the DAGs in that class. Note that this partition is different from Markov equivalence classes. We make use of the well-known results that for any two variables $I$ and $J$, the estimated slope coefficient of the regression of $J$ on $I$ is given by $\frac{\cov(J,I)}{\var(I)}$, and that in a regression of a variable $J$ on variables $I$ and $K$ the slope coefficient on $I$ is given by $\frac{\cov(I,J)\var(K)-\cov(I,K)\cov(K,J)}{\var(I)\var(K)-\cov(I,K)^2}$. Throughout, variance and covariance operators refer to the DGP.


\medskip
\textbf{Class 1} consists of all DAGs that posit no direct or indirect effect of $A$ on $Y$. Hence, regardless of the DGP, $\hat{\alpha}^G =0$ for all $G$ in this class.

\medskip
\textbf{Class 2} consists of all DAGs with $A\in G(Y)$, and there is no $I\in N$ such that $(A,I,Y)$ is a $v$-collider. That is, $A$ has a direct influence on $Y$, and no other variable has a direct influence on $Y$. While some of the system of linear regressions represented by a DAG in this class might calculate the total effect of $A$ on $Y$ as the sum of the direct effect of $A$ on $Y$ and the indirect effect of $A$ on $Y$ through one or more of the covariates, the total predicted effect of $A$ on $Y$ is the same. This follows directly from Proposition 2 in \citeappendix{Spiegler.Review}. For any DAG $G$ in this class,
\bmathnn
\hat{\alpha}^G = \frac{\cov(A,Y)}{\var(A)}.
\emathnn

\medskip
\textbf{Class 3} consists of all DAGs with $G(X)= \{A\}$ and $G(Y)=\{X\}$. That is, $A$ does not have an indirect influence on $Y$, but a direct influence on $Y$ through $X$, and $Z$ does not (directly or indirectly) influence $Y$. For any DAG $G$ in this class,
\bmathnn
\hat{\alpha}^G = \frac{\cov(A,X)}{\var(A)}\frac{\cov(X,Y)}{\var(X)}.
\emathnn

\medskip
\textbf{Class 4} consists of all DAGs with $G(Z)= \{A\}$ and $G(Y)=\{Z\}$. This class parallels Class 3 with the positions of $X$ and $Z$ switched. For any DAG $G$ in this class,  \bmathnn
\hat{\alpha}^G = \frac{\cov(A,Z)}{\var(A)}\frac{\cov(Z,Y)}{\var(Z)}.
\emathnn

\medskip
\textbf{Class 5} consists of the single DAG $G: A\rightarrow X \rightarrow Z \rightarrow Y$. We have
\bmathnn
\hat{\alpha}^G = \frac{\cov(A,X)}{\var(A)}\frac{\cov(X,Z)}{\var(X)}\frac{\cov(Z,Y)}{\var(Z)}.
\emathnn

\medskip
\textbf{Class 6} consists of the single DAG $G: A\rightarrow Z \rightarrow X \rightarrow Y$. It parallels Class 5 with the positions of $X$ and $Z$ switched. We have
\bmathnn
\hat{\alpha}^G = \frac{\cov(A,Z)}{\var(A)}\frac{\cov(Z,X)}{\var(Z)}\frac{\cov(X,Y)}{\var(X)}.
\emathnn

\medskip
\textbf{Class 7} consists of all DAGs that contain the $v$-collider $(A,X,Y)$ and no other $v$-colliders. For any DAG $G$ in this class,
\bmathnn
\hat{\alpha}^G = \frac{\cov(A,Y)\var(X)-\cov(A,X)\cov(X,Y)}{\var(A)\var(X)-\cov(A,X)^2}.
\emathnn

\medskip
\textbf{Class 8}  consists of all DAGs that contain the $v$-collider $(A,Z,Y)$ and no other $v$-colliders. It parallels Class 7 with the positions of $X$ and $Z$ switched. For any DAG $G$ in this class,
\bmathnn
\hat{\alpha}^G = \frac{\cov(A,Y)\var(Z)-\cov(A,Z)\cov(Z,Y)}{\var(A)\var(Z)-\cov(A,Z)^2}.
\emathnn

\medskip
\textbf{Class 9} consists of all DAGs for which $G(Y)=\{A,X,Z\}$ and $A\not\in G(X),G(Z)$. For any DAG $G$ in this class,
{\tiny
\bmathnn
\hat{\alpha}^G &=& \frac{\cov(AY)(\var(X)\var(Z)-\cov(X,Z)^2)}{\var(A)\var(X)\var(Z) + 2\cov(A,X)\cov(A,Z)\cov(X,Z)-\cov(X,Z)^2\var(A)-\cov(A,X)^2\var(Z)-\cov(A,Z)^2\var(X)}\\
&&-\frac{\cov(A,X)(\cov(X,Y)\var(Z) - \cov(X,Z)\cov(Z,Y)) + \cov(A,Z)(\cov(Z,Y)\var(X)-\cov(X,Z)\cov(X,Y))}{\var(A)\var(X)\var(Z) + 2\cov(A,X)\cov(A,Z)\cov(X,Z)-\cov(X,Z)^2\var(A)-\cov(A,X)^2\var(Z)-\cov(A,Z)^2\var(X)},
\emathnn
}
which is the slope coefficient on $A$ in the regression of $Y$ on the three regressors $A, X, Z$.

\medskip
\textbf{Class 10} consists of the single DAG $G: A\rightarrow X \rightarrow Y \leftarrow Z$. We have
\bmathnn
\hat{\alpha}^G = \frac{\cov(A,X)}{\var(A)}\frac{\cov(X,Y)\var(Z)-\cov(X,Z)\cov(Z,Y)}{\var(X)\var(Z)-\cov(X,Z)^2}.
\emathnn

\medskip
\textbf{Class 11} consists of the single DAG $G: A\rightarrow Z \rightarrow Y \leftarrow X$. It parallels Class 10 with the positions of $X$ and $Z$ switched. We have
\bmathnn
\hat{\alpha}^G = \frac{\cov(A,Z)}{\var(A)}\frac{\cov(Z,Y)\var(X)-\cov(X,Z)\cov(X,Y)}{\var(X)\var(Z)-\cov(X,Z)^2}.
\emathnn

\medskip
\textbf{Class 12} consists of the single DAG {\scriptsize
\adjustbox{valign=t}{\begin{tikzpicture}[node distance={7mm}, main/.style = {}, scale=0.8, every node/.style={transform shape}]  \node[main] (1) {$A$};
  \node[main] (2) [below of=1] {$X$};  \node[main] (4) [right of=1] {$Z$};  \node[main] (3) [below of=4] {$Y$};  \draw[->] (1) -- (2);
  \draw[->] (2) -- (3);
  \draw[->] (4) -- (2);
\end{tikzpicture}}

}.

We have
\bmathnn
\hat{\alpha}^G = \frac{\cov(A,Z)\var(X)-\cov(A,Z)\cov(X,Z)}{\var(A)\var(Z)-\cov(A,Z)^2}\frac{\cov(X,Y)}{\var(X)}.
\emathnn

\medskip
\textbf{Class 13} consists of the single DAG {\scriptsize
\adjustbox{valign=t}{\begin{tikzpicture}[node distance={7mm}, main/.style = {}, scale=0.8, every node/.style={transform shape}]  \node[main] (1) {$A$};
  \node[main] (2) [below of=1] {$X$};  \node[main] (4) [right of=1] {$Z$};  \node[main] (3) [below of=4] {$Y$};  \draw[->] (1) -- (4);
  \draw[->] (4) -- (3);
  \draw[->] (2) -- (4);
\end{tikzpicture}}

}.

We have
\bmathnn
\hat{\alpha}^G = \frac{\cov(A,X)\var(Z)-\cov(A,X)\cov(X,Z)}{\var(A)\var(X)-\cov(A,X)^2}\frac{\cov(Z,Y)}{\var(Z)}.
\emathnn

\medskip
\textbf{Class 14} consists of the single DAG {\scriptsize
\adjustbox{valign=t}{\begin{tikzpicture}[node distance={7mm}, main/.style = {}, scale=0.8, every node/.style={transform shape}]  \node[main] (1) {$A$};
  \node[main] (2) [below of=1] {$X$};  \node[main] (4) [right of=1] {$Z$};  \node[main] (3) [below of=4] {$Y$};  \draw[->] (1) -- (2);
  \draw[->] (2) -- (3);
  \draw[->] (4) -- (2);
  \draw[->] (4) -- (3);
\end{tikzpicture}}

}.
We have
{\small
\bmathnn
\hat{\alpha}^G = \frac{\cov(A,Z)\var(X)-\cov(A,Z)\cov(X,Z)}{\var(A)\var(Z)-\cov(A,Z)^2} \frac{\cov(X,Y)\var(Z)-\cov(X,Z)\cov(Z,Y)}{\var(X)\var(Z)-\cov(X,Z)^2}.
\emathnn
}

\medskip
\textbf{Class 15} consists of the single DAG {\scriptsize
\adjustbox{valign=t}{\begin{tikzpicture}[node distance={7mm}, main/.style = {}, scale=0.8, every node/.style={transform shape}]  \node[main] (1) {$A$};
  \node[main] (2) [below of=1] {$X$};  \node[main] (4) [right of=1] {$Z$};  \node[main] (3) [below of=4] {$Y$};  \draw[->] (1) -- (4);
  \draw[->] (4) -- (3);
  \draw[->] (2) -- (4);
  \draw[->] (2) -- (3);
\end{tikzpicture}}

}.
We have
{\small
\bmathnn
\hat{\alpha}^G = \frac{\cov(A,X)\var(Z)-\cov(A,X)\cov(X,Z)}{\var(A)\var(X)-\cov(A,X)^2} \frac{\cov(Z,Y)\var(X)-\cov(X,Z)\cov(X,Y)}{\var(X)\var(Z)-\cov(X,Z)^2}.
\emathnn
}
This is a comprehensive list of all DAGs we consider for the experiment. We exclude the DAG
{\scriptsize
\adjustbox{valign=t}{\begin{tikzpicture}[node distance={7mm}, main/.style = {}, scale=0.8, every node/.style={transform shape}]  \node[main] (1) {$A$};
  \node[main] (2) [below of=1] {$X$};  \node[main] (4) [right of=1] {$Z$};  \node[main] (3) [below of=4] {$Y$};  \draw[->] (1) -- (2);
  \draw[->] (2) -- (3);
  \draw[->] (1) -- (4);
  \draw[->] (4) -- (3);
\end{tikzpicture}}

}

because one of its characteristic independence relationships, $A\indep Y | (X, Z)$, involves conditioning on two variables simultaneously, which is not information that we provide to subjects.


\subsubsection{Pairwise comparison of promises}\label{app: highest promise}\label{app: correct prediction}


We next demonstrate how to select the mean of the action in the DGP to change which of two given models yields the higher promise. To do so, we will make use of lemma  \ref{lem: correct mean}. An immediate implication of this lemma is that if the action is set to its mean in the data, then  any model predicts the same mean outcome. This observation will prove useful to compare the predicted payoffs of the recommended actions across models.


\begin{lemma}\label{lem: correct mean}
Consider a system of linear equations represented by the DAG $G = (N,E)$, where $G(A)=\emptyset$. For every DGP and $I\in N$, we have
\begin{align*}
\mathbb{E}[\mathbb{E}_G[I\mid A]] = \mathbb{E}[I].
\end{align*}
\end{lemma}

\begin{proof} We prove this statement by induction.
To anchor the induction, consider any variable $I\in N$ for which $G(I) = \emptyset$. If $I=A$, this holds trivially. If $I\neq A$, then $\mathbb{E}_G[I\mid A] = \mathbb{E}[I]$, since $G$ treats $I$ and $A$ as exogenous variables, and therefore as independent. Hence, $\mathbb{E}[\mathbb{E}_G[I\mid A]] = \mathbb{E}[I]$.

Next, consider any node $J$ and suppose that the induction hypothesis $\mathbb{E}[\mathbb{E}_G[I\mid A]] = \mathbb{E}[I]$ holds for every $I\in G(J)$. Let $\hat{\beta}_{IJ}$ denote the slope coefficient on variable $I$ in the OLS regression of $J$ on all its parents. Then, the constant term in that regression, $\hat{\beta}_J$ is given by
\begin{align}\label{eq2}
\hat{\beta}_J = \mathbb{E}[J] - \sum_{I\in G(J)} \hat{\beta}_{IJ}\mathbb{E}[I].
\end{align}
Furthermore, applying the conditional expectation operator $E_G[\cdot|A]$ to the regression equation that defines $J$ according to $G$ yields
\begin{align}\label{eq3}
\mathbb{E}_G[J\mid A] = \hat{\beta}_J + \sum_{I\in G(J)} \hat{\beta}_{IJ} \mathbb{E}_G[I\mid A].
\end{align}
Substituting \ref{eq2} into equation \ref{eq3}, and taking the expectation over the action, we obtain:
\begin{align*}
\mathbb{E}[\mathbb{E}_G[J\mid A]] = \mathbb{E}[J] + \sum_{I\in G(J)} \hat{\beta}_{IJ} (\mathbb{E}[\mathbb{E}_G[I\mid A]] - \mathbb{E}[I])
\end{align*}
By the induction hypothesis, the term in parentheses is zero. Hence, $\mathbb{E}[\mathbb{E}_G[J\mid A]] = \mathbb{E}[J]$ as was to be shown.


\end{proof}


We can now state the key result, Proposition \ref{lem: highest promise}, that the model with the lower action recommendation make the higher promise if and only if the mean action exceeds some threshold.

\begin{proposition}\label{lem: highest promise}
Consider two models, $G$ and $G'$, where $G(A) = G'(A)=\emptyset$. Let $a_G$ and $a_{G'}$ denote the corresponding action recommendations, and suppose $a_G > (<) a_{G'}$. Then $V_G(a^G) \geq V_{G'}(a^{G'})$ if and only if $\mathbb{E}[A] \leq (\geq) \frac{a^G + a^{G'}}{2}$.
\end{proposition}

\begin{proof}
First, recall that for every DAG, $G$, we can write the predicted conditional mean of the bonus as a linear function of the action, $a$, specifically, $\mathbb{E}_G[Y\mid A=a] = \hat{\alpha}^G + \hat{\alpha}_A^G a.$ By Lemma \ref{lem: correct mean}, we thus have
\begin{align*}
\mathbb{E}[Y]  &= \mathbb{E}[\mathbb{E}_G[Y\mid A]] = \hat{\alpha}^G + \hat{\alpha}_A^G \mathbb{E}[A] \\
 \mathbb{E}[Y] & = \mathbb{E}[\mathbb{E}[Y\mid A]] = \alpha^* + \alpha^*_A \mathbb{E}[A].
\end{align*}

Combining the two equations and solving for $\hat{\alpha}^G$ yields:
\begin{align}\label{eq: mean outcome}
\hat{\alpha}_G = \alpha^* + (\alpha^*_A - \hat{\alpha}^G_A)\mathbb{E}[A].
\end{align}

We use (\ref{eq: mean outcome}) to express the model's promise as a function of $\mathbb{E}[A]$:
\begin{align*}
V_G(a^G) &= \hat{\alpha}^G + \hat{\alpha}^G_A a^G - \frac{c}{2}\left(a^G\right)^2 \\
&= \alpha^* + (\alpha^*_A - \hat{\alpha}^G_A)\mathbb{E}[A] + \hat{\alpha}^G_A a^G - \frac{c}{2}\left(a^G\right)^2 \\
&= \alpha^* + (\alpha^*_A - c\cdot a^G)\mathbb{E}[A] + \frac{c}{2}\left(a^G\right)^2,
\end{align*}
where the third equality uses $a^G = \frac{\hat{\alpha}^G_A}{c}$. We use this expression to write the  difference between the promises associated with models $G$ and $G'$, respectively, as follows:
\begin{align*}
V_G(a^G) - V_{G'}(a^{G'}) &=& c\cdot(a^{G'} - a^G)\mathbb{E}[A] + \frac{c}{2}\left(\left(a^G\right)^2 - \left(a^{G'}\right)^2\right) \\
&=& c\cdot(a^{G} - a^{G'})\left(\frac{a^G+a^{G'}}{2}-\mathbb{E}[A]\right).
\end{align*}
This concludes the proof.

\end{proof}

\subsubsection{The Worst-Case Promise criterion}\label{app:maxmin}

Here we show that in menus of two linear models, the Worst-Case Promise criterion is equivalent to selecting the model that promises the \emph{lower} payoff if correct.

\begin{proposition}\label{lem: maxmin}
Let $\mathcal{G}$ be the set of available DAGs, with $|\mathcal{G}|= 2$. Then, DAG $G^*\in \mathcal{G}$ implies the lowest promise if and only if
\begin{align*}
G^* \in \arg\max_{G' \in \mathcal{G}}\min_{G \in \mathcal{G}} V_G(a^{G'}).
\end{align*}
\end{proposition}

\begin{proof}
Consider two DAGs $G$ and $G'$. The expected payoff predicted by model $G$ if recommendation $a^{G'}$ is implemented is given by:
\begin{align*}
V_G(a^{G'}) &= \hat{\alpha}_G + \hat{\alpha}^G_A a^{G'} - \frac{c}{2}\left(a^{G'}\right)^2\\
& = \alpha^* + (\alpha^*_A-\hat{\alpha}^{G}_A)\mathbb{E}[A] + \hat{\alpha}^{G}_A a^{G'} - \frac{c}{2}\left(a^{G'}\right)^2\\
& = \alpha^* + (\alpha^*-c\cdot a^{G})\mathbb{E}[A] + c\cdot a^{G}a^{G'} - \frac{c}{2}\left(a^{G'}\right)^2,
\end{align*}
where the second equality follows from the steps used in the proof of Lemma \ref{lem: highest promise}, and the third equality follows from $a^G = \frac{\hat{\alpha}^G_A}{c}$.

The expected payoff according to $G$ when action recommendation $a^{G'}$ is implemented is higher than the expected payoff according $G'$ when the action recommendation $a^{G}$ is implemented if and only if
\begin{align*}
V_G(a^{G'}) - V_{G'}(a^G) & = c\cdot(a^{G'} - a^G_A)\mathbb{E}[A] + \frac{c}{2}\left(\left(a^G\right)^2 - \left(a^{G'}\right)^2\right)\geq 0,
\end{align*}
or equivalently,
\begin{align*}
c\cdot(a^{G} - a^{G'})\left(\frac{a^G+a^{G'}}{2}-\mathbb{E}[A]\right)\geq 0.
\end{align*}

By Lemma \ref{lem: highest promise}, $V_G(a^G)>V_{G'}(a^{G'})$ if and only if $c\cdot(a^{G} - a^{G'})\left(\frac{a^G+a^{G'}}{2}-\mathbb{E}[A]\right)> 0$. Which in turn is equivalent to $V_G(a^{G'}) > V_{G'}(a^G)$.  Furthermore, by definition of $a^{G'}$, and $a^{G'}\neq a^G$, we have $V_{G'}(a^{G'})\geq V_{G'}(a^G)$. Hence, $V_{G'}(a^G)\leq  \min\{V_G(a^{G'}),V_{G'}(a^{G'})\}$, this completes the proof.
\end{proof}


\newpage
\section{Identification and estimation}\label{appendixIdentification}


\subsection{Distance between types}\label{appendix:typeDistanceHeatmap}

Figure \ref{distTypesOriginal} shows the distance between any pair of types on the choice sets of Experiment 1. We measure the distance between types $t$ and $t'$ as $d(t, t') = \left(\mat{M}(0)\mat{I}_t\right) \left(\mat{M}(0)\mat{I}_{t'}\right)'$ where $\mat{M}(0)$, as defined in section \ref{appendix:GMM}, is the matrix of theoretically predicted moments when the probability of noisy choices equals $q=0$ and $\mat{I}_t$ and $\mat{I}_{t'}$ are column vectors that have entries one in positions $t$ and $t'$, respectively, and zero everywhere else.

\begin{figure}[htb]
\caption{Distances between types \label{distTypesOriginal}}
\begin{center}
\includegraphics[width=0.95\textwidth]{tablesHandmade/originalHeatmapDistTypes.pdf}
\end{center}
\footnotesize
\textit{\textbf{Notes:}} The number in each cell and the cell's color indicate the distance between the two types defining that cell. Distances normalized from 0 to 1. Each type is listed as a pair of criteria (Fact-Based, Utility-Based). In each class, a period ($\cdot$) stands for `none.' The remaining criteria are encoded as follows.
\textit{Data-based}:
$u$: Direct Links,
$U$: Unconditional correlations,
$C$: Conditional correlations.
$*$: All correlations.
\textit{Utility-based}:
$P$: Best-Case Promise,
$p$: Worst-Case Promise,
$A$: Maximize Investment,
$a$: Minimize Investment.
\end{figure}

\FloatBarrier


\subsection{GMM estimation}\label{appendix:GMM}


Let $n$ denote the number of types, and let $\mat{t} = (t_1, \ldots, t_n)$ denote an element of the $n$-simplex $\Delta^n$ that represents the distribution over the types. Each type $i$ is associated with a matrix $T^i$ with elements $T^i_{c, m}$ that indicates the probability with which type $i$ chooses option $c$ from menu $m$.

Hence, if $i$ makes a unique choice on menu $m$, then $T^i_{c, m} = 1$ if $c$ is the chosen option and 0 otherwise. If $i$ randomizes, then $T^i_{c, m} = 1/2$.

We use the generalized method of moments to obtain an estimate $\hat{t}$ of the type vector and an estimate $\hat{q}$ of the noise probability. To state the optimization problem formally, and to prove identification of our model, note that the probability that type $i$ chooses option $c$ in menu $m$ when the noise probability is $q$ is given by $\tilde{p}_{c,m}^i(q) = (1-q) T_{c,m}^i + q\frac{1}{2}$. Similarly, $\tilde{p}_{(c,m),(c',m')}^i(q) = (1-q)^2 T_{c,m}^i T_{c',m'}^i + q^2 \frac{1}{4}+ q(1-q)\frac{1}{2}(T_{c,m}^i  +  T_{c',m'}^i)$ is the probability that type $i$ chooses option $c$ from menu $m$ \emph{and} option $c'$ from menu $m'$. Given $(t, q)$, our model then predicts first moments $\tilde{p}_{c,m} = \sum_{i=1}^n t_i \tilde{p}_{c,m}^i(q)$, and second moments $\tilde{p}_{(c,m)(c',m')} = \sum_{i=1}^n t_i  \tilde{p}_{(c,m),(c',m')}^i(q)$. Note that some of these moments are redundant since (conditional) choice probabilities across all options in a menu must sum to one. We remove redundant moments.


Let $\tilde{p}^i(q)$ denote the column vector of type $i$'s non-redundant first and second moments. Define $\mat{M}(q) = \big(\tilde{p}^1(q), \ldots, \tilde{p}^n(q)\big)$. Given a type distribution $\mat{t}$, the vector of theoretically predicted moments is $\mat{M}(q)\cdot\mat{t}$. Let $\tilde{\mat{\mathcal{E}}}$ denote the corresponding empirical moments. Our estimator is then defined as
\bmath \label{eq:estimationPre}
(\hat{\mat{t}},\hat{q}) = \arg \min_{\mat{t}, q}  \left(\mat{M}(q)\mat{{t}} - \tilde{\mat{\mathcal{E}}}\right) \mat{W}  \left(\mat{M}(q)\mat{{t}} - \tilde{\mat{\mathcal{E}}}\right)^{\intercal}  \textrm{ s.t. } \mat{t}\in \Delta^n, q \in [0,1]

\emath
For the weighting matrix $\mat{W}$ we use the optimal weighting matrix derived from two-stage feasible GMM.


Regarding identification, note that for a given noise parameter $q$, the type frequencies are identified only if $\mat{M}(q) \mat{\bar{t}} = \mat{M}(q)\mat{\bar{t}}'$ implies $\mat{\bar{t}} = \mat{\bar{t}}'$, that is, if the nullspace of the linear map $\mat{M}(q)$ from the type space to the moment space is empty, a condition that we check for our set of menus $\mathcal{M}$, as well as separately for the training and test sets  $\mathcal{M}^{train}$ and  $\mathcal{M}^{test}$, respectively. To estimate the model with endogenous $q$, we start the estimation procedure on a grid of initial values for $q$ that spans the unit interval and check the local identification condition at the resulting estimates.


\newpage
\section{Experiment design}

\subsection{Details}\label{appendix:designDetails}

Table \ref{tab:experimentStructure} shows an overview of the structure of the study, which was coded in Qualtrics and javascript. Here, we list details about each of the stages.


\begin{table}[hbt]
\caption{Experiment structure}\label{tab:experimentStructure}
\begin{center}
\begin{tabular}{|p{0.97\textwidth}|}
\hline
\begin{enumerate}[1.]
\item Comprehensive Part
	\begin{enumerate}
	\item Instructions and two comprehension checks
	\item Preliminary rounds
	\item Main decisions
	\end{enumerate}
\item Utility-Focused Part
	\begin{enumerate}
	\item Instructions
	\item Block-randomized decisions, with comprehension check right before
	\end{enumerate}
\item Additional decisions
	\begin{enumerate}
	\item Ambiguity preference elicitation
	\item Risk preference elicitation
	\item Explanation of own decision-making (free response and multiple choice)
	\item Questions eliciting the understanding of data charts
	\item Cognitive Response Test
	\item Pseudoscience scale
	\item Educational background and demographics
	\end{enumerate}
\end{enumerate}\\
\hline
\end{tabular}
\end{center}
\footnotesize
\textbf{\textit{Notes:}} The experiment proceeds in the order listed. Within each part of section 2, the order of rounds is randomized at the individual level. The two rounds of risk elicitation are also shown in individually randomized order.
\end{table}


\paragraph{Instructions and comprehension check}\label{appendix:instructions}

We display all instructions on screen. The entire experiment is in English and was advertised as such. A good command of English is a curricular requirement for all students in our subject pool.


\begingroup
\newgeometry{textwidth=\dimexpr\textwidth+2in\relax}
\small


\setlength{\tabcolsep}{3pt}
\begin{longtable}{c c c p{0.20\textwidth} c c c c c}
\caption{Practice Menus \label{tab:menusAppendix}}\\
(1)&(2)&(3)&\multicolumn{1}{c}{(4)}&\multicolumn{1}{c}{(5)}&\multicolumn{1}{c}{(6)}&\multicolumn{1}{c}{(7)}&\multicolumn{1}{c}{(8)}&\multicolumn{1}{c}{(9)} \\ \toprule
Menu& DGP & Competitor & \multicolumn{1}{c}{Description} & \multicolumn{3}{c}{Criterion identifies} & \multicolumn{2}{c}{Model chosen by} \\
& & & & \multicolumn{3}{c}{misspecified model} & & \\
\cmidrule(r){5-7}\cmidrule(r){8-9}
& & & & Direct & Uncond. & Cond. & Best-Case & Min. \\
& & & & Links & corr. & corr. & Promise & Invest.\\
\midrule
\endfirsthead
\caption[]{Practice Menus (continued)}\\
(1)&(2)&(3)&\multicolumn{1}{c}{(4)}&\multicolumn{1}{c}{(5)}&\multicolumn{1}{c}{(6)}&\multicolumn{1}{c}{(7)}&\multicolumn{1}{c}{(8)}&\multicolumn{1}{c}{(9)} \\ \toprule
Menu& DGP & Competitor & \multicolumn{1}{c}{Description} & \multicolumn{3}{c}{Criterion identifies} & \multicolumn{2}{c}{Model chosen by} \\
& & & & \multicolumn{3}{c}{misspecified model} & & \\
\cmidrule(r){5-7}\cmidrule(r){8-9}
& & & & Direct & Uncond. & Cond. & Best-Case & Min. \\
& & & & Links & corr. & corr. & Promise & Invest.\\
\midrule
\endhead
\rule[-0.13000\textheight]{0pt}{0.14000\textheight}$P_1$ & \multirow{1}{*}{\begin{tikzpicture}[>=Stealth, scale=1]
\scriptsize
\node (A) at (0,2) {$I$};
\node (X) at (0,1) {$C_1$};
\node (Y) at (0,0) {$Y$};
\node (Z) at (0.5,1.5) {$C_2$};
\draw[->] (A) to[out=180, in=180, looseness=1.3] (X);
\draw[->] (A) to[out=0, in=90, looseness=1] (Z);
\draw[->] (X) to[out=0, in=-90, looseness=1] (Z);
\draw[->] (X) -- (Y);
\end{tikzpicture}} & \multirow{1}{*}{\begin{tikzpicture}[>=Stealth, scale=1]
\scriptsize
\node (A) at (0,2) {$I$};
\node (X) at (0,1) {$C_2$};
\node (Y) at (0,0) {$Y$};
\node (Z) at (0.5,1.5) {$C_1$};
\draw[->] (A) to[out=180, in=180, looseness=1.3] (X);
\draw[->] (A) to[out=0, in=90, looseness=1] (Z);
\draw[->] (X) to[out=0, in=-90, looseness=1] (Z);
\draw[->] (X) -- (Y);
\end{tikzpicture}} & \multirow[t]{1}{*}{\scriptsize\parbox[t]{0.20\textwidth}{Your Action indirectly influences your Bonus through one of the Counters. The second Counter is a symptom of both your Action and the first Counter.}} & No & No & Yes & DGP & Comp. \\
\rule[-0.13000\textheight]{0pt}{0.12000\textheight}$P_2$ & \multirow{1}{*}{\begin{tikzpicture}[>=Stealth, scale=1]
\scriptsize
\node (A) at (-0.5,2) {$I$};
\node (X) at (-0.5,1) {$C_1$};
\node (Y) at (0,0) {$Y$};
\node (Z) at (0.5,2) {$C_2$};
\draw[->] (X) to[out=-90, in=180, looseness=1] (Y);
\draw[->] (Z) to[out=-90, in=0, looseness=0.7] (Y);
\draw[->] (A) -- (X);
\end{tikzpicture}} & \multirow{1}{*}{\begin{tikzpicture}[>=Stealth, scale=1]
\scriptsize
\node (A) at (-0.5,2) {$I$};
\node (X) at (-0.5,1) {$C_2$};
\node (Y) at (0,0) {$Y$};
\node (Z) at (0.5,2) {$C_1$};
\draw[->] (X) to[out=-90, in=180, looseness=1] (Y);
\draw[->] (Z) to[out=-90, in=0, looseness=0.7] (Y);
\draw[->] (A) -- (X);
\end{tikzpicture}} & \multirow[t]{1}{*}{\scriptsize\parbox[t]{0.20\textwidth}{Your Action indirectly influences your Bonus through one of the counters. The other counter is not influenced by anything, but it has an additional, direct influence on your Bonus.}} & Yes & Yes & Yes & Comp. & Comp. \\
\\
\bottomrule
\end{longtable}
\vspace{-0.3cm}
\begin{center}
\begin{minipage}{0.83\linewidth}
\setstretch{1}
\footnotesize \textit{\textbf{Notes}}: $I$ denotes the investment, referred to as Action in the videos. Column 4 shows the text spoken in the video. In the screens that correspond to Figure \ref{fig:interface}, the counters are referred to by color. In the case of $M_1$, for instance, the text for one of the models is ``Your Bonus only depends on the red counter. Your Action influences that Counter both directly and through the blue counter.'  \normalsize
\setstretch{}
\end{minipage}
\end{center}


\restoregeometry
\endgroup


We randomize the following display elements:
(i) The order in which the list of charts is displayed. For each individual, we randomly select an order of charts displaying unconditional relations that we keep constant for the entire experiment.
(ii) Which advisor is on the left of the screen and which is right is randomized in each round for each individual.
(iii) The position of models and recommendations within the advisor speech bubbles. This is kept constant for a given subject. For half of the subjects, the models are on top, for the other half the recommendations are on top.
(iv) Whether an advisor's promise is listed above or below his claim about how much the subject can expect to earn from following the competing advisor. A subject either sees all or none of the models presented in the opposite order, both in the Comprehensive Part and in the Utility-Focused Part.
(v) We randomly redraw the colors of the advisors in each round for each individual to prevent subjects from forming beliefs such as `the red advisor tends to be correct.'


\paragraph{Practice menus}

All subjects first completed the two practice menus listed in Table \ref{tab:menusAppendix} in random order. Practice menus were not identified as such and could determine payments. As preregistered, we discard choices from practice menus.


\paragraph{Parameters}


Table \ref{tab:parameters} displays the parameters used in each round. Table \ref{tab:utilities} displays the resulting model recommendations, promises, and claims about the competing models' recommendation when fit to these DGP, for the High Stakes condition. In the Low Stakes condition, the corresponding amounts are two thirds lower.


\begin{table}[htb]
\caption{DGP parameters}\label{tab:parameters}
\setlength{\tabcolsep}{4pt}
\begin{footnotesize}
\begin{center}
\begin{tabular}{lrrrrrrrrrrrrrr}
\toprule
Round & $\beta_A$ & $\beta_X$ & $\beta_Y$ & $\beta_{Z}$ & $\beta_{AX	}$ & $\beta_{AY}$ & $\beta_{AZ}$ & $\beta_{XY}$ & $\beta_{XZ}$ & $\beta_{YZ}$ & $\sigma_{A}$ & $\sigma_{X}$ & $\sigma_{Y}$ & $\sigma_{Z}$ \\
\midrule

$P_{1}$ & 0.99 & 0.00 & 1.50 & 0.50 & 3.00 & 0.00 & -1.90 & 1.00 & 0.84 & 0.00 & 1.00 & 2.00 & 2.00 & 1.00 \\
$P_{2}$ & 2.01 & 0.00 & 0.99 & 0.50 & 1.12 & 0.00 & 0.00 & 2.00 & 0.00 & 1.00 & 1.00 & 1.00 & 1.00 & 1.00 \\
$M_{1}$ & 3.46 & 2.00 & 5.17 & -2.50 & -1.70 & 0.00 & 2.84 & 2.10 & 0.84 & 0.00 & 1.00 & 0.01 & 6.50 & 3.70 \\
$M_{2}$ & 1.67 & 4.00 & 0.74 & -3.50 & -1.70 & 0.00 & 2.84 & 2.10 & 0.84 & 0.00 & 1.00 & 0.01 & 6.50 & 3.70 \\
$M_{3}$ & 1.34 & 0.50 & -1.78 & 0.50 & 1.30 & 0.00 & 0.00 & 0.00 & 1.50 & 1.72 & 1.00 & 0.01 & 4.40 & 2.76 \\
$M_{4}$ & 1.77 & 1.00 & 0.54 & -1.20 & 0.00 & 3.78 & 0.00 & 2.00 & 2.00 & 1.40 & 1.00 & 1.00 & 1.00 & 1.00 \\
$M_{5}$ & 3.52 & 1.00 & -7.23 & 3.00 & 0.00 & 3.75 & 0.00 & 2.04 & 2.00 & 1.40 & 1.00 & 1.00 & 1.00 & 1.00 \\
$M_{6}$ & 1.07 & 0.05 & 1.26 & 0.00 & 0.00 & 3.08 & -2.00 & 2.00 & 0.00 & 2.39 & 1.00 & 0.90 & 0.03 & 0.19 \\
$M_{7}$ & 2.86 & 2.00 & -7.93 & 3.00 & 0.00 & 3.09 & -2.00 & 2.00 & 0.00 & 2.39 & 1.00 & 0.90 & 0.03 & 0.29 \\
$M_{8}$ & 1.59 & -1.00 & 2.93 & 0.50 & 1.60 & 0.00 & 0.00 & -3.15 & 2.00 & 2.00 & 1.00 & 0.50 & 1.00 & 3.00 \\
$M_{9}$ & 3.38 & -1.00 & 16.93 & -5.50 & 1.60 & 0.00 & 0.00 & -3.15 & 2.00 & 2.00 & 1.00 & 0.50 & 1.00 & 3.00 \\
$M_{10}$ & 1.60 & 0.00 & -0.52 & 1.00 & 3.80 & 0.00 & 0.00 & 0.95 & 0.00 & 0.87 & 1.00 & 0.90 & 1.70 & 4.54 \\
$M_{11}$ & 1.08 & -0.59 & 2.86 & 0.00 & 0.00 & 3.09 & 0.00 & 2.77 & 0.00 & 1.00 & 1.00 & 1.00 & 0.01 & 1.72 \\
$M_{12}$ & 2.77 & 1.00 & -3.93 & 1.00 & 0.00 & 3.00 & 0.00 & 3.43 & 0.00 & 1.00 & 1.00 & 1.00 & 0.10 & 2.00 \\
$M_{13}$ & 0.25 & 1.00 & 0.37 & 0.50 & 0.75 & 0.00 & 0.00 & 3.00 & 0.20 & 0.39 & 1.00 & 0.31 & 0.75 & 1.25 \\
$M_{14}$ & 1.65 & 0.50 & -0.41 & 0.50 & 1.05 & 0.00 & 0.00 & 2.00 & 0.20 & 2.00 & 1.00 & 1.00 & 1.00 & 1.00 \\
$W_{1}$ & 0.23 & 1.00 & 2.43 & 1.00 & 0.00 & 2.49 & 3.00 & 1.00 & 0.00 & 0.00 & 1.00 & 1.00 & 1.00 & 1.00 \\
$W_{2}$ & 4.02 & -2.72 & -0.07 & 0.00 & 1.31 & 0.00 & 0.00 & 1.53 & 2.12 & 1.53 & 1.00 & 0.01 & 0.43 & 0.07 \\
$W_{3}$ & 1.50 & -1.45 & 0.85 & 1.00 & 3.80 & 0.00 & 0.00 & 1.00 & 0.00 & 0.40 & 1.00 & 1.00 & 5.20 & 0.74 \\
$W_{4}$ & 2.78 & -1.00 & 1.28 & 1.00 & 1.67 & 0.00 & 0.00 & 1.80 & 0.45 & 0.00 & 1.00 & 1.00 & 1.00 & 1.50 \\


\bottomrule
\end{tabular}
\end{center}
\end{footnotesize}
\footnotesize
\textbf{\textit{Notes:}} Each model in our setting is a system of linear Gaussian equations. For any variable $i$, $\beta_i$ denotes the constant term in the equation corresponding to endogenous variable $i$, and $\sigma_i$ is the standard deviation of the corresponding error term. For any endogenous variable $i$ that depends on some other variable $j$, $\beta_{ij}$ is the slope coefficient on variable $j$ in the equation corresponding to endogenous variable $i$.
\end{table}


\begin{table}[htb]
\caption{Models' recommendations, promises, and claims about the competitor's recommendation}\label{tab:utilities}
\setlength{\tabcolsep}{4pt}
\begin{center}
\resizebox{\textwidth}{!}{
\begin{tabular}{lccccccc}
\toprule
Round & \multicolumn{3}{c}{Model 1} & \multicolumn{3}{c}{Model 2} \\
\cmidrule(r){2-4}\cmidrule(r){5-8}
& Recommendation & Promise & Claim about  & Recommendation & Promise & Claim about  \\
&  &  &  competitor &  &  &  competitor \\
\midrule

$P_{1}$ & 54.00 & 72.00 & 41.89 & 3.47 & 48.02 & 17.92 & \\
$P_{2}$ & 30.11 & 48.00 & 17.89 & 0.00 & 72.00 & 41.90 & \\
$M_{1}$ & 12.48 & 72.00 & 41.87 & 81.40 & 48.01 & 17.88 & \\
$M_{2}$ & 12.48 & 48.00 & 17.87 & 81.40 & 72.01 & 41.88 & \\
$M_{3}$ & 67.57 & 72.01 & 42.01 & 7.52 & 48.02 & 18.02 & \\
$M_{4}$ & 85.73 & 72.03 & 41.95 & 14.25 & 48.05 & 17.97 & \\
$M_{5}$ & 84.38 & 48.00 & 17.99 & 13.74 & 71.98 & 41.96 & \\
$M_{6}$ & 56.92 & 72.04 & 41.98 & 4.25 & 48.03 & 17.98 & \\
$M_{7}$ & 57.21 & 48.02 & 17.90 & 4.31 & 72.00 & 41.89 & \\
$M_{8}$ & 11.10 & 48.01 & 17.87 & 77.81 & 72.01 & 41.88 & \\
$M_{9}$ & 11.10 & 72.06 & 41.92 & 77.81 & 48.00 & 17.87 & \\
$M_{10}$ & 78.19 & 72.00 & 41.89 & 11.26 & 48.01 & 17.90 & \\
$M_{11}$ & 57.44 & 72.00 & 41.99 & 4.41 & 48.04 & 18.03 & \\
$M_{12}$ & 54.00 & 47.98 & 17.89 & 3.47 & 72.02 & 41.93 & \\
$M_{13}$ & 30.38 & 76.67 & 46.29 & 0.00 & 56.50 & 8.94 & \\
$M_{14}$ & 26.46 & 48.00 & 21.54 & 0.00 & 72.03 & 34.42 & \\
$W_{1}$ & 0.00 & 48.06 & 17.92 & 30.13 & 72.00 & 41.87 & \\
$W_{2}$ & 24.10 & 72.03 & 41.85 & 108.23 & 48.05 & 17.87 & \\
$W_{3}$ & 74.05 & 72.03 & 42.02 & 9.78 & 48.01 & 18.01 & \\
$W_{4}$ & 54.22 & 48.04 & 17.89 & 3.51 & 72.05 & 41.90 & \\

 \\
\bottomrule
\end{tabular}}
\end{center}
\footnotesize
\textbf{\textit{Notes:}} This table shows the numbers for the High Stakes condition. In the Low Stakes condition, all numbers are two thirds lower.
\end{table}

\FloatBarrier

\paragraph{Risk preference elicitation}

There are two rounds. In each subjects chose one of six lotteries each of which offers a 2/3 chance of obtaining a higher amount and a 1/3 chance of obtaining a lower amount. In the first round, the amounts in Swiss Francs in the Low Stakes treatment are (15, 15), (16, 14), (20, 10), (24, 6), (28, 2). In the second round, they are (18.75, 18.75), (20, 17.5), (25, 12.5), (30, 7.5), (35, 2.5). In the High Stakes treatment, all amounts are tripled. We randomize whether the lotteries are ordered from safe to risky or from risky to safe.


\FloatBarrier

\paragraph{Demographics and other characteristics}

In addition to the characteristics listed in Section \ref{sec:designDetails}, we also elicit the following characteristics:
(i) native language,
(ii) country of origin,
(iii) age,
(iv) degree level the subject is working towards,
(v) monthly spending,
(vi) religiosity
(vii) eligibility to vote in political elections in Switzerland,
(viii) how much the person agrees with the political party to which they are closest

In terms of educational background, we elicit the institution and faculty at which the subject is enrolled in their main field of study. For the purpose of analysis, we will classify these institutions into STEM, business/economics, and other.


\subsection{Previous laboratory study}\label{appendix:previousLabStudy}


The present paper answers broadly similar research questions as a previous laboratory study that we ran with 485 subjects at the Laboratory for Experimental and Behavioral Economics at the University of Zurich in April and May 2023. We circulated that study the title `Competing Causal Interpretations: An Experimental Study.' The present paper supersedes that study, which suffered from several design flaws. We provide the working paper that contains the complete description and all results of that study in the replication package.

The previous study differed from the current study in the following ways:

\textit{Unnatural assistance for subjects.} The previous design provided subjects with substantial interpretive assistance in three ways. First, the instructions included two pages explaining the correlational implications of archetypal causal structures, which could be viewed at any point during the survey. Second, the interface highlighted the charts that distinguished the competing models and hid all other charts behind an ``additional charts'' button. Because subjects performed well, we could not tell whether this performance reflected genuine ability or simply the help we provided. The current experiment removes all such aids. Third, in the previous design, subjects knew that in each round, one advisor's model was correct, so that ruling out an incorrect model in a two-advisor menu ensured that the remaining advisor's model would be correct. This strategy is not available in the current study that included menus $\mathcal{W}$ in which both advisors' models are misspecified.

\textit{Missing information for the Worst-Case Promise criterion.} The previous experiment did not display each advisor's model's predicted payoff implications from following the competing advisor's recommendation. This information is essential for applying the Worst-Case Promise criterion and too complex for subjects to derive themselves. The current design provides it directly.

\textit{Problematic set of choice criteria.} The previous experiment was more ambitious in identifying choice criteria. In addition to advice-based and utility-based criteria, it sought to identify structure-based criteria, defined as preferences over the structure of DAGs independent of their fit to the data or their utility implications. It focused on six such criteria: prefer more or prefer less, crossed with number of links, subsets of links, and number of root causes. However, the structural model assumes that the set of choice criteria is comprehensively specified, and the space of plausible notions of complexity is very large. For instance, one could imagine subjects applying such criteria only to nodes that lie on the causal path from action to outcome, or only to nodes that are ancestral to the outcome. This explosion of cases violates the assumption that the set is comprehensively specified. The current experiment rules out structure-based preferences by design: each menu pairs two DAGs that are mirror images of each other, with the two counters interchanged. Any notion of simplicity that does not depend on node labeling, such as number of links, subsets of links, or number of root causes, treats the two DAGs as equivalent.

\textit{No designation into training and test sets} The previous experiment included out-of-sample prediction exercises that parallel the spirit of Section \ref{sec:outOfSample}. These suffered from two issues. First, there was preregistered separation into training and test sets. Instead, we relied on a leave-one-out approach. Second, the separation in the current experiment is clean in the sense that no DAG in the training set appears in the test set and vice versa. The previous experiment could not guarantee such separation.

\textit{Less natural presentation.} The current experiment introduces causal connections through a video that gradually reveals the hypothesized structure and uses natural-language terms such as ``cause'' and ``symptom.'' The previous experiment presented only pictures of DAGs and had to explicitly explain to subjects what a DAG is, potentially training them into an understanding they would not otherwise have possessed.

The working paper also includes a laboratory study (`Experiment 2') ran at the University of Zurich based on a different identification strategy that produced results consistent with the main study (`Experiment 1'). It also included a study with prolific subjects. In that study, subjects chose from three-node models. While we could exclude the set of structure-based criteria assumed in our model, that setting does not permit excluding structure-based criteria more broadly. Accordingly, inference from that experiment might be confounded by structure-based criteria from outside the set we model. Similar to our pilot studies on Prolific (described in Subsection \ref{appendix:pilots}) subjects rushed through the study. In the treatment paralleling the pilots for the current study, 74\% of subjects never viewed any of the charts that would be needed to distinguish the models, and fewer than 18\% viewed one of the charts in at least half the rounds.


\subsection{Pilot studies}\label{appendix:pilots}


Between February and May 2025, we ran five pilot studies on Prolific with US-based English-speaking participants, summarized in Table \ref{tab:pilotsOverview}. Subjects were paid a fixed completion fee plus an incentive-compatible bonus from one randomly selected decision round.

The pilots were planned to inform the design that we would eventually run in the laboratory with student subjects. We planned for the main sessions to take part in person because we wanted certainty that subjects would not be able to use outside help. The pilots further helped to make the preregistration manageable. This is due to the fact that any thorough experiment conducts analysis of the mechanisms that drive a main result, but the nature of these analyses may differ depending on the nature of the main result. Knowing the main result in advance eliminates the need to specify, in advance, all the analyses one would have conducted had the data come out qualitatively differently than expected.

In all pilots, we chose parameters to approximate the following: one advisor's model predicts a payoff of \$6 from following him and of \$1.5 from following his competitor, while the other advisor's model predicts a payoff of \$4.5 from following her and of \$3.5 from following her competitor. In the high-stakes pilot, we increased all amounts by a factor of 10. As in the main study, all pilots began with 2 practice rounds, not identified to the subject as such, which we discarded.

The first pilot, P1, tested whether subjects would understand the new design. Subjects did not engage with the study; they rarely opened the charts they would need to rule out misspecified models (we registered all clicks). Specifically, 71\% of subjects never opened a chart in any of the main rounds, and merely 12.5\% opened at least one chart in at least half of the main rounds.

We hypothesized that given the moderate stakes, subjects were trying to rush through the study with minimal effort. We therefore ran pilot P2, which mirrored P1, except that all incentive amounts were increased by a factor of 10. Contrary to our expectations, we found that 77\% of subjects never opened any chart in any of the main rounds.

A difference between the main study and all pilots is that the instructions of the main study required subjects to open at least one data chart showing unconditional correlations and one showing conditional correlations. In principle, this difference could explain the diverging rates of chart-checking across the main and online samples. However, both the previous laboratory study (Appendix \ref{appendix:previousLabStudy}) and its implementation on Prolific required subjects to click on a chart in the instructions, and still found a large (though slightly smaller) gap in charts-checking across the subject pools.
Based on these pilots, we expected that a large fraction of laboratory subjects would make utility-based rather than fact-based decisions. Our first laboratory experiment (see Section \ref{appendix:previousLabStudy}) suggested that a substantial fraction of subjects might choose in line with the Best-Case Promise criterion (though this finding was possibly due to the fact that that experiment did not show the information needed for applying the Worst-Case Promise criterion). We realized that such a finding would contrast with a large body of work on decision making under ambiguity that usually detects behavior much more closely related to maxmin (Worst-Case Promise) than to maxmax (Best-Case Promise) decision making. If our experiment were to find a substantial fraction of Best-Case Promise decision making, we would therefore need to be able to explain why our results differed from that literature. Accordingly, we devised the experiment of Section 4.4 (the Utility-Focused part).


Pilot P3 tested an initial version of that experiment that only included the Spread condition. Pilot P4 added the Dominance Condition. Pilot P5 added the variation that in one version of each menu, we flipped the location of the payoff information on the screen in one but not the other speech bubble. We also realized that the elicitation of ambiguity preferences we had been using to date suffered from a potential issue, and added an elicitation in the style of  \citet{EckelGrossman}. The results of pilots P3 to P5 were stark and closely resembled the respective findings from the laboratory experiment (Section 4.4)

Once we were confident that we had understood the drivers of choice in the utility-focused part, we moved to conducting the main laboratory sessions.


\begin{table}[H]
\centering
\small
\caption{Pilot studies: overview\label{tab:pilotsOverview}}
\begin{tabular}{@{}llrcccl@{}}
\toprule
Index & Date & $n$ & Stakes & Utility-focused part  \\
\midrule
P1  & 2025-02-05 & 72 & Low  & No  \\
P2  & 2025-03-05 & 26 & High & No  \\
P3  & 2025-04-04 & 72 & Low  & Yes \\
P4  & 2025-04-12 & 50 & Low  & Yes \\
P5  & 2025-05-14 & 61 & Low  & Yes \\
\bottomrule
\end{tabular}
\end{table}


\newpage

\FloatBarrier
\section{Supplementary Analysis}


\subsection{Summary statistics}\label{app:summaryStats}

Table \ref{tab:summaryStats} shows summary statistics of our samples. Our sample skews slightly female and to the political left, as is common for student subject pools. Slightly under half of our subjects are from German-speaking Europe, and a bit over a quarter are from Asia.


\begin{table}[htb]
\caption{Subject characteristics \label{tab:summaryStats}}
\setstretch{1}
\small
\begin{center}
\begin{tabular}{lc}
\toprule

Male &     0.461 \\
Female &     0.539 \\
Age &      24.5 \\
\textit{Place of origin} \\
\quad German-speaking Europe &     0.225 \\
\quad Non-German-speaking Europe &     0.068 \\
\quad Asia &     0.135 \\
\quad Other &     0.572 \\
\textit{Current degree program} \\
\quad BA &     0.192 \\
\quad MA &     0.263 \\
\quad PhD and MD/JD &     0.029 \\
\quad Not working toward a degree &     0.025 \\
\textit{Field of study} \\
\quad STEM &     0.643 \\
\quad Economics or business &     0.118 \\
\quad Other field &     0.219 \\
\textit{Statistical knowledge} \\
\quad Can name $P(A|B)$ &     0.246 \\
\quad Can complete ``Correlation does not...'' &     0.536 \\
\quad Can spell out DAG &     0.077 \\
\quad Took class on causal inference &     0.225 \\
\textit{Psychological measures} \\
\quad CRT score (0–7) &     5.072 \\
\quad Pseudoscience score (20–100) &    59.560 \\
Religiosity (1–5) &     1.696 \\
\textit{Closest political party} \\
\quad SVP &     0.028 \\
\quad FDP &     0.066 \\
\quad BDP &     0.031 \\
\quad CVP &     0.045 \\
\quad GLP &     0.117 \\
\quad SP &     0.130 \\
\quad Green &     0.058 \\
\quad PdA &     0.033 \\
Subjects &       414 \\

\\
\bottomrule
\end{tabular}
\end{center}
\footnotesize
\textbf{\textit{Notes: }} Political parties are listed in order of overall stance on the political spectrum, beginning with most conservative. CVP is the center party.
\end{table}


\subsection{Order effects}\label{app:orderEffects}

Panel A of Figure \ref{responseTimes} plots the median response time against the position at which the subject made the corresponding decision. Subjects take substantially longer on the first decision, presumably to familiarize themselves with the interface. While response times decline across the entire experiment, this decline appears to reflect learning rather than decreased attention, as Panel B shows. For each decision position, it plots the fraction of subjects who viewed at least one data chart. Approximately 85\% of subjects do so in any given round.


\begin{figure}[H]
\caption{Order effects \label{responseTimes}}
\begin{center}
\begin{tabular}{cc}
\textbf{A. Response time} & \textbf{B. Data viewing }\\
\includegraphics[width=0.48\textwidth]{graphs/orderEffectsTime}&
\includegraphics[width=0.48\textwidth]{graphs/orderEffectsDataViewing}\\
\end{tabular}
\end{center}
\end{figure}


\FloatBarrier


\subsection{Best-fitting types} \label{appendix:rawTypes}

Table \ref{tab:pureTypes} lists the estimated frequencies of all types estimated on the union of training and test set, along with heteroscedasticity-robust standard errors, separately for the Low and High Stakes treatments.


Use a width-based resize on the two panels. Notes stay unscaled.


\begin{table}[htp]
\caption{Most common types \label{tab:pureTypes}}
\centering

\resizebox{\linewidth}{!}{
\begin{minipage}[t]{0.73\linewidth}
\centering
\textbf{A. Low stakes} \medskip \\
\begin{tabular}{lcccccccccccccccccccc}
\toprule
\multicolumn{10}{c}{Criterion} & Freq. & s.e. \\
\cmidrule(r){1-10}
\multicolumn{5}{c}{Utility-based} &
\multicolumn{5}{c}{Fact-based} \\
\cmidrule(r){1-5}\cmidrule(r){6-10}\cmidrule(r){11-12}

\rotatebox{90}{None} & \rotatebox{90}{High Promise} & \rotatebox{90}{Low Promise} & \rotatebox{90}{High Action} & \rotatebox{90}{Low Action}& \rotatebox{90}{None} & \rotatebox{90}{Direct Links} & \rotatebox{90}{Unconditonal} & \rotatebox{90}{Conditional} & \rotatebox{90}{All}& \\
\cmidrule(r){1-5}\cmidrule(r){6-10} \cmidrule(r){11-12}
 \rowcolor{gray!20} $ \circ $ &  &  &  &  &  &  &  &  &  $ \bullet $ &  0.654 & (0.049)\\
 $ \circ $ &  &  &  &  &  &  $ \bullet $ &  &  &  &  0.056 & (0.026)\\
 \rowcolor{gray!20} &  &  $ \bullet $ &  &  &  &  &  $ \bullet $ &  &  &  0.047 & (0.012)\\
 $ \circ $ &  &  &  &  &  &  &  &  $ \bullet $ &  &  0.046 & (0.066)\\
 \rowcolor{gray!20} &  &  &  &  $ \bullet $ &  $ \circ $ &  &  &  &  &  0.038 & (0.007)\\
 &  &  $ \bullet $ &  &  &  $ \circ $ &  &  &  &  &  0.037 & (0.009)\\
 \rowcolor{gray!20} &  &  $ \bullet $ &  &  &  &  &  &  $ \bullet $ &  &  0.029 & (0.011)\\
 $ \circ $ &  &  &  &  &  &  &  $ \bullet $ &  &  &  0.027 & (0.035)\\
 \rowcolor{gray!20} &  $ \bullet $ &  &  &  &  $ \circ $ &  &  &  &  &  0.025 & (0.010)\\
 &  $ \bullet $ &  &  &  &  &  $ \bullet $ &  &  &  &  0.020 & (0.009)\\
 \rowcolor{gray!20} &  &  &  &  $ \bullet $ &  &  $ \bullet $ &  &  &  &  0.012 & (0.007)\\
 &  &  &  $ \bullet $ &  &  $ \circ $ &  &  &  &  &  0.009 & (0.010)\\
 \rowcolor{gray!20} &  &  &  &  $ \bullet $ &  &  &  $ \bullet $ &  &  &  0.000 & (0.017)\\
 &  &  &  &  $ \bullet $ &  &  &  &  $ \bullet $ &  &  0.000 & (0.019)\\
 \rowcolor{gray!20} &  $ \bullet $ &  &  &  &  &  &  $ \bullet $ &  &  &  0.000 & (0.015)\\
 &  &  $ \bullet $ &  &  &  &  $ \bullet $ &  &  &  &  0.000 & (0.008)\\
 \rowcolor{gray!20} &  $ \bullet $ &  &  &  &  &  &  &  $ \bullet $ &  &  0.000 & (0.007)\\


\end{tabular}
\end{minipage}\hfill
\begin{minipage}[t]{0.73\linewidth}
\centering
\textbf{B. High stakes} \medskip \\
\begin{tabular}{lcccccccccccccccccccc}
\toprule
\multicolumn{10}{c}{Criterion} & Freq. & s.e. \\
\cmidrule(r){1-10}
\multicolumn{5}{c}{Utility} &
\multicolumn{5}{c}{Factual} \\
\cmidrule(r){1-5}\cmidrule(r){6-10}\cmidrule(r){11-12}

\rotatebox{90}{None} & \rotatebox{90}{High Promise} & \rotatebox{90}{Low Promise} & \rotatebox{90}{High Action} & \rotatebox{90}{Low Action}& \rotatebox{90}{None} & \rotatebox{90}{Direct Links} & \rotatebox{90}{Unconditonal} & \rotatebox{90}{Conditional} & \rotatebox{90}{All}& \\
\cmidrule(r){1-5}\cmidrule(r){6-10} \cmidrule(r){11-12}
 \rowcolor{gray!20} $ \circ $ &  &  &  &  &  &  &  &  &  $ \bullet $ &  0.569 & (0.048)\\
 &  &  &  &  $ \bullet $ &  $ \circ $ &  &  &  &  &  0.069 & (0.010)\\
 \rowcolor{gray!20} &  &  $ \bullet $ &  &  &  &  &  $ \bullet $ &  &  &  0.057 & (0.012)\\
 $ \circ $ &  &  &  &  &  &  $ \bullet $ &  &  &  &  0.047 & (0.019)\\
 \rowcolor{gray!20} &  &  &  $ \bullet $ &  &  $ \circ $ &  &  &  &  &  0.047 & (0.010)\\
 &  &  $ \bullet $ &  &  &  $ \circ $ &  &  &  &  &  0.043 & (0.011)\\
 \rowcolor{gray!20} $ \circ $ &  &  &  &  &  &  &  &  $ \bullet $ &  &  0.036 & (0.062)\\
 &  &  $ \bullet $ &  &  &  &  &  &  $ \bullet $ &  &  0.034 & (0.011)\\
 \rowcolor{gray!20} $ \circ $ &  &  &  &  &  &  &  $ \bullet $ &  &  &  0.032 & (0.036)\\
 &  $ \bullet $ &  &  &  &  $ \circ $ &  &  &  &  &  0.027 & (0.008)\\
 \rowcolor{gray!20} &  $ \bullet $ &  &  &  &  &  $ \bullet $ &  &  &  &  0.019 & (0.008)\\
 &  &  &  &  $ \bullet $ &  &  &  &  $ \bullet $ &  &  0.012 & (0.019)\\
 \rowcolor{gray!20} &  &  $ \bullet $ &  &  &  &  $ \bullet $ &  &  &  &  0.004 & (0.007)\\
 &  &  &  &  $ \bullet $ &  &  $ \bullet $ &  &  &  &  0.003 & (0.005)\\
 \rowcolor{gray!20} &  &  &  &  $ \bullet $ &  &  &  $ \bullet $ &  &  &  0.000 & (0.012)\\
 &  $ \bullet $ &  &  &  &  &  &  &  $ \bullet $ &  &  0.000 & (0.007)\\
 \rowcolor{gray!20} &  $ \bullet $ &  &  &  &  &  &  $ \bullet $ &  &  &  0.000 & (0.011)\\


\end{tabular}
\end{minipage}
}

\flushleft
\footnotesize\textbf{\textit{Notes:}} The symbol $\bullet$ indicates that the corresponding criterion is being used, $\circ$ indicates that no criterion from the corresponding class is being applied. The top row contains only a single symbol because the Conditional Correlations criterion prevents the identification of structure- and advice-based criteria. Heteroskedasticity-robust standard errors in parentheses.
\end{table}


\subsection{Distribution of decision criteria}\label{app:distCritPrereg}

Figure \ref{fig:criteriaDistributionSplit} shows the distribution of decision criteria estimated only on the training set, separately for the low and high stakes conditions.

\begin{figure}[H]
\caption{Distribution of decision criteria on the training set, split by stakes \label{fig:criteriaDistributionSplit}}
\begin{center}
\begin{tabular}{cc}
\textbf{A. Fact-based criteria} &
\textbf{B. Utility-based criteria} \\
\includegraphics[width=0.48\textwidth]{graphs/freqAccStruct} &
\includegraphics[width=0.48\textwidth]{graphs/freqMotStruct} \\
\end{tabular}
\end{center}
\vspace{-0.5cm}
\footnotesize
\textit{\textbf{Notes}}: Whiskers show 95\%-confidence intervals, truncated at 0. \textit{Panel A:}   \textit{Panel B:} Estimates of advice-based criteria in Experiment 1 are shown conditional on not using the Conditional Correlations criterion.
\end{figure}


\subsection{Utility-Focused Part}\label{app:part2}
Table \ref{tab:part2} shows choice frequencies and statistical tests for differences across treatment conditions, pooling over the presentation-order condition.

\begin{table}[htb]
\small
\caption{Utility-Focused Part: Econometric Tests \label{tab:part2}}
\begin{center}
\setlength{\tabcolsep}{3pt}
\resizebox{\textwidth}{!}{
\begin{tabular}{lcccccccc} & (1) & (2) & (3) & (4) & (5) & (6) & (7) & (8)\\ \toprule
& \multicolumn{2}{c}{\textbf{Spread condition}} & \multicolumn{2}{c}{\textbf{Dominance condition}} &\multicolumn{4}{c}{Difference between columns} \\
Dependent variable&\multicolumn{2}{c}{Maxmax choice}&\multicolumn{2}{c}{Dominant choice}& \multicolumn{4}{c}{$p$-values} \\
\cmidrule(r){2-3}\cmidrule(r){4-5} \cmidrule(r){6-9}
Stakes & Low & High & Low & High & 1-2 & 3-4 & 1-3 & 2-4 \\ \midrule

\textit{Model frame} \\ \quad \textit{Levels} \\ \qquad State constant &  \makecell[c]{0.357***\\(0.030)}  &  \makecell[c]{0.321***\\(0.030)}  &  \makecell[c]{0.762***\\(0.026)}  &  \makecell[c]{0.716***\\(0.029)}  & 0.393 & 0.236 & 0.000 & 0.000 \\
\qquad Action constant &  \makecell[c]{0.095***\\(0.016)}  &  \makecell[c]{0.064***\\(0.016)}  &  \makecell[c]{0.962***\\(0.011)}  &  \makecell[c]{0.978***\\(0.009)}  & 0.164 & 0.264 & 0.000 & 0.000 \\
\quad \textit{Difference} &  \makecell[c]{-0.262***\\(0.029)}  &  \makecell[c]{-0.257***\\(0.031)}  &  \makecell[c]{0.200***\\(0.028)}  &  \makecell[c]{0.262***\\(0.029)}  & 0.914 & 0.124 & 0.000 & 0.000 \\
\textit{Gamble frame} \\ \quad \textit{Levels} \\ \qquad State constant &  \makecell[c]{0.379***\\(0.030)}  &  \makecell[c]{0.287***\\(0.028)}  &  \makecell[c]{0.724***\\(0.028)}  &  \makecell[c]{0.745***\\(0.028)}  & 0.027 & 0.589 & 0.000 & 0.000 \\
\qquad Action constant &  \makecell[c]{0.133***\\(0.020)}  &  \makecell[c]{0.081***\\(0.017)}  &  \makecell[c]{0.952***\\(0.013)}  &  \makecell[c]{0.985***\\(0.006)}  & 0.047 & 0.023 & 0.000 & 0.000 \\
\quad \textit{Difference} &  \makecell[c]{-0.224***\\(0.032)}  &  \makecell[c]{-0.240***\\(0.031)}  &  \makecell[c]{0.190***\\(0.028)}  &  \makecell[c]{0.270***\\(0.029)}  & 0.711 & 0.049 & 0.000 & 0.000 \\
\textit{p-values: Model vs. Gamble}\\ \quad State Constant & 0.415 & 0.191 & 0.161 & 0.294 &  &  &  &  \\
\quad Action Constant & 0.074 & 0.309 & 0.481 & 0.469 &  &  &  &  \\\\
Subjects & 210 & 204& 210& 204 \\
Observations & 1680 & 1632& 1680& 1632 \\\bottomrule


\end{tabular}}
\end{center}
\end{table}

\label{app:UtilityFocusedIndividual}
We complement this aggregate analyis with individual-level analysis to ensure the aggregates do not mask important individual-level heterogeneity.

To do so, we define the following 12 types. A subject seeks to make either maxmax or maxmin choices in the Model frame and may have the same or the opposite objective in the Gamble frame. She might successfully choose according to her preferences throughout, she might be confused in the state-constant (but not in the action-constant) presentation mode and select the utility-minimizing option there, or she might be confused in the action-constant (but not in the state-constant) presentation mode. For rounds in which one alternative dominates another, both maxmax and maxmin decision makers prefer that alternative. We assign each subject to the type whose choices coincide with that subjects' choices on the largest number of rounds. In case a subjects' choices deviate from multiple different types by the same number of profiles, we calculate type frequencies by assigning equal probability mass to each these multiple types.

The modal subject, 62.7\%, makes maxmin choices throughout and are not systematically confused by any frame (65.2\% and 59.3\% of subjects in the low and high-stakes conditions, respectively). The next biggest group of subjects, 21.2\% make maxmin choices throughout, but are systematically confused by the state-constant frame where they inadvertently make maxmax choices (20.2\% and 22.2\$ in the low and high stakes conditions, respectively). Of the remaining subjects, 11.5\% are approximately uniformly distributed across the remaining types that do not make systematic mistakes, and merely 5.1\% are assigned to types that make systematic mistakes but have objectives other than minmin in both frames.


\FloatBarrier


\newpage

\section{Experiment instructions} \label{app:instructions}


Due to page restrictions, the complete instructions are available online at
\url{https://narrativesstudy.s3.us-east-2.amazonaws.com/instructions.pdf}


\ifx\undefined\bysame
\newcommand{\bysame}{\leavevmode\hbox to\leftmargin{\hrulefill\,\,}}
\fi
\begin{thebibliography}{xx}

\harvarditem[Pearl]{Pearl}{2009}{Pearl.2009}
{\bf Pearl, Judea}, {\it Causality}, Cambridge University Press, 2009.

\harvarditem[Spiegler]{Spiegler}{2020}{Spiegler.Review}
{\bf Spiegler, Ran}, ``Behavioral implications of causal misperceptions,'' {\it
  Annual Review of Economics}, 2020, {\it 12}, 81--106.

\end{thebibliography}


\end{appendix}

\end{document}
