\documentclass[smallheadings]{article}
\usepackage[latin9]{inputenc}
\usepackage[margin=1in]{geometry}
\usepackage{stix}

\usepackage{placeins}
\geometry{verbose}
\usepackage{array}
\usepackage{booktabs}
\usepackage{calc}
\usepackage{url}
\usepackage{pdfpages}
\usepackage{tablefootnote}
\usepackage{amsmath}

\usepackage{graphicx}
\usepackage{setspace}
\usepackage{esint}
\usepackage[authoryear]{natbib}
\setstretch{1.5}
\usepackage[unicode=true,
 bookmarks=false,
 breaklinks=false,pdfborder={0 0 1},backref=section,colorlinks=false]
 {hyperref}
\usepackage{breakurl}

\makeatletter


\providecommand{\\}{\\}

\newcommand{\lyxdot}{.}


\usepackage{amsfonts}
\usepackage{mathrsfs}
\usepackage{mdwlist}
\usepackage{soul}
\usepackage{eucal}
\usepackage{amsthm}
\usepackage{epstopdf}
\usepackage{dsfont}
\usepackage{bm}
\usepackage{enumerate}
\usepackage{caption}

\usepackage{multirow}
\usepackage{pdflscape}
\usepackage{float}
\usepackage{minitoc}
\usepackage{soul}


\usepackage{textcomp}


\usepackage{bigints}


\usepackage{multibib}
\newcites{appendix}{Appendix References}


\renewcommand{\thepart}{}
\renewcommand{\partname}{}

\setcounter{MaxMatrixCols}{10}


\definecolor{blue}{rgb}{0,0,1}
\definecolor{purple}{rgb}{0.7,0,0.7}
\floatplacement{table}{H}
\clubpenalty = 10000
\widowpenalty = 10000
\DeclareMathOperator{\plim}{plim}
\DeclareMathOperator{\sgn}{sgn}
\renewcommand{\theenumi}{(\roman{enumi})}
\renewcommand{\labelenumi}{\theenumi}
\newcommand{\bmath}{\begin{eqnarray}}
\newcommand{\emath}{\end{eqnarray}}
\newcommand{\bmathnn}{\begin{eqnarray*}}
\newcommand{\emathnn}{\end{eqnarray*}}
\newcommand{\bi}{\begin{itemize}}
\newcommand{\ei}{\end{itemize}}
\newcommand{\I}{\mathcal{I}}
\newcommand{\var}{\mathrm{var}}
\newcommand{\logit}{\mathrm{logit}}
\newcommand{\abs}{\mathrm{abs}}
\newcommand{\red}{\color{red}}
\newcommand{\purple}{\color{purple}}

\definecolor{darkgreen}{rgb}{0.1, 0.45, 0.1}

\newcommand{\green}{\color{darkgreen}}
\newcommand{\black}{\color{black}}
\newcommand{\gray}{\color[rgb]{0.7,0.7,0.7}}
\theoremstyle{plain}
\newtheorem{proposition}{Proposition}\newtheorem*{example}{Example}\setcounter{section}{0}
\newtheorem{theorem}[proposition]{Theorem}\newtheorem*{lemma}{Lemma}\newtheorem{corollary}[]{Corollary}\newtheorem{result}{Result}\newtheorem{hypothesis}{Hypothesis}\theoremstyle{definition}
\newtheorem{definition}{Definition}\newtheorem*{assumption}{Assumptions}\newtheorem*{remark}{Remark}\theoremstyle{plain}
\DeclareMathOperator*{\argmax}{argmax}
\DeclareMathOperator*{\argmin}{argmin}
\DeclareMathOperator{\core}{Core}
\DeclareMathOperator{\cov}{cov}


\renewcommand{\thesection}{\Roman{section}}
\renewcommand{\thesubsection}{\Alph{subsection}}
\renewcommand{\thesubsubsection}{\arabic{subsubsection}}
\makeatletter    \def\p@subsection{\thesection.}   \makeatother
\makeatletter    \def\p@subsubsection{\thesection.\thesubsection.}   \makeatother


\makeatother

\begin{document}

\title{{\huge{}{}{Evaluating Deliberative Competence: \\ A Simple Method
\\ with an Application to Financial Choice}}}

\author{Sandro Ambuehl, B. Douglas Bernheim, and Annamaria Lusardi\thanks{Ambuehl: University of Zurich, Department of Economics, Bl\"uemlisalpstrasse
10, 8006 Z\"urich, Switzerland, sandro.ambuehl@econ.uzh.ch. Bernheim:
Department of Economics, Stanford University, 579 Serra Mall, Stanford,
CA 94305, bernheim@stanford.edu. Lusardi: The George Washington University
School of Business, 2201 G Street, NW, Suite 450E, Washington, DC
20052, alusardi@email.gwu.edu. This paper incorporates some material contained in previous working papers
entitled ``Financial education, financial competence, and
consumer welfare'' and ``A method for evaluating the quality of
financial decision making, with an application to financial education''.
We thank Stefano DellaVigna (the editor), Charles Sprenger, Steven Sheffrin, Glen Weyl, our referees, and
participants at various conferences and seminars for helpful comments
and suggestions. Fulya Ersoy and David Zuckerman provided excellent research assistance. We appreciate funding from the TIAA-CREF, the Alfred P. Sloan Foundation, and the Department of Economics at Stanford University. Experiment
A was approved in Stanford IRB protocol 29615, Experiment B in University
of Toronto REB protocol 34511.}\vspace{0.5cm}
 }

\maketitle
\vspace{-0.5cm}

\begin{abstract}

We examine methods for evaluating interventions designed to improve decision-making quality when people misunderstand the consequences of their choices. In an experiment involving financial education, conventional outcome metrics (financial literacy and directional behavioral responses) imply that two interventions are equally beneficial even though only one reduces the average severity of errors. We trace these failures to violations of the assumptions embedded in the conventional metrics. We propose a simple, intuitive, and broadly applicable outcome metric that properly differentiates between the interventions, and is robustly interpretable as a measure of welfare loss from misunderstanding consequences even when additional biases distort choices.


\end{abstract}
\vspace{0.5cm}

\vspace{0.2cm}


\doparttoc
\faketableofcontents

\part{}


\newpage{}

\setcounter{page}{1}


Mounting evidence documents low-quality decision making in a wide range of policy-relevant contexts, such as household finance \citep{Beshearsetal.Review}, health insurance \citep{Loewensteinetal.HealthInsurance}, and the consumption of durable goods \citep{GrubbOsborne,AllcottTaubinsky.Lightbulbs}. An important class of corrective policies seek to address the resulting inefficiencies by influencing consumers' decisions without changing either their choice sets or the factual information to which they have access (\emph{opportunity-neutral} interventions).
Examples include (1) programs that seek, through education and training, to equip decision makers with the knowledge and skills necessary to process and interpret naturally occurring information, (2) efforts to ``nudge'' people toward better decisions through salient or graphic labels and other measures that attempt to change the ``framing'' of decision tasks   \citep[also known as \emph{choice architecture}, ][]{NudgeBook}, and (3) measures that influence decision making by altering the nature or frequency of personal interactions with professional advisers, family members, or peers. Evaluating the welfare effects of such policies presents challenges. For example, if we proceed from the premise that financial decisions are often misguided, then we cannot evaluate financial well-being by deferring indiscriminately to choices, as standard welfare analysis prescribes. Instead, we require measures of decision quality. Moreover, suitable methods must account for the various ways in which different biases (and the policies that seek to address them) counteract or reinforce each other. The purpose of this paper is to devise a simple method of evaluation that overcomes these challenges.

While the approach we develop targets a variety of potential applications, for the sake of concreteness we focus on the domain of personal financial decisions. Three features of this setting make it ideal for our purposes. First, because financial illiteracy is pervasive \citep[see, e.g.,][]{Hastingsetal.Review,LusardiMitchellReview}, poor decision making is likely widespread. Second, it is easy to imagine opportunity-neutral interventions that could potentially improve the quality of financial decision making. Third, financial education is already the subject of a vast academic literature \citep[see][for reviews]{Fernandesetal.Meta,Kaiseretal.FinEdReview}, and has become a ubiquitous public policy tool.\footnote{Most OECD member countries, as well as India and China, have implemented policies to enhance financial education. Altogether, these initiatives seek to reach more than five billion people in sixty countries \citep{OECDpolicyHandbook}. The European Commission recently identified financial literacy as a key objective in its \emph{capital markets union 2020 action plan} \citep{EUactionPlan} and, in cooperation with the OECD, has launched the development of a financial competence framework for its 27 member states \citep{OECDfinLitEU}. Additionally, in 2013, private non-profit organizations spent nearly half a billion dollars on the provision of financial education in the US alone \citep{CFPBprivate}.}

Two methods for evaluating financial education interventions dominate the literature \citep[see][for reviews]{Hastingsetal.Review, LusardiMitchellReview,Beshearsetal.Review, Kaiseretal.FinEdReview}. The first investigates whether an intervention changes behavior in a direction that counteracts a suspected bias. For example, on the assumption that people save too little, some studies ask whether financial education leads them to save more \citep[e.g.][]{BernheimGarrettMaki,BernheimGarrett,ColeShastry,Bruhnetal.Mexico,Bruhnetal.}.\footnote{Other directional biases include the tendency to choose low-deductible health insurance plans \citep{Sydnor,Loewensteinetal.HealthInsurance}, na\"ive diversification \citep{BenartziThaler2001}, and investors' tendency to exit the market in downturns \citep{Vanguard.AdvisorsAlpha}.} A second approach uses exam-style questions to assess comprehension of financial principles  \citep[e.g.][]{Mandell,MandellKlein, Walstadetal.,Carpenaetal.,Collins,Heinbergetal.FiveSteps,Lusardietal.VisualTools, Bruhnetal.Mexico,Bruhnetal.}. Both methods are easy to implement, but neither measures the quality of decision making directly.

Still, if education not only improves financial literacy but also shifts behavior in a direction that counters a presumed bias \citep[as the recent meta-analysis of][suggests]{Kaiseretal.FinEdReview}, there is a tendency to infer that it achieves the right changes for the right reasons and consequently makes consumers better off.

The first part of this paper (Section \ref{experiment}) explores these evaluative metrics in the context of a laboratory experiment, and then develops our alternative. Experimental methods are widely used in research on the quality of decision making both in the lab and in the field because they provide the ability to control the environment tightly and to collect the type of detailed measurements required for rigorous inference about welfare \citep[see, e.g.][for reviews]{BernheimTaubinsky,Beshearsetal.Review}.\footnote{A reasonable question is whether, with larger real-life stakes, people might deliberate more carefully and at greater length, and might be more likely to employ analytic tools or seek advice. In practice, most people make financial decisions without qualified assistance \citep{BernheimInMitchellSchieber,BenartziThaler.RetirementSavings,LusardiMitchellOUP}, spend less than an hour making retirement saving decisions \citep{BenartziThaler.RetirementSavings}, fail to develop explicit financial or retirement plans  \citep{LusardiMitchellOUP}, rely on simple rules of thumb \citep{Bernheim.UnsophisticatedPlans}, and respond to non-substantive nudges  \citep{MadrianShea,BernheimPopovFradkin}. Notably, the fraction of subjects reporting that they did not seek advice in our experiment (three quarters) matches experience in the field \citep{LusardiMitchellOUP}. See also \citet{Enkeetal.Stakes} for general evidence that large stakes do not remove cognitive biases.} We focus on a fundamental concept in personal finance: compound interest. Some of our subjects receive financial education pertaining to compound interest, while others do not.  We use two educational interventions that closely follow a popular text on investing and differ from each other only in that one provides practice with individualized feedback, while the other does not.  We elicit subjects' reservation prices for interest-bearing investments (e.g. ``{We will invest \$$a$ in an account with $X$\% interest per day. Interest is compounded daily. We will pay you the proceeds in $t$ days.}''), and for each corresponding future payment stated transparently (e.g. ``{We will pay you \$$r$ in $t$ days.}''). We label the former \emph{complex framing} and the latter \emph{simple framing}.

Given the well-documented tendency for people to underestimate exponential growth (\emph{exponential growth bias}, \citealp{WagenaarSagaria,EisensteinHoch,StangoZinman,AlmenbergGerdes,LevyTasoff}), there is a strong presumption that effective education would increase the demand for interest-bearing assets. Subjects also take an incentivized test on compound interest that requires them to apply, rather than merely to recite, the principles they learned.

According to both conventional metrics, the two interventions are effective: they improve  performance on the exam-style questions, and they increase reservation prices for interest-bearing investments. Moreover, in each case, their effects are nearly identical. Such analyses appear to imply that the basic intervention significantly improves the quality of decision making, and that the addition of practice and feedback neither amplifies nor diminishes these benefits.

Next, we demonstrate that the inferences one would draw from the conventional metrics are, in fact, misleading. Because subjects performed equivalent valuation tasks with both complex and simple framing, we can assess the magnitudes of the biases resulting from misapprehension of compound interest. A subject who understands interest compounding and who applies that knowledge when making decisions will exhibit the same valuation for an interest-bearing investment as for the future payment it yields. For subjects with poor comprehension, valuations will differ across the frames.  Using these data, we show that the two interventions have profoundly different effects. For the ideal opportunity-neutral intervention, the treatment effect would be positively correlated with the initial bias: it would be larger for those who undervalue interest-bearing assets more severely. And yet, without practice and feedback, the intervention increases subjective valuations of interest-bearing assets across the board, even though subjects' initial biases are highly heterogeneous. It therefore leads some subjects to overcorrect for their initial biases, and it yields greater bias among those whose initial estimates of exponential growth are roughly correct or excessive.\footnote{\citet{Godaetal.EGB} and \citet{LevyTasoffMeta} also document considerable heterogeneity with respect to the perceived benefits of compounding. Relatedly, \citet{Harrisonetal.IndexInsurance} finds that increased take-up of seemingly beneficial products (index insurance) can lead to welfare losses due to subject heterogeneity. See also \citet{Oster}, who shows that health recommendations have the greatest impact on those who are already attuned to health issues, which raises the analogous possibility that treatment effects are negatively correlated with biases in that context.} In contrast, with practice and feedback, initial bias and treatment effects are indeed positively correlated: the intervention reduces the degree to which subjects undervalue complexly framed opportunities without increasing the prevalence or degree of overvaluation. The benefits of education are therefore unambiguous with practice and feedback, but not without it. Figure \ref{CDF} of Section \ref{welfare} encapsulates these findings.

Why do the conventional metrics lead us astray? When an improvement in knowledge accompanies a directionally appropriate shift in behavior, we tend to assume that the relationship is causal -- i.e., that behavior changes \emph{because} knowledge improves. Further analysis involving two additional treatments falsifies that assumption. For one treatment, we remove most of the substantive material from the intervention and maintain only its rhetorical elements (e.g. ``Albert Einstein is said to have called compound interest the most powerful force in the universe''). The resulting intervention largely reproduces the behavioral effect of the full intervention, but has almost no effect on measured comprehension. For the second treatment, we retain the substantive material and remove the rhetoric. That treatment roughly reproduces the effect of the original intervention on measured comprehension, but largely fails to alter behavior.\footnote{Related dissociations have been observed in other contexts. \citet{enkeZimmermann} shows that many people tend to neglect correlations in decision making, despite knowing how to account for them when prompted explicitly. \citet{taubinskyReesJones} find that many consumers underreact to excise taxes, even though they can properly compute tax-inclusive prices when prompted to do so.} Thus, without practice and feedback, the effects of the educational intervention on test performance arise mainly from the substantive elements of instruction, but its behavioral effects arise mainly from the rhetorical elements. These findings explain why the intervention can enhance test performance and shift behavior in the ``desired'' direction without improving the quality of decision making overall.

Our critique of the conventional evaluative metrics points to a simple alternative: compute the average distance between simply and complexly framed valuations in equivalent tasks (our proposed measure of \emph{deliberative competence}). This metric is the absolute value (or square) of the \emph{price-metric bias} studied in Behavioral Public Economics \citep[see][]{BernheimTaubinsky}. In the absence of biases other than consumers' misunderstanding of the consequences of their choices, the price-metric bias is interpretable as the ideal price correction (tax or subsidy). As we explain, it has another interpretation that renders it useful for welfare analyses of opportunity-neutral policies: intuitively, the absolute size of the bias gauges the dollar denominated harm the consumer might suffer due to errors resulting from misunderstanding her options. Additional analysis of the data from our experiment reveals that our measure of deliberative competence captures the essential patterns that conventional evaluative metrics miss: it implies that the intervention is effective if it includes practice and feedback, but is not effective if these elements are absent.

The second part of this paper (Section \ref{definingFinancialCompetence}) supplements our intuitive welfare arguments with formal analysis. Our first objective is to establish that deliberative competence, as we define it, is  interpretable as a measure of the welfare loss resulting from poor comprehension of the consequences that follow from choice options (rather than merely as a measure of bias). Our second objective is to study the robustness of this measure with respect to the possibility that other biases may counteract or reinforce the effects of poor comprehension.

We begin by providing theoretical foundations for interpreting deliberative competence as a welfare measure under the restrictive assumption that the consumer's decision-making apparatus involves no defects other than the misunderstandings that the pertinent policy interventions target. Loosely, this assumption allows us to treat choices in the simple frame as revealing `true' preferences. Methodologically, our analysis of this special case parallels other recent work in Behavioral Public Economics, in that we evaluate naturalistic choices based on the decisions consumers would make if they properly understood the relationships between options and consequences.\footnote{For foundations, see \citet{BernheimRangelWelfare}, \citet{BernheimWelfareReview}, \citet{BernheimWelfareReview2}, \citet{BernheimTaubinsky}, \citet{bernheim2021defense}. \citet{BernheimTaubinsky} also survey applications.} In the case of our experiment, the complexly framed tasks are naturalistic, and the simply framed tasks avoid misunderstandings associated with compound interest.

An obvious limitation of the analysis summarized in the last paragraph is that the consumer's decision-making apparatus may well involve additional defects. If other biases contaminate simply framed choices, those choices may not provide valid normative benchmarks. Such possibilities raise \emph{second-best} issues in the sense of \citet{LipseyLancaster}. Concretely, in our experiment, considerations such as present bias and excessive suspicion concerning the likelihood the experimenter will renege on promised future payments may suppress simply framed valuations. Arguably, a policy that overcorrects for exponential growth bias is beneficial because it counteracts these additional biases.\footnote{Similar issues arise elsewhere in Behavioral Public Economics. For example, as noted by \citet{BlumenthalVolpp}, accurate information about the caloric content of food can exacerbate excessive consumption if people tend to overestimate calories. Likewise, according to \citet{Downsetal.AERPP}, ``dietary information is likely to improve self-protective behavior only if existing biases encourage unhealthy eating, but the reverse is equally likely. When it comes to smoking, for example, there is evidence that smokers tend to overestimate the health risks, in which case providing risk information could undermine their motivation to quit.'' } We reject this argument on the grounds that it implicitly addresses the wrong question. Specifically, it introduces additional biases without considering the interventions policy makers might use to address them (e.g., for present bias, commitment options and corrective taxation).

Ideally, one would account for second-best concerns by analyzing all biases and all associated policy options comprehensively. Unfortunately, we lack the requisite comprehensive understanding of human behavior to proceed in this manner. Practical considerations necessitate a compartmentalized approach, which is precisely what one finds in the existing literature on Behavioral Public Economics.

The literature does not, however, offer a fully satisfactory theoretical framework for conducting compartmentalized welfare analysis. The typical paper focuses on one or two biases at a time and assumes (usually implicitly) that the process generating the observed data is otherwise free of defects -- in effect, that any other biases are either orthogonal or have been resolved. In an analogous framework involving multiple distortions, \citet{LipseyLancaster} demonstrated that this approach to welfare analysis is conceptually fraught. Pasting together a collection of such solutions does not generally yield an overall optimum. We fill this gap in the literature by proposing the notion of \emph{idealized welfare analysis}, which acknowledges that other biases infect the available data, but assumes that remedies for them will be forthcoming. In contrast to the usual compartmentalized approach, this perspective allows the analyst to arrive at an overall solution to multiple interacting problems by solving them one at a time.

We then demonstrate that deliberative competence is interpretable as a valid measure of the idealized welfare loss despite the presence of biases outside the scope of the analysis, even when relatively little is known about their nature and magnitudes. This result requires a restrictive separability assumption, but it encompasses many of the types of biases that might infect a study such as ours. We also address other concerns that could in principle limit the applicability of our approach, showing for example that a simple adjustment removes a class of potential confounds.

This paper contributes to several literatures. It adds to the growing literatures in Behavioral Public Economics and Behavioral Welfare Economics (cited above) by showing that, under specified conditions, simple transformations of the price-metric bias are robustly interpretable as measures of the welfare loss from poor decision making, suitable for evaluating opportunity-neutral interventions. With respect to the literature on financial education (also cited above), it documents important limitations of conventional evaluative metrics and offers an easily implemented alternative. It also contributes to a smaller methodological literature on measures of decision-making quality. Alternative measures include the frequencies of (a) either dominated or dominant choices \citep{ErnstFarrisKing,CalvetCampbellSodiniJPE,CalvetCampbellSodiniAER, Agarwaletal., BaltussenPost, Choietal.100Bills,Aufenageretal.,Bhargavaetal.DominatedHealthcarePlans}, or (b) WARP or GARP violations \citep{Afriat72,Karivetal.,EcheniqueLeeShum}. Those methods are not easily adapted to the types of decisions that provide the focus for applications such as ours. Consistency-based methods are unlikely to detect problems arising from a consistent misunderstanding (e.g., of compound interest), and dominance-based methods are inapplicable when the best choice depends on preferences.


\section{Experimental evaluations of financial education interventions}

\label{experiment}

For our application, we investigate
the efficacy of two educational interventions aimed at improving intertemporal
decision making by enhancing comprehension of compound interest. Interest compounding is one
of the most fundamental concepts in finance, and a core topic in the vast majority of courses and books on the principles of financial decision making.
We begin by describing the experiment (Section \ref{design}) and its implementation (Section \ref{preliminaryAnalysis}). After evaluating the educational interventions based on conventional metrics (Section \ref{conventionalMeasures}), we demonstrate that their implications are untenable and their underlying assumptions invalid (\ref{welfare}). We then propose, intuitively justify, and implement a simple alternative outcome metric and show that it yields sensible conclusions. Section \ref{theory} provides formal theoretical foundations for this alternative approach.


\subsection{Design}

\label{design}

Our investigation involves two experiments. Both consist of three
stages. First, subjects participate in a randomly assigned financial
education intervention. Second, subjects complete paired valuation
tasks. Third, subjects answer a battery of exam-style questions on
compound interest. Table \ref{overview} presents a schematic overview. Additional detail concerning each stage follows.

\begin{table}[htp!]
\caption{Experiments overview \label{overview}}
\setlength{\tabcolsep}{2pt}
\begin{tabular}{lcccccc}
\toprule
                                       & \multicolumn{4}{c}{Experiment A}                                & \multicolumn{2}{c}{Experiment B}       \\
                                       \cmidrule(r){2-5}\cmidrule(r){6-7}
                                       & Treatment        & Control     & Rhetoric-only & Subst.-only & Treatment  & Control                   \\
\midrule
\textit{Stage 1}: Education intervention        &                  &             &               &                &            &                           \\
\quad Topic                   &   Compound               & Index  &  Compound      & Compound   & Compound & Portfolio \\
				& interest & funds & interest & interest & interest & diversification \\
\quad Rhetoric                               & \checkmark       & \checkmark           & \checkmark    &                & \checkmark & \checkmark                         \\
\quad Substance                              & \checkmark       & \checkmark           &               & \checkmark     & \checkmark & \checkmark                         \\
\quad Practice and feedback &             &               &                & & \checkmark & \checkmark                \\ \\
\textit{Stage 2}: Valuation tasks               &  \checkmark                &    \checkmark         & \checkmark              &    \checkmark            &    \checkmark        &         \checkmark                  \\ \\
\textit{Stage 3}: Exam-style questions          &                  &             &               &                &            &                           \\
\quad Interest compounding                   & \checkmark       & \checkmark  & \checkmark    & \checkmark     & \checkmark & \checkmark                \\
\quad Index funds                            & \checkmark       & \checkmark  & \checkmark    & \checkmark     &            &                           \\
\quad Portfolio diversification              &                  &             &               &                & \checkmark & \checkmark            \\
\bottomrule
\end{tabular}
\end{table}


\paragraph{Stage 1: Education intervention.}

Using material from a popular investment guide, \emph{The Elements
of Investing: Easy Lessons for Every Investor} by \citet{MalkielEllis},
we produced an instructional video on compound interest that covers
the topic through a narrated slide presentation.\footnote{We chose this approach because existing research indicates that financial
education videos are generally more effective than written text \citep{Lusardietal.VisualTools}.} The narration is verbatim from the text (with a few minor adjustments),
while the slides summarize key points. Stylistically, the video resembles
those offered through the popular educational internet platform \emph{www.khanacademy.org.}\footnote{While the intervention is brief, it is important to bear in mind that
financial education in the workplace is also brief. A meta-analysis
by \citet{Fernandesetal.Meta} finds that the average financial education
program involves only 9.7 hours of instruction. That time is divided
among a long list of complex topics. For example, \citet{SkimmyhornBootcamp}
reports that a financial education program used by the U.S. military
covers compound interest, the focus of our current study, along with
a collection of several more complex topics -- retirement concepts,
the Thrift Savings Plan, military retirement programs, and investments
-- all within a single two-hour session.}

For Experiment A, the video constitutes the entire Treatment. For
Experiment B, we split the video into three parts, each of which is
followed by practice problems with automated, individualized feedback. If a subject
provides an answer that is mistaken but consistent with the calculation
of simple interest, the intervention mentions the likely source of
the error, explains why the answer is wrong, and suggests how to
get started with a calculation of the correct response. There are
six practice questions in total. For the first four, subjects who
do not answer correctly in three attempts move on to the next part
of the Treatment. For the last two questions, subjects still receive
feedback, but they must select the unique correct answer from 13 options
before they can continue.\footnote{A single subject dropped out of the study when attempting to answer
these questions.} We reproduce the complete practice stage in Appendix \ref{practiceProblems}.

Subjects in the Control conditions for each experiment participate
in interventions that are stylistically similar and that require comparable
amounts of cognitive effort as the respective Treatments.
In Experiment A, subjects watch a video based on a section about index
funds from the same investment guide. Experiment B employs a Control
condition concerning portfolio diversification that also contains
practice problems with personalized feedback, based on \citet{Malkiel.RandomWalk},
supplemented with material from \citet{MalkielEllis}. Neither control intervention mentions compound interest or the time value of money.


In order to isolate the features of the intervention that affect test
scores and behavior, Experiment A employs two additional treatments
that dissect the main Treatment into its constituent parts. The complete
educational module begins with a simple explanation of compound interest
illustrated through a standard iterative calculation. The remainder of the text consists of two components:
\begin{enumerate}
\item Substantive content. An explanation of a simple, memorable, and potentially valuable heuristic,
the rule of 72, along with five illustrative applications.\footnote{We used this particular investment guide in part because it teaches
a useful quantitative heuristic. Some investment guides and educational
interventions cover this topic without offering useful quantitative
tools.} The rule of 72 is a method for approximating an investment's doubling
period; one can also use it to approximate the growth in an investment's
value over a fixed holding period. It states that the percentage interest
rate on an investment multiplied by the number of periods required
for its value to double equals 72 (approximately).
\item Rhetorical material. The section opens with the observation that ``Albert
Einstein is said to have described compound interest as the most powerful
force in the universe.'' It provides various anecdotes concerning
small investments that grew to impressive sums (in some cases millions
of dollars) over long time periods. These anecdotes do not include
any computations, and hence are not helpful for understanding the
mechanics of compound interest. The text also explicitly exhorts readers
to behave frugally, and characterizes compounding as a ``miracle.''
\end{enumerate}


\noindent Subjects in the \emph{Substance-Only} treatment view a video
covering all of the substantive material, but omitting exhortations
and atmospheric quotes.\footnote{In cases where it was impossible to remove sentences containing rhetorical
material, we substituted neutral language. For instance, the first
example of compounding presented in the original text is preceded
by the transitional question, ``Why is compounding so powerful?''
In the Substance-Only-treatment, we substituted the question, ``How
does compounding work?''} In contrast, subjects in the \emph{Rhetoric-Only} treatment view
a video containing all of the rhetorical material, as well as the
introductory explanation of compound interest, but omitting all material
on the rule of 72.

\paragraph{Stage 2: Valuation tasks}

\begin{figure}[htbp!]
\caption{Elicitation of valuations.  \label{MPL}}
\begin{centering}
\fbox{
\begin{tabular}{lp{0.35\textwidth}p{0.35\textwidth}}
 & \textsf{You will get the specified dollar amount within two days from
today}  & \textsf{We will invest \$$a$ in an account with $X$\% interest per
day. Interest is compounded daily. We will pay you the proceeds in
$t$ days.} \\
\midrule
\textsf{\$20}  & \multicolumn{1}{c}{$\bigcirc$} & \multicolumn{1}{c}{$\bigcirc$}\\
\textsf{\$18}  & \multicolumn{1}{c}{$\bigcirc$} & \multicolumn{1}{c}{$\bigcirc$}\\
\textsf{\$16}  & \multicolumn{1}{c}{$\bigcirc$} & \multicolumn{1}{c}{$\bigcirc$}\\
\ldots{}  & \multicolumn{1}{c}{\ldots} & \multicolumn{1}{c}{\ldots}\\
\textsf{\$2}  & \multicolumn{1}{c}{$\bigcirc$} & \multicolumn{1}{c}{$\bigcirc$}\\
\textsf{\$0}  & \multicolumn{1}{c}{$\bigcirc$} & \multicolumn{1}{c}{$\bigcirc$}\\
\end{tabular}}
\par\end{centering}
\footnotesize
\textbf{\textit{Notes:}} The figure displays the first decision list of a round with complex framing. Payment amounts on the left include all even dollar amounts between and including \$0 and \$20. For simply framed choices, the text on the upper right is replaced by ``We will pay you $R$ in $t$ days.''
\end{figure}


Subjects perform 10 pairs of valuation tasks. Each task elicits an equivalent
current dollar value for a reward $r$ to be received in either 36
or 72 days. With \emph{simple framing}, we describe the reward as
follows: ``We will pay you \$$r$ in $t$ days.\textquotedblright \ With
\emph{complex framing}, we describe the same reward in terms of a
return on an initial investment, as follows: ``We will invest \$$a$
at an interest rate of X\% per day. Interest is compounded daily.
We will pay you the proceeds in $t$ days.\textquotedblright \ Subjects
make two sets of choices pertaining to each future reward, one with
simple framing, the other with complex framing. For each frame $f$, we elicit each subject $j$'s immediate dollar
equivalent of a payment $R$ received in $t$ days, using the iterated
multiple price list method with a resolution of \$0.20 \citep{AndersenMPL}.
Figure \ref{MPL} presents an example of the first decision list of
a round for a complexly framed prospect.

We randomize the order of the valuation tasks at the subject level.
Subjects are not told that some of the tasks are substantively equivalent,
and they typically do \emph{not} perform equivalent simply and complexly
framed tasks consecutively.


Table \ref{DecisionProblems} lists the parameters $t$, $X$, $a$,
and $R$ used for the paired valuation tasks. We chose time horizons
of 36 and 72 days to simplify applications of the rule of 72.\footnote{We use two different time frames so subjects face a greater variety
of decision problems, and hence are less likely to consider successive
problems highly similar.} This feature increases the likelihood our study will find {beneficial} behavioral
effects of the Treatment interventions.


\begin{table}[hbt]
\caption{Decision problems. \label{DecisionProblems} }
\begin{centering}
\begin{tabular}{cccc}
\toprule
Future Reward $R$  & Investment Amount $a$  & Daily Interest Rate $X$  & Number of Doublings \\
\midrule
 &  &  & \\
\multicolumn{4}{c}{Duration: 72 days}\\
\midrule
\$20  & \$10  & 0.01  & 1 \\
\$18  & \$4.5  & 0.02  & 2 \\
\$16  & \$2  & 0.03  & 3 \\
\$14  & \$0.9  & 0.04  & 4 \\
\$12  & \$2  & 0.025  & 2.5 \\
 &  &  & \\
\multicolumn{4}{c}{Duration: 36 days}\\
\midrule
\$20  & \$10  & 0.02  & 1 \\
\$18  & \$4.5  & 0.04  & 2 \\
\$16  & \$2  & 0.06  & 3 \\
\$14  & \$0.9  & 0.08  & 4 \\
\$12  & \$2  & 0.05  & 2.5 \\
\bottomrule
\end{tabular}
\par\end{centering}
\footnotesize
\textbf{\textit{Notes:}} \emph{Number of doublings} is the number of times
the initial investment doubles over the investment horizon according
to the rule of 72. Final amounts are calculated using the rule of
72. Exact final amounts differ by no more than \$0.80, except for
the 4\% interest rate over 72 days, where the rule understates the
future value by \$1.16. Our analysis controls for these differences.
\end{table}


\paragraph{Stage 3: Exam-style questions.}


The final stage of each experiment is an incentivized test consisting
of the five questions about compound interest listed in Table \ref{TestQuestions},
as well as five questions about the material covered in the video
shown to the control group.\footnote{The test questions for the material in the control interventions are
in Appendix Table \ref{testQuestionsControl}. We randomize the order
of all ten test questions at the subject level.} The exam-style questions have right and wrong answers which we specifically ask subjects to determine. These features distinguish the exam-style questions from the valuation tasks: for the latter, best choices depend on preferences, and we do not explicitly ask subjects to compute future values. Subjects are aware that test performance is incentivized prior to participating
in the education intervention.


\begin{table}[htbp!]
\caption{Exam-style questions. }
\label{TestQuestions}
\begin{centering}
\hrulefill{}
\par\end{centering}
\textsf{\small{}{}Q1. If the interest rate is 10\% per year (interest
is compounded yearly), how many years does it take until an investment
doubles?}{\small{}{}}\\
 \emph{\small{}{}7 years, 7.2 years, 7.4 years, 7.8 years, 8 years}{\small{}{}}\\

\textsf{\small{}{}Q2. If somebody tells you an investment should
double in four years, what rate of return (per year) is he promising?}{\small{}{}
}\\
 \emph{\small{}{}15\%, 16\%, 17\%, 18\%, 19\%, 20\%}{\small{}{}}\\

\textsf{\small{}{}Q3. If the interest rate is 7\% per year (interest
is compounded yearly), about how long does it take until an investment
has grown by a factor of four (i.e. is four times as large as it was
originally)?}{\small{}{} }\\
 \emph{\small{}{}About 5 years}{\small{}{} to }\emph{\small{}{}about
40 years}{\small{}{}, in steps of 5 years.}\\

\textsf{\small{}{}Q4. Paul had invested his money into an account
which paid 9\% interest per year (interest is compounded yearly).
After 8 years, he had \$500. How big was the investment that Paul
had made 8 years ago?}{\small{}{}}\\
 \emph{\small{}{}\$200}{\small{}{} to }\emph{\small{}{}\$400}{\small{}{}
in steps of \$10}\\

\textsf{\small{}{}Q5. If an investment grows at 8 percent per year
(interest is compounded yearly), by how much has it grown after 4
years? }{\small{}{} }\\
 \emph{\small{}{}By 30\%}{\small{}{}, to }\emph{\small{}{}by 40\%}{\small{}{}
in steps of one percentage point.}\\
 \vspace{-0.5cm}

\begin{centering}
\hrulefill{}
\par\end{centering}

\footnotesize
\textbf{\textit{Notes:}} Questions were presented in random order and
intermingled with the five questions concerning material covered in
the respective control interventions.
\end{table}


\paragraph{Payment}

All subjects know from the outset that they will be paid according
to one randomly selected decision from the entire experiment with
75\% probability, and according to their performance on the test with
25\% probability, in which case they earn \$1 for each of the
ten questions answered correctly. We guarantee subjects that they will receive payment within at most two days of the promised date. The instructions emphasize that subjects ``should
make every decision as if it is the one that counts, because it might
be!'' In addition to the incentive payment, each subject receives
a completion payment of \$10.


\subsection{Implementation and preliminary analysis}

\label{preliminaryAnalysis}

We conducted both experiments through Amazon
Mechanical Turk (AMT), an online labor market. Our primary reason for employing this subject pool is that the typical
member has a poor understanding of compound interest.\footnote{We opted for this subject pool after pilot experiments at Stanford
and at The Ohio State University indicated that the vast majority
of student subjects at these universities correctly apply compound
interest calculations.} Also, this group resembles the target populations for many financial
education programs in terms of age and income. We ran Experiment A in eight sessions with a total
of 504 subjects during April and May 2014 and Experiment B in two
sessions with a total of 401 subjects in October 2018, all on weekday
mornings. Each subject participated in exactly one treatment of exactly
one experiment. We restricted participation to subjects who resided in the US and
were at least 18 years of age. We took several precautionary measures
to ensure that subjects were able to view the videos and that they
would pay attention. We describe these measures and additional implementation details in
Appendix \ref{experimentDetails}.\footnote{A replication package is available in \citet{replicationData}.}


Each experiment begins with a two-and-a-half minute video recording
of one of the authors (Bernheim) vouching that we will pay subjects
exactly the amount we promise within at most two days of the promised
date,\footnote{The video invites subjects to click a link to the authors' homepages
so they can verify the authenticity of the video. Before participating
in the main stages of the experiment, subjects also complete an unincentivized
questionnaire concerning demographics, as well as a standard battery
of five questions designed to assess financial literacy \citep{LusardiMitchellOrdinaryConsumers}.
We reproduce the five questions in Appendix Table \ref{financialLiteracyQuestionnaire}.} and ends with a number of non-incentivized questions about subjects'
decision making.\footnote{Appendix Table \ref{selfReportsTable} details the questions and responses
by experiment and treatment. } We intentionally place no restriction on the use of resources such
as calculators, the internet, or personal advice when making decisions,
as subjects always have those options when making real-world decisions.
Roughly a quarter of our subjects in Experiment A and less than eight
percent in Experiment B report using such resources when completing
the incentivized test, a fraction that does not vary meaningfully
across treatments.
Subjects could complete all valuation tasks at their own pace. Subjects
in Experiment A took 62 minutes on average (s.d. 22 minutes) to complete
the study, while those in Experiment B took 75 minutes (s.d. 25 minutes).
The difference in completion times is attributable to the longer interventions
in Experiment B.

On average, subjects earned \$22.86 in Experiment A and \$23.14 in
Experiment B. In both experiments, earnings include a fixed \$10 participation
fee and range from a low of \$10 to a high of \$30.47. In comparison,
AMT participants typically earn about \$5 per hour \citep{MasonSuri,Haraetal.TurkEarnings}.


\paragraph{Multiple switching.}

Any subject with coherent preferences will switch her choice from
the immediate payment to the future reward at most once within a single
price list. Hence, we informed subjects that ``most people begin
a decision list by preferring the option on the left and then switch
to the option on the right.'' In practice, 9.7\% of subjects (49
of 504) in Experiment A and 13.2\% of subjects (53 of 401) in Experiment
B switched two or more times in at least one price list. This number
does not differ significantly across treatments ($p=0.85$ and $p=0.18$
for Experiments A and B, respectively). In laboratory studies of risky
choices by undergraduate subjects (such as \citealp{HoltLaury}),
the corresponding figure typically falls in the range of 10 to 15\%.
Following the usual convention (see, for example, \citealp*{Harrisonetal.}),
we focus attention on the 803 subjects who respected monotonicity.

\paragraph{Demographics.}

While our conclusions about the validity of methods for assessing
decision quality do not depend on the demographic composition of our
sample, we note that our participants are slightly more financially
literate than other pools of U.S. subjects (see \citealp{LusardiMitchellOrdinaryConsumers},
and \citealp{LusardiRepresentative}). There are also differences
between the samples used for Experiments A and B, possibly due to
changes in the composition of the AMT subject pool.\footnote{In addition, for Experiment B, we required subjects to have at least
500 completed Human Intelligence Tasks (HITs) and at least a 98\%
approval rating.} Subjects in Experiment A are on average five years younger, the fraction
of women is 6.6 percentage points lower, and their median income is
30\% lower. Appendix \ref{Demographics} lists all demographic
details. Due to the differences between the samples, we analyze each experiment
separately, without making direct statistical comparisons. Appendix
\ref{demoControlsAppendix} complements our main analysis by combining
data across experiments. It shows that the relative performance of
the two compound-interest interventions are driven by their differences
rather than by divergences between the subject pools.

\paragraph{Randomization and attrition}

Randomization into treatments was successful. Of the 68 $F$-tests
we perform to assess the differences in demographic characteristics
across treatments and experiments (one for each characteristic and
each experiment), two are significant at the 5\%-level, and five more
are significant at the 10\% level (see Appendix \ref{Demographics}).
These figures are well within the expected range.

Because we conducted the experiment over the internet, attrition is
a possibility. In Experiment A, we find it to be negligible and unrelated
to the treatments. Only four subjects who reached the stage at which
they may have viewed a treatment video failed to complete the study
(compared to 504 who completed it).\footnote{A larger number of subjects quit before reaching that stage, but that
type of attrition is necessarily independent of the treatment, and
hence largely innocuous.} In Experiment B, two subjects in the Control condition and nine subjects
in the Treatment condition began but did not finish the intervention.
While these magnitudes are small compared to the sample size of 401
subjects, they differ statistically at the 5\%-level.


\subsection{Assessment based on conventional outcome measures}

\label{conventionalMeasures}

In this section, we analyze the effects of the Treatment interventions
in each experiment using the two conventional metrics, exam-style
questions that test comprehension, and directional effects on behavior. We defer the analysis of the Substance-Only and Rhetoric-Only treatments
of Experiment A to Section \ref{welfare}.\footnote{All results reported in this section are robust with respect to various statistical controls and alternative specifications. For details, see Appendix \ref{Robustness}.}


\paragraph{Effects on tested knowledge}

\begin{figure}[h]
\caption{Assessment by conventional methods.  \label{figConventional}}
\label{trad} \begin{centering}
\begin{tabular}{cc}
A. Test scores  & B. Valuations in complex frame \\
\includegraphics[width=0.5\textwidth]{graphs/testScores}  & \includegraphics[width=0.5\textwidth]{graphs/deltaComplex} \\
\end{tabular}
\par\end{centering}
\footnotesize
\textbf{\textit{Notes:}} Panel A: Fraction of correct answers
on the five exam-style questions about compound interest. Panel B:
Mean valuation of complexly framed future payments, $r^{i,R,t}_c$, rescaled to range from 0 to 100 (percentage points). Whiskers indicate 95\% confidence intervals. Panel B includes multiple observations per subject; standard errors are clustered on the subject level.
\end{figure}

Panel A of Figure \ref{figConventional} shows how the treatments
affect subjects' average scores on the five test questions pertaining
to compound interest. In the Control condition of Experiment A, the
average subject answers just under two of five, or 39\%, of the questions
correctly. The Treatment intervention increases the average score
by roughly 29 percentage points (1.4 additional correct answers),
to 68\%.\footnote{See Appendix \ref{individualTestQuestions} for the effects
on individual test questions.} The numbers in Experiment B are remarkably similar, with 37\% correct responses in the Control condition and 69\% in the Treatment condition.


\begin{table}[htbp!]
\setlength{\tabcolsep}{3pt}


\begin{centering}
\caption{Performance in exam-style test. }
\label{Test} \begin{tabular}{lcccc}
\toprule

 & (1)  & (2)  & (3)  & (4) \\
 & \multicolumn{2}{c}{Test score compounding} & \multicolumn{2}{c}{Test score control module}\\
 & \multicolumn{2}{c}{(out of 5)} & \multicolumn{2}{c}{(out of 5)}\\
\cmidrule(r){2-3} \cmidrule(r){4-5} Experiment  & A  & B  & A  & B \\
\midrule
 &  &  &  & \\
Levels  &  &  &  & \\
\hspace{0.4cm} \textit{Control}  & 1.963{*}{*}{*}  & 1.849{*}{*}{*}  & 3.284{*}{*}{*}  & 3.078{*}{*}{*} \\
 & (0.140)  & (0.110)  & (0.114)  & (0.101) \\
\hspace{0.4cm} \textit{Treatment}  & 3.406{*}{*}{*}  & 3.450{*}{*}{*}  & 2.226{*}{*}{*}  & 2.704{*}{*}{*} \\
 & (0.135)  & (0.092)  & (0.092)  & (0.098) \\
Difference  & 1.442{*}{*}{*}  & 1.601{*}{*}{*}  & -1.058{*}{*}{*}  & -0.374{*}{*}{*} \\
 & (0.194)  & (0.143)  & (0.146)  & (0.141) \\
 &  &  &  & \\
Observations  & 215  & 348  & 215  & 348 \\
\midrule

 &  &  &  & \\
\end{tabular}
\par\end{centering}
\footnotesize
\textbf{\textit{Notes:}} Each column displays the coefficients of a separate OLS regression.  Standard errors in parentheses. {*}{*}{*}$p<0.01,${*}{*}{*}$p<0.05,${*}$p<0.1$.
\end{table}

Columns 1 and 2 of Table \ref{Test} summarize these results and
present tests of equality between the Treatment and Control conditions
using OLS regressions. The treatment effects in both experiments are
highly statistically significant ($p<0.01$). Columns 3 and 4 establish
that the improvements in test performance are not due to effects of
the Treatment on general motivation or attention. If the effects were
spurious, we would find comparable effects for subjects' scores on
the five test questions concerning the Control interventions. On the
contrary, in each experiment, subjects in the Treatment condition
perform substantially worse on questions about the respective Control
intervention than subjects in the Control condition. The differences of 1.06 and 0.37 questions in Experiments A and B, respectively, are again highly statistically significant
($p<0.01$). The variation in effect size across experiments is attributable
to the fact that the two experiments employ different Control conditions
and different test questions about those conditions.

\begin{table}[htbp!]
\setlength{\tabcolsep}{3pt}
\caption{Valuations in complexly framed tasks}
\label{directionalBehavior}

\begin{centering}
\begin{tabular}{lcccccc}
\toprule
 & (1)  & (2)  & (3)  & (4)  & (5)  & (6) \\
Delay in days  & 72 and 36  & 72 and 36  & 72  & 72  & 36  & 36 \\
\cmidrule(r){2-3} \cmidrule(r){4-5} \cmidrule(r){6-7} Experiment  & A  & B  & A  & B  & A  & B \\
\midrule
 &  &  &  &  &  & \\
Levels  &  &  &  &  &  & \\
\hspace{0.4cm} \textit{Control}  & 58.949{*}{*}{*}  & 60.244{*}{*}{*}  & 56.401{*}{*}{*}  & 58.181{*}{*}{*}  & 61.496{*}{*}{*}  & 62.308{*}{*}{*} \\
 & (2.275)  & (2.115)  & (2.336)  & (2.134)  & (2.374)  & (2.169) \\
\hspace{0.4cm} \textit{Treatment}  & 73.261{*}{*}{*}  & 75.378{*}{*}{*}  & 70.629{*}{*}{*}  & 71.747{*}{*}{*}  & 75.893{*}{*}{*}  & 79.010{*}{*}{*} \\
 & (2.569)  & (1.886)  & (2.718)  & (1.904)  & (2.621)  & (2.001)\\
Difference  & 14.312{*}{*}{*}  & 15.134{*}{*}{*}  & 14.227{*}{*}{*}  & 13.566{*}{*}{*}  & 14.398{*}{*}{*}  & 16.702{*}{*}{*} \\
 & (3.431)  & (2.834)  & (3.583)  & (2.860)  & (3.536)  & (2.951) \\
 &  &  &  &  &  & \\
Observations  & 2,150  & 3,480  & 1,075  & 1,740  & 1,075  & 1,740 \\
Subjects  & 215  & 348  & 215  & 348  & 215  & 348\\
\bottomrule
\end{tabular}
\par\end{centering}
\footnotesize
\textbf{\textit{Notes:}} Valuations in the complex frame, $r^{i,R,t}_c$, in percentage
points. Each column
displays the coefficients of a separate OLS regression. Standard errors
in parentheses, clustered by subject. {*}{*}{*}$p<0.01,${*}{*}{*}$p<0.05,${*}$p<0.1$.
\end{table}


\paragraph{Directional behavioral effects}

\label{directionalBehavioralChange}


As we have noted, it is well-established that people, on average,
tend to underestimate the power of compound interest. Consequently,
following the approach sometimes adopted in the literature, one would
deem an intervention potentially welfare-improving if it causes subjects
to state higher valuations for investments that involve compound interest.
To make these comparisons, we express each valuation as a percentage
of the associated future reward. Formally, letting $\tilde{r}_{f}^{i,R,t}$
denote subject $i$'s valuation for the future reward $R$ received
with delay $t$ when presented in frame $f\in\{s,c\}$, we define
the normalized valuation as $r_{f}^{i,R,t}={\tilde{r}_{f}^{i,R,t}}/{R}$.\footnote{Because we elicit valuations in multiple price lists, they are interval coded. Throughout, we use the midpoint of the pertinent interval for analysis.
For further details on the iterated multiple price lists, see Appendix \ref{experimentDetails}.}


Panel B of Figure \ref{figConventional} displays normalized valuations
for complexly framed prospects by experiment and condition. Averaged
across reward amounts and timeframes, subjects in the Control condition
in both experiments regard the interest-bearing investment as equivalent
to an immediate payment of about 60\% of the future reward amount.
Subjects in the respective Treatment conditions state valuations that
are about 15 percentage points higher on average. This pattern is
precisely what one would expect if the Treatment in each experiment
successfully counteracts exponential growth bias. Given the magnitude
of the bias documented in the existing literature \citep{StangoZinman},
the size of the average treatment effect does not appear to raise
concerns about systematic overcorrection. Following the standard approach
in the literature, we would thus conclude that the Treatment interventions
substantially increase the quality of financial decision making, and
do so similarly across the two experiments.\footnote{Some prominent studies  do not directly observe behavior, but rely on self-reported data about behavior \citep[e.g.][]{Bruhnetal.}. Our experiments also elicit such data. At the end of the study, subjects report the number of decision problems for which  they explicitly calculated future values, whether they have used the rule of 72 in complexly framed problems, whether they have used it in simply framed problems, and whether they have used external help to make their decisions. As we find in our analysis on directional behavioral effects, our analysis of self-reports suggests that both interventions improve decisions and that they are similarly effective. See Appendix \ref{selfReports} for details.}

We formalize these findings by regressing normalized valuations in
the complex frame, $r_{c}^{i,R,t}$, on treatment indicators, clustering
standard errors by subject. Column 1 of Table \ref{directionalBehavior}
shows the results for Experiment A. In the Control condition, subjects
value an investment that compounds to one future dollar at 59 cents
on average. The Treatment condition raises average valuations by a
substantial and statistically significant increment, to 73 cents ($p<0.01$).
The estimates for Experiment B, displayed in column 2, are remarkably
similar. Subjects in the Control condition value a future dollar at
60 cents on average, and the Treatment condition increases this valuation
by a substantial and statistically significant increment, to 75 cents
($p<0.01$). Columns 3 - 6 of Table \ref{directionalBehavior} replicate
this analysis separately for decisions with 36-day delays and those
with 72-day delays, respectively. The same conclusions follow in each
instance.


Taken at face value, results based on these conventional metrics seem to imply that both versions
of the educational intervention are
highly beneficial: they shift behavior in the desired direction, apparently for the right reasons. Moreover, given the similarity of their effects, it is tempting to conclude that the provision of opportunities for practice and individualized feedback yield no incremental benefits.


\subsection{Problems with assessments based on conventional outcome measures}

\label{welfare}

Further analysis reveals that assessments based on conventional outcome metrics are misleading in this setting. Moreover, their failures here are traceable to problems that could plausibly arise in other applications.

Financial education interventions seek to ensure that people understand the consequences of their options when they make choices. In our experiment, simple framing simulates perfect education by making those consequences transparent. The degree to which financial education closes the gap between complexly and simply framed valuations, $r_{c}^{i,R,t}$ and $r_{s}^{i,R,t}$, therefore provides a gauge of the intervention's success.

Figure \ref{CDF} displays the empirical cumulative distribution function of the valuation difference $r_{c}^{i,R,t}-r_{s}^{i,R,t}$ for the Treatment and Control conditions in each experiment. In each Control condition, roughly 60\% of subjects are afflicted by exponential growth bias ($r_{c}^{i,R,t}<r_{s}^{i,R,t}$), while a nontrivial fraction of subjects are well-calibrated, or overestimate compound interest ($r_{c}^{i,R,t}\geq r_{s}^{i,R,t}$).\footnote{Appendix \ref{simpleNoise} tests and refutes the hypothesis that the overvaluation of complexly framed opportunities is solely attributable to noisy choice. A fraction of subjects in \citet{Godaetal.EGB} and in \citet{LevyTasoff} also overestimate compound interest.}

 \begin{figure}[htbp!]
\caption{CDFs of valuation differences.}
\label{CDF}
\vspace{-0.3cm}
\begin{centering}
\begin{tabular}{cc}
\multicolumn{2}{c} {\textbf{}}\\
Experiment A  & Experiment B \\
\includegraphics[width=0.5\textwidth]{graphs/cdfOld}  & \includegraphics[width=0.5\textwidth]{graphs/cdfNew} \\
 & \\
\end{tabular}
\par\end{centering}
\footnotesize
\textbf{\textit{Notes:}}  CDFs of valuation differences $r_{c}^{i,R,t}-r_{s}^{i,R,t}$ across framings for equivalent instruments in the Treatment and Control conditions of Experiments A and B, respectively. Mass to the left of zero indicates
underestimation of compound interest (exponential growth bias). Mass to the right of zero indicates overestimation.
\end{figure}


 An effective intervention would increase valuations in complexly framed tasks for subjects who underestimate compound interest, and would \emph{decrease} them for subjects who overestimate compound interest. As a result, the CDF of the valuation difference $r_{c}^{i,R,t}-r_{s}^{i,R,t}$ would become steeper around zero. We observe a much different effect in Experiment A: the Treatment shifts the entire CDF to the right. It increases valuations for complexly framed tasks across the board, irrespective of whether a subject initially underestimates compound interest, estimates it correctly, or overestimates it. This indiscriminate effect helps some subjects but hurts others. In sharp contrast, the effects of the Treatment in Experiment B are sensitive to subjects' initial bias: it increases valuations of complexly framed tasks only for subjects who would have underestimated compound interest, and not for those who would have estimated compound interest correctly, or who would have overestimated it.\footnote{In principle, a change in the valuation difference $r_{c}^{i,R,t}-r_{s}^{i,R,t}$ may reflect changes in either $r_{c}^{i,R,t}$ or $r_{s}^{i,R,t}$. We perform this decomposition in Section \ref{subsec:ConfoundingFraming}.} Unlike the Experiment A Treatment, the Experiment B Treatment is therefore unambiguously successful. Yet neither of the conventional outcome metrics detects this difference.

Why do the conventional metrics fail to reveal the problems with the Experiment A Treatment? With respect to directional effects on behavior, the issues are obvious in light of Figure \ref{CDF}: asking whether the average change in complexly framed valuations counters the typical initial bias ignores heterogeneity in both. Despite the fact that the average complexly framed valuation moves toward the average simply framed valuation, many subjects overshoot, and some apparently double down on their pre-existing biases.

The problems with knowledge assessments are more subtle.
Improved performance
on batteries of exam-style questions implies better financial decisions
if (i) subjects employ that knowledge when making decisions, and
(ii) changes in financial choices are only due to the additional knowledge,
and do not involve mechanisms such as propaganda or brow-beating.\footnote{This approach also assumes that the effects of knowledge on the quality
of decision making are monotonic, which may not be correct if ``a
little knowledge is a dangerous thing.''} Further investigation falsifies these assumptions.

We evaluate assumptions (i) and (ii) directly by studying the effects of the Rhetoric-only and Substance-only interventions in Experiment A. First, we examine their effects on assessed comprehension by regressing test scores on each of the four treatments in Experiment A, clustering standard errors at the subject level. Column 1 of Table \ref{Mechanisms} displays the results. The Substance-Only intervention increases performance on test questions concerning compound interest by a similar amount as the full intervention, while the Rhetoric-Only intervention has a much smaller effect.\footnote{The fact that the effect of that intervention is positive may be attributable to the inclusion of an example that illustrates the calculation of compound interest.} To verify that these results are not due to an effect of the Substance-Only intervention on general motivation, Column 2 uses performance on test questions about the Control intervention as the dependent variable. Indeed, subjects in the Control intervention perform significantly better on these questions than subjects in any other treatment. Accordingly, increases in performance on our assessment of comprehension arise through the expected mechanism (substantive instruction).


\begin{table}[htbp!]
\caption{Separate effects of rhetoric and substance in Experiment A. }
\label{Mechanisms}
\begin{centering}

\begin{tabular}{lcccccc}
\toprule
 & (1)  & (2)  & (3)  & (4)   \\
VARIABLES  & \multicolumn{2}{c}{Test scores on questions about} & \multicolumn{2}{c}{Valuations in frame}  \\
\cmidrule(r){2-3} \cmidrule(r){4-5}  & Treatment  & Control  & Complex  &Simple  \\
\midrule
 &  &  &  &  \\
Levels  &  &  &  & \\
\hspace{0.4cm}\textit{Substance-Only}  & 3.234{*}{*}{*}  & 1.945{*}{*}{*}  & 62.969{*}{*}{*}  & 72.273{*}{*}{*}   \\
 & (0.123)  & (0.099)  & (2.373)  & (2.030)   \\
\hspace{0.4cm}\textit{Rhetoric-Only}  & 2.455{*}{*}{*}  & 2.205{*}{*}{*}  & 77.538{*}{*}{*}  & 77.623{*}{*}{*} \\
 & (0.146)  & (0.095)  & (2.785)  & (2.119)  \\
\hspace{0.4cm}\textit{Treatment A}  & 3.406{*}{*}{*}  & 2.226{*}{*}{*}  & 73.261{*}{*}{*}  & 72.657{*}{*}{*} \\
 & (0.135)  & (0.092)  & (2.566)  & (2.139)   \\
\hspace{0.4cm}\textit{Control A}  & 1.963{*}{*}{*}  & 3.284{*}{*}{*}  & 58.949{*}{*}{*}  & 72.255{*}{*}{*}  \\
 & (0.140)  & (0.114)  & (2.272)  & (2.089)  \\


$p$-value of difference  &  &  &\\
to Control  &  &  &   \\
\hspace{0.4cm}\textit{Treatment A}  & 0.000  & 0.000  & 0.000  & 0.893   \\
\hspace{0.4cm}\textit{Substance-Only}  & 0.000  & 0.000  & 0.222  & 0.995  \\
\hspace{0.4cm}\textit{Rhetoric-Only}  & 0.015  & 0.000  & 0.000  & 0.072   \\

Observations  & 455  & 455  & 4,550  & 4,550  \\
Subjects  & 455  & 455  & 455  & 455   \\
\bottomrule
\end{tabular}
\par\end{centering}
\footnotesize
\textbf{\textit{Notes:}} Each column displays the coefficients of a separate OLS regression. Standard errors in parentheses, clustered by subject. {*}{*}{*}$p<0.01,${*}{*}{*}$p<0.05,${*}$p<0.1$.
\end{table}


Next, we probe the connection between effects on comprehension and effects on behavior. Assumption (i) maintains that behavior responds to knowledge. Because we have just demonstrated that the Substance-Only treatment improves knowledge, this assumption implies that it ought to change valuations in complexly framed decision problems. Assumption (ii) maintains that behavior only responds to features of the intervention that enhance comprehension. Because we have just demonstrated that the Rhetoric-Only treatment has only a modest effect on knowledge, this assumption implies that it ought to change valuations in complexly framed decisions to a much smaller degree than the full intervention.

In fact, we find precisely the opposite. Column 3 displays the coefficients of a regression of complexly framed valuations on treatment indicators. The Substance-Only intervention does not have a statistically discernible effect on these valuations. In contrast, the Rhetoric-Only intervention yields effects comparable to those of the full intervention. Column 4 confirms that the effects of the interventions are largely confined to the complexly framed valuations; simply framed valuations are largely unaffected.\footnote{Simply framed valuations are a bit higher for the Rhetoric-Only treatment than for the Control $(p<0.1)$. For the other three treatments, these valuations are essentially identical.}

We conclude that the financial education intervention largely impacts knowledge through its substantive component, but largely impacts behavior through its rhetorical component. In other words, behavioral effects are wholly disconnected from effects on comprehension. Thus, neither assumption (i) nor assumption (ii) holds in Experiment A. These findings invalidate inferences about the quality of decision making based on assessments of comprehension in Experiment A, and raise more general questions about the reliability of programmatic evaluations based on measures of knowledge and directional behavioral effects.


\subsection{Assessments based on Deliberative Competence}
\label{AssessDC}

In this subsection, we propose a simple alternative outcome metric, justify it intuitively, and use it to evaluate the various interventions studied in Experiments A and B. We demonstrate formally that this metric robustly captures welfare losses in Section \ref{theory}.

Divergences between valuations in equivalent simply and complexly framed tasks are of concern because they imply that people may commit errors when deciding whether to purchase instruments. The greater the divergence, the greater the potential harm these errors cause. For a given valuation pair involving future reward $R$ and delay $t$, we therefore gauge subject $i$'s \emph{deliberative competence} by computing the absolute value of the difference between the normalized complexly and simply framed valuations: \[
d_M^{i,R,t}=-\left|r_{c}^{i,R,t}-r_{s}^{i,R,t}\right|.
\]
In the language of Behavioral Public Economics, $d_M^{i,R,t}$ is the absolute value of subject $i$'s \emph{price-metric bias} \citep{BernheimTaubinsky}. The sign convention used here aids interpretation because it associates
larger numerical values with higher quality decisions.\footnote{As discussed in Section \ref{definingFinancialCompetence}, the distance metric  $-\frac{1}{2}\big(r_{c}^{i,R,t}-r_{s}^{i,R,t}\big)^{2}$ also has a valid welfare interpretation. Appendix \ref{resultsSquared} reproduces our analysis using this metric. Results are generally similar.} We average over tasks to arrive at the typical magnitude of decision error for subject $i$, and then average over subjects to obtain an overall measure of deliberative competence.

By definition, the absolute difference between $r_{c}^{i,R,t}$ and $r_{s}^{i,R,t}$ measures the consumer's valuation error. However, we are interested in measuring something else: the harm (or welfare loss) that such errors inflict on consumers in settings where they must decide whether to purchase instruments. Our theoretical analysis shows that the same absolute differential also measures the potential harm.

To understand this point, note that harm can befall the consumer either because she purchases an instrument when she should not, or because she fails to purchase it when she should.  Suppose first that a complexly framed security paying $R$ at time $t$ is available for a price $p< \tilde{r}_{s}^{i,R,t}$, and that subject $i$ chooses not to buy it because $\tilde{r}_{c}^{i,R,t} < p$. In that case, she foregoes a net gain of $ \tilde{r}_{s}^{i,R,t} - p$. Subject to the constraint that the price remains between the two valuations (so the decisions differ), this expression reaches a maximum of $\tilde{r}_{s}^{i,R,t} - \tilde{r}_{c}^{i,R,t}$ at $p = \tilde{r}_{c}^{i,R,t}$. Next suppose that the same security is available for a price $p> \tilde{r}_{s}^{i,R,t}$, and that subject $i$ chooses to buy it because $\tilde{r}_{c}^{i,R,t} > p$. In that case, she overpays by $p - \tilde{r}_{s}^{i,R,t}$. Subject to the constraint that the price remains between the two valuations (so the decisions differ), this expression reaches a maximum of $\tilde{r}_{c}^{i,R,t} - \tilde{r}_{s}^{i,R,t}$ at $p = \tilde{r}_{c}^{i,R,t}$. Thus, in both cases, $d_M^{i,R,t}$ represents the maximum loss to which the subject is exposed, normalized by $R$. Section \ref{theory} formalizes this logic and explores its robustness.


\begin{figure}[htbp!]
\caption{Assessment based on Deliberative Competence.}
\label{DCAssess} \begin{centering}
\begin{tabular}{cc}
\multicolumn{2}{c}{\textbf{}}\\
Experiment A  & Experiment B \\
\includegraphics[width=0.5\textwidth]{graphs/finCompLinesA}  & \includegraphics[width=0.5\textwidth]{graphs/finCompLinesB} \\
\end{tabular}
\par\end{centering}
\footnotesize
\textbf{\textit{Notes:}}  Mean Deliberative Competence, $d_M^{i,R,t}=-|r_{c}^{i,R,t}-r_{s}^{i,R,t}|$, with $r_{f}^{i,R,t}$  rescaled to range from 0 to 100 (percentage points). Whiskers represent 95\% confidence intervals with standard errors
clustered on the subject level.
\end{figure}


Figure \ref{DCAssess} shows the levels of deliberative competence across Control and Treatment groups in Experiments A and B, respectively, averaged over all valuation pairs and subjects. In each experiment, deliberative competence in the Control condition is approximately -25\% (-24.4\% in Experiment A and -25.7\% in Experiment B). A key finding is that treatment effects differ dramatically across Experiments A and B. For Experiment A, the Treatment has no discernible effect on deliberative competence. In stark contrast, for Experiment B, the Treatment increases deliberative competence by a substantial and statistically significant increment, from -25.7\% to $-18.6\%$. Anticipating the results of Section \ref{definingFinancialCompetence}, we can say that the Treatment in Experiment B eliminates 27.6 percent of the welfare loss associated with poor comprehension of consequences in the complex frame ($1-0.186/0.257=0.276$).\footnote{These estimates are based on equal weighting of observed deliberative competence across subjects and decision problems. We discuss the implications of equal weighting of subjects in Section \ref{subsec:Aggregation}.} As a practical matter, this finding suggests that incorporating practice and feedback can have a dramatic impact on the efficacy of financial education, even when the feedback is fully automated and relatively simple. Thus, using deliberative competence rather than conventional outcome metrics reverses our conclusions concerning the role of practice and feedback.


\begin{table}[htbp!]
\setlength{\tabcolsep}{3pt}


\begin{centering}
\caption{Deliberative Competence. \label{finComp} }
\begin{tabular}{lcccccc}
\midrule
 & (1)  & (2)  & (3)  & (4)  & (5)  & (6) \\

Delay in days  & 72 and 36  & 72 and 36  & 72  & 72  & 36  & 36 \\
\cmidrule(r){2-3} \cmidrule(r){4-5} \cmidrule(r){6-7} Experiment  & A  & B  & A  & B  & A  & B \\
\midrule
Levels \\
 \quad \textit{Control} & -24.448*** & -25.673*** & -24.076*** & -25.945*** & -24.820*** & -25.401*** \\
 & (1.633) & (1.357) & (1.744) & (1.428) & (1.682) & (1.393) \\
\quad \textit{Treatment}  & -22.864*** & -18.569*** & -22.959*** & -18.441*** & -22.769*** & -18.697*** \\
 & (1.741) & (1.177) & (1.799) & (1.208) & (1.886) & (1.253) \\
\quad \textit{Substance-Only}  & -22.012*** &  & -21.257*** &  & -22.767*** &  \\
 & (1.433) &  & (1.422) &  & (1.620) &  \\
\quad \textit{Rhetoric-Only}  & -19.797*** &  & -20.191*** &  & -19.403*** &  \\
 & (1.406) &  & (1.458) &  & (1.506) &  \\
 &  &  &  &  &  &  \\
$p$-value of difference \\
to Control \\
\quad \textit{Treatment}    & 0.507 & 0.000 & 0.656 & 0.000 & 0.418 & 0.000 \\
\quad \textit{Substance-Only} & 0.263 &  & 0.211 &  & 0.380 &  \\
 \quad \textit{Rhetoric-Only}  & 0.031 &  & 0.088 &  & 0.017 &  \\
Observations & 4,550 & 3,480 & 2,275 & 1,740 & 2,275 & 1,740 \\
Subjects & 455 & 348 & 455 & 348 & 455 & 348 \\
\bottomrule
\end{tabular}
\par\end{centering}
\footnotesize
\textbf{\textit{Notes:}} Each column displays the coefficients of a separate OLS regression of deliberative competence on treatment indicators. Standard errors in parentheses, clustered by subject. Deliberative competence measured as  $d_M^{i,R,t}=-|r_{c}^{i,R,t}-r_{s}^{i,R,t}|$, with  $r_{c}^{i,R,t}$  rescaled to range from 0 to 100 (percentage points). ***$p<0.01,$***$p<0.05,$*$p<0.1$
\end{table}

To evaluate treatment effects econometrically, we regress $d_M^{i,r,t}$ on a treatment indicator and cluster standard errors by subject, separately for each experiment. Table \ref{finComp} displays the results. Column 1 shows that the Treatment intervention in Experiment A increases deliberative competence by a small and statistically insignificant 1.5 percentage points. Column 2 exhibits the corresponding regression for Experiment B. It shows that while Control subjects display a level of deliberative competence similar to those in Experiment A, the Treatment intervention substantially raises deliberative competence. The increase of 7.1 percentage points is highly statistically significant $(p<0.01)$. Columns 3 and 4 replicate this analysis separately using the set of investments with 36-day delays, whereas Columns 5 and 6 do so for investments with 72-day delays. The same conclusions follow in both cases.

Column 1 also shows that the Substance-Only intervention leaves deliberative competence unchanged, despite its positive effect on knowledge. While the Rhetoric-Only intervention produces a small improvement in deliberative competence, we know from our earlier analysis that this effect does not flow from improved comprehension. Rather, the motivational rhetoric provides a nudge that proves fortuitously beneficial to a small degree. Columns 3 and 5 show that we obtain similar results for tasks with 36-day and 72-day delays.


\section{Theoretical framework and supplemental empirical analyses}

\label{definingFinancialCompetence}\label{model}\label{theory}

In Section \ref{AssessDC}, we provided an intuitive justification for using differentials between simply and complexly framed valuations of the same future payment to evaluate policy interventions that seek to improve comprehension of the consequences that follow from choices. According to our empirical analysis, such metrics capture important aspects of policy effects that conventional outcome measures overlook. The objectives of this section are to demonstrate that deliberative competence, as we define it, is robustly interpretable as a measure of the welfare loss resulting from poor decision making, even in contexts that raise second-best issues, and to clarify the types of settings to which these conclusions apply.

After describing the setting (Subsection \ref{Setting}), we articulate our welfare criteria (\ref{prelimWelfare}). The most novel aspect of this discussion concerns our treatment of decision-making defects outside the scope of analysis. We proceed from the premise that it is impractical to analyze all biases and remedial policies at once (\emph{comprehensive} welfare analysis). Instead, studies in Behavioral Public Economics invariably focus on one or two biases and policies at a time (\emph{compartmentalized} welfare analysis). Yet the literature offers no framework for conducting such analyses in a manner that avoids the conceptual ``second-best'' problems identified by \citet{LipseyLancaster}. To fill this gap, we propose and justify the concept of \emph{idealized welfare analysis}.

We formalize the welfare interpretation of our deliberative competence measures under the restrictive assumption that the consumer's decision-making apparatus involves no defects other than the misunderstandings that the pertinent policy interventions target (Section \ref{OneBias}), as well as under less restrictive assumptions that permit us to conduct idealized welfare economics while acknowledging the existence of other biases (Section \ref{ManyBiases}). We then discuss a method for detecting and adjusting for a class of potential confounds (Sections \ref{subsec:ConfoundingFraming}), as well as a strategy for addressing issues involving interpersonal aggregation (Section \ref{subsec:Aggregation}), both of which we implement using our experimental data.

\subsection{Setting}
\label{Setting}

Following the literature on Behavioral Public Economics (\citealp [reviewed in][]{BernheimTaubinsky}), we distinguish between choice options and the preference-relevant outcomes they induce.  For example, people make choices over financial portfolios but care about the patterns of returns those portfolios generate, rather than the portfolios themselves. Likewise, people make choices among over-the-counter medications but care about the health consequences those products yield, rather than the medications themselves. We refer to such choice options as \emph{instruments} because they are a means to an end; they do not matter  intrinsically. The distinction between an instrument and its intrinsically valued consequences is important because it allows for the possibility that people may not understand the implications of their choices, a phenomenon known in Behavioral Public Economics as \emph{characterization failure} \citep{BernheimTaubinsky}.

Our method of measuring deliberative competence involves simple binary decisions: whether to purchase an instrument $I \in \mathcal{I}$ at a price $p$. We assume the consumer cares about current cash, $m$, and a vector $y$ of future consequences (e.g., specifying available cash at each point in time and state of nature). She starts with current cash $m_0$; without loss of generality, we simplify notation by setting $m_0=0$. If she foregoes the instrument, she anticipates the consequences $y_{0}$. In frame $f$, she interprets the instrument $I$ as offering consequences $y_{f}$. In other words, she thinks the incremental effect of the instrument will be $y_{f}-y_{0}$. We assume there is a simple way to frame the instrument, $f=s$, that presents its consequences transparently, thereby ensuring the consumer interprets them correctly. Thus, $y_{s}$ represents the instrument's actual consequences. With naturalistic (complex, non-transparent) framing, $f=c$, she misinterprets the instrument as offering the outcome $y_c \ne y_{s}$.\footnote{As noted in the text, this framework accommodates uncertainty, in the sense that we can interpret $y$ as a vector of state-contingent consequences. Under that interpretation, it would appear that false beliefs must pertain to those consequences and not to their likelihoods. However, if baseline consequences ($y_0$) are either constant over states or independent of the instrument's returns, we can reformulate a setting with false beliefs about probabilities as one with correct beliefs about probabilities and false beliefs about state-contingent payments, which our framework subsumes. To take a simple example, imagine that a lottery pays $x$ with probability $\frac{1}{3}$ and $z$ with probability $\frac{2}{3}$, but the consumer falsely believes the probabilities are reversed. We can represent this setting as one in which the lottery yields a payment $y_s^\omega$ in state of nature $\omega \in \{1,2,3\}$, where $y_s^1 = x$, $y_s^2=y_s^3 = z$, and the states are equally probable; furthermore, the consumer has correct beliefs about the probabilities of the underlying states but incorrectly believes the lottery pays $y_c^1 = y_c^2 = x$ and $y_c^3 = z$.} In other words, she experiences a form of characterization failure. Our object is to evaluate policies designed to mitigate those characterization failures. Where we distinguish between policies explicitly, we use $y_{c\theta}$ to denote the consumer's beliefs in the complex frame about the instrument's consequences when policy $\theta$ is in place. In our experiment, the instrument is a claim on a future payment, simply framed tasks transparently specify the future payment associated with the available instrument, complexly framed tasks describe the future payment in terms of compound interest, and the policies of interest target comprehension of compounding.

We posit the existence of a utility function, $V(m,y_f)$, that rationalizes observed choices. By assumption, the characteristics of the instrument and the frame only impact the consumer's choices insofar as they determine her perceptions of consequences. Some of our analysis concerns cases in which the decision maker suffers from additional and possibly unknown valuation biases that the aforementioned simple framing does not remove. We will assume that, with these ancillary biases removed through other means, the utility function $U(m,y_f)$ would rationalize choices in the simple and complex frames. Under these assumptions, $U(m,y_s)$ plays the role of ``true preferences."\footnote{To be clear, our approach does not require the existence of a single ``true preference" relation. As we discuss in Section \ref{GeneralResult}, it extends directly to cases in which welfare-relevant choices are context-dependent, and consequently admit multiple frame-dependent utility representations, as permitted in the welfare framework of \citet{BernheimRangelWelfare}.} We interpret $V$ and $U$ as indirect utility functions, in the sense that $m$ and $y$ could represent income rather than consumption. We assume throughout that these functions are unbounded above and below in $m$, that they have bounded first and second derivatives, and that their first derivatives with respect to $m$ are strictly positive.

We define the consumers' reservation price (alternatively, \emph{valuation}) given her interpretation of the instrument's consequences, $y_f$, assessed according to utility function $u \in \{U,V\}$, as the highest price at which she purchases the instrument  (denoted $r^u(y_f)$). This valuation is given by the following condition:\footnote{Here, we assume that in case of indifference, the consumer purchases the instrument. Existence of $r^u$ follows from continuity, monotonicity, and unboundedness.}
\begin{equation}
u(-r^u(y_f),y_f)=u(0,y_0)\label{eq:Indiff}
\end{equation}


\subsection{Framework for welfare analysis}
\label{prelimWelfare}


We consider a social planner who seeks to quantify the utility loss a consumer sustains from characterization failures in the domain targeted by the policies we aim to evaluate. The planner uses a utility function $u$ (the \emph{evaluation standard}) for this purpose. For reasons that will become apparent in Section \ref{ManyBiases}, we allow for the possibility that $u$ is either $U$ or $V$.


\subsubsection{Welfare criteria}

We evaluate welfare treating the price $p$ of an instrument $I$ as a random realization. This perspective is appropriate in the context of our experiment because $p$ is literally random. Of course, measuring welfare is important in an experiment such as ours not because we care about the experimental tasks per se, but because those tasks arguably distill the types of tradeoffs people may confront in practice. When interpreting our welfare measures, one can think of the randomness in $p$ as representing uncertainty about the particular tradeoffs that will prove relevant to real-world decisions.

We adopt two approaches to aggregating welfare over possible realizations. The first envisions a welfarist planner concerned with the consumer's expected utility. We assume the future price $p$ for the instrument $I$ is distributed according to some atomless CDF $H$ with density $h$. The loss, in terms of expected utility, associated with employing valuation $r$ rather than the utility-maximizing valuation  $r^u(y_s)$ is then:
\begin{equation}
l_{W}^{u}\left(r,y_s,H\right)=\left\{ \begin{array}{cc}
\intop_{r}^{r^u(y_s)}\left[u\left(-p,y_s\right)-u\left(0,y_0\right)\right]h(p)dp\quad & \mathrm{if\;}r\leq r^u(y_s)\\
\intop_{r^u(y_s)}^{r}\left[u\left(0,y_0\right)-u\left(-p,y_s\right)\right]h(p)dp\quad & \mathrm{if\;}r\geq r^u(y_s)
\end{array}\right.\label{eq:ObjectiveW}
\end{equation}
The subscript $W$ reminds us that this measure reflects a welfarist perspective.

The welfarist approach assumes the probability distribution of the price $p$ is known. While it is known for any given experiment, it is hard to say which prices give rise to tradeoffs that are likely to emerge in practice. Treating the probability distribution as unknown, we can employ an alternative approach from Computer Science that has recently found its way into Economics: evaluate an option based on the maximum possible loss it could induce.\footnote{See, for example, \citet{Carroll.Review}, who adopts this approach to study optimal mechanism design in settings where the distribution of agent characteristics is unknown. Other applications include decision theory \citep{MachinaSiniscalchi.AmbiguityReview} and public policy \citep{Sunstein.Maximin}. } The maximal loss associated with using the valuation $r$ rather than the utility-maximizing valuation $r^u(y_s)$ is:
\begin{equation}
l_{M}^{u}\left(r,y_s\right)=\left\{ \begin{array}{cc}
\min_{p\in\left[r,r^u(y_s)\right]}\left[u\left(-p,y_s\right)-u\left(0,y_0\right)\right]\quad & \mathrm{if\;}r\leq r^u(y_s)\\
\min_{p\in\left[r^u(y_s),r\right]}\left[u\left(0,y_0\right)-u\left(-p,y_s\right)\right]\quad & \mathrm{if\;} r \geq r^u(y_s)
\end{array}\right.\label{eq:ObjectiveL}
\end{equation}
The subscript $M$ reminds us that this function measures the maximal loss.

Because the scale of $u$ is arbitrary, in both cases we convert to money-metric utility. Specifically, we divide the welfare loss by the perceived marginal utility of income according to $u$ when the individual does not purchase the instrument:\footnote{This adjustment approximates the reduction in income without the instrument that would produce the same utility decrement as the consumer's decision error. Alternatively, one could define $L_e^u$ (for $e \in \{W,M\}$) as the \emph{exact} reduction in income sufficient to produce that utility decrement: $u(0,y_0)-u(-L_e^u,y_0)=l_e^u$. Because we ultimately take limiting approximations, these two scaling methods yield the same results.}
\bmathnn
L_{W}^{u}\left(r,y_s,H\right)&=&\frac{1}{u_m(0,y_0)} l_{W}^{u}\left(r,y_s,H\right), \\
L_{M}^{u}\left(r,y_s\right)&=&\frac{1}{u_m(0,y_0)} l_{M}^{u}\left(r,y_s\right).
\emathnn

\subsubsection{Accounting for additional biases}


In many applications, it is reasonable to think that consumers suffer from biases other than misapprehension of the consequences $y_s$. The existence of such biases may cast doubt on our assumption that simply framed choices provide a valid normative benchmark. Loosely speaking, if we suspect that biases infect simply framed choices, we must accept the possibility that they may not reveal ``true preferences.'' Formally, the issue is that such biases cause the utility function $V$ to diverge from $U$ (other than in the sense of a monotonic transformation), affecting choices in both frames.

As an illustration of the problem, in our experiment, simple framing eliminates the need to understand compound interest, but it does not address considerations such as present-focus or excessive suspicion that the experimenter will renege on promised payments.\footnote{Present focus may not be an issue in our experiment because the earliest payments involve a two-day delay.}  If these considerations bias subjects' choices toward lower valuations of future prospects in \emph{both} frames, then one might be tempted to conclude that the ``overshooting'' documented in Section \ref{welfare} is actually beneficial.

\citet{LipseyLancaster} reasoned that the right way to handle these types of issues is to analyze all distortions (here, biases affecting the valuation $r^V(r_c)$) and all remedies for them simultaneously. We call this approach \emph{comprehensive second-best analysis}. While conceptually attractive, it is infeasible in practice.\footnote{For example, while the general theoretical framework of \citet{FarhiGabaix} encompasses such analyses, they acknowledge that empirical implementation would be a ``momentous task'' (p. 311).} Indeed, the field of Behavioral Economics has not yet arrived at the comprehensive structural understanding of imperfections in human decision making that this approach requires.

Because of these practical challenges, the overwhelming preference of economists, as revealed by research in this area, is to analyze policies one (or two) at a time. The idea is to focus on solving individual pieces of the overall welfare puzzle, thereby progressively building up a complete solution. But the compartmentalization of policy analysis raises challenging conceptual issues. Indeed, any approach other than comprehensive second-best analysis necessarily involves conceptual compromises. Still, to the extent the requirement of tractability necessitates a compromise, we ought to make the best one possible. Table \ref{WelfareTypes} lists three frameworks for compartmentalized welfare analysis. Two are familiar; the third is our proposal.


\begin{table}[hbt]
\caption{Frameworks for compartmentalized welfare analysis. \label{WelfareTypes} }
\begin{centering}
\begin{tabular}{lcccc}
\toprule
Framework  & Evaluated   & Evaluation    & Welfare  \\
  &  choice  &  standard   &  criterion \\
\midrule
 Narrow second-best & $r^{V}(y_{c\theta})$ & $U$ & $L_{M}^U\left(r^{V}(y_{c\theta}),y_{s}\right)$ or $L_{W}^U\left(r^{V}(y_{c\theta}),y_{s},H\right)$\\
 \quad welfare analysis&  &  & &\\
 \\
 Myopic welfare analysis & $r^{V}(y_{c\theta})$ & $V$ & $L_{M}^V\left(r^{V}(y_{c\theta}),y_s\right)$ or  $L_{W}^V\left(r^{V}(y_{c\theta}),y_s,H\right)$\\
 &  &  & &\\
 Idealized welfare analysis & $r^{U}(y_{c\theta})$ & $U$ & $L_{M}^U\left(r^{U}(y_{c\theta}),y_{s}\right)$ or $L_{W}^U\left(r^{U}(y_{c\theta}),y_{s},H\right)$ \\\bottomrule
\end{tabular}
\par\end{centering}
\footnotesize
\end{table}

The first framework, which we label \emph{narrow second-best welfare analysis}, involves a perspective that \citet{LipseyLancaster} attribute to \citet{Meade}. We imagine that there are many biases affecting the valuation $r^V(r_c)$ and corresponding policy options that target them. We evaluate one policy targeting one bias (or a small collection of policies targeting a small number of biases) accounting for the existence of the other biases, but holding the rest of the policies fixed, even if they are suboptimal. Within our setting, this approach quantifies the welfare cost of choosing $r^{V}(y_{c\theta})$ (which reflects all biases, as they exist under prevailing policies) rather than $r^{U}(y_{s})$ (which reflects the absence of all biases), using $U$ as the standard of evaluation. In other words, as shown in Table \ref{WelfareTypes}, the pertinent measures of the welfare loss for policy $\theta$ are $L_{M}^U\left(r^{V}(y_{c\theta}),y_{s}\right)$ and $L_{W}^U\left(r^{V}(y_{c\theta}),y_{s},H\right)$.


Setting aside the enormous practical difficulty of identifying and properly accounting for all pertinent biases, this approach suffers from a critical conceptual flaw: if the policy of interest is merely one component of a broader effort to design interventions that improve the quality of decision making, then treating other policies as fixed constraints can yield incorrect conclusions. To illustrate, consider the subjects in our experiment who overstate the returns to investments paying compound interest. If they suffer from present bias, then, as suggested at the outset of this section, one might be tempted to conclude that any improvement in their understanding of compounding would cause them harm. And yet, they would clearly derive the greatest benefit from a combination of policies that address both biases, such as education along with commitment devices. Thus, insisting that a policy ought to compensate for all biases that may infect choice, and not merely for the one it is designed to address, can yield misleading conclusions concerning the policy of interest.

The second framework in Table \ref{WelfareTypes}, \emph{myopic welfare analysis}, focuses on small number (usually just one) of biases and associated policy instruments while assuming, at least implicitly, that the consumer's decision-making apparatus is otherwise faultless, perhaps because effective remedies for other valuation biases are already in place. Within our setting, this approach calls for evaluating the loss from choosing $r^{V}(y_{c\theta})$, the naturalistic selection, rather than $r^{V}(y_{s})$, according to the utility function $V$. In other words, as shown in Table \ref{WelfareTypes}, the pertinent measures of the welfare loss for policy $\theta$ are $L_{M}^V\left(r^{V}(y_{c\theta}),y_s\right)$ and  $L_{W}^V\left(r^{V}(y_{c\theta}),y_s,H\right)$. This form of compartmentalized policy analysis dominates the literature. Examples of studies that focus on a single bias include \citet{GruberKoszegi.AddictionRational, GruberKoszegi.TaxIncidence, DellaVignaMalmendier.ContractDesign, oDonoghueRabin.SinTaxes}. A few studies consider two biases (and associated policies) at a time; for instance \citet{AllcottMullainathanTaubinsky}. While such analyses highlight some important second-best considerations, they inevitably ignore others.

The problem with myopic welfare analysis is that the data are not in fact generated by an otherwise faultless decision process. As \citet{LipseyLancaster} pointed out in an analogous setting involving multiple distortions, there is no reason to think that solutions derived in this manner (which they called ``piecemeal'') are in fact desirable. Nor can one coherently paste together solutions for different biases derived in this way, because each solution implicitly assumes that the other solutions are unnecessary. In the context of our experiment, for instance, pasting together a solution for poor comprehension, devised under the assumption of no remedy for present bias, with a solution for present bias, devised under the assumption of no remedy for poor comprehension, does not generally yield policies that complement each other well and produce a desirable overall outcome.

The third approach listed in Table \ref{WelfareTypes}, \emph{idealized welfare analysis}, reformulates piecemeal policy analysis in a manner that avoids Lipsey and Lancaster's criticism. Viewing the analysis as part of a broader effort to diagnose and correct flaws in decision making, one evaluates policies designed to address a single bias (or a small collection of biases) under the assumption that effective remedies for other biases \emph{will be} forthcoming. In contrast to myopic welfare analysis, which assumes that other biases have \emph{already been} addressed, one must therefore account for the possibility that the observed data are infected by biases other than those the policy of interest seeks to target. Approaching each component problem in this manner, one can potentially arrive at a medley of compartmentalized solutions that complement each other and together achieve an overall solution.\footnote{By way of analogy, consider the problem of maximizing a function $f(x_1,...,x_N)$. Let $x^*$ denote the solution. Idealized welfare analysis is akin to maximizing $f(x_n,x_{-n}^*)$ for each $n$.} For our setting, idealized welfare analysis gauges the welfare cost of choosing $r^U(y_{c\theta})$ rather than $r^U(y_{s})$ (unobserved valuations purged of other biases), which potentially differ from $r^V(y_{c\theta})$ and $r^V(y_{s})$ (observed valuations infected with other biases), using $U$ as the evaluation standard. Thus, unlike myopic welfare analysis, it does not take observed simply framed valuations at face value as valid normative benchmarks. In the context of our experiment, the approach calls for assessing the loss associated with the complexly framed valuations we would elicit in an idealized setting that removes all valuation biases other than a misunderstanding of compound interest, based on simply framed valuations elicited in that same scenario. As shown in Table \ref{WelfareTypes}, the pertinent measures of the welfare loss for policy $\theta$ are $L_{M}^U\left(r^{U}(y_{c\theta}),y_{s}\right)$ and $L_{W}^U\left(r^{U}(y_{c\theta}),y_{s},H\right)$.

A natural first reaction to this proposed framework is that, if practical limitations render comprehensive second-best welfare analysis infeasible, then they must also preclude idealized welfare analysis. After all, idealized welfare analysis, too, requires knowledge of $U$, which the data do not reveal. And yet, as we prove below, there are reasonably general conditions under which idealized and myopic welfare measures coincide (up to a multiplicative factor), at least within the specific context of analyses involving comparisons of valuations between simple and complex frames.

Anything short of comprehensive welfare analysis inevitably involves conceptual compromises, and idealized welfare analysis is no exception. One must assume that other biases will be fully resolved, rather than either partially resolved or left untreated, and that the policies targeting them will not directly impact the bias of interest -- for instance, that a policy targeting present bias or false beliefs concerning experimenter reliability will not meaningfully improve comprehension of compound interest.

The fact that one can perform idealized welfare analysis without recovering the idealized utility function $U$ -- potentially without understanding or even knowing about other biases -- is counterintuitive. After providing formal results, we therefore offer a simple illustrative example, which also shows that the local approximations used in our main results can perform well for large distortions.


\subsubsection{Approximations}

In the following subsections, we provide formal justifications for our metrics of deliberative competence. Our approach is, in effect, to employ first- and second-order Taylor series approximations for welfare losses. Within Public Economics, this is a standard strategy for quantifying inefficiencies resulting from market distortions. For example, it provides the foundation for the famous Harberger triangle \citep{Harberger.Triangle}.

Formally, we define
\[
y_{f}^{\alpha}\equiv\alpha y_{f}+\left(1-\alpha\right)y_{0}
\]
This formulation allows us to rescale the consequences of the instrument as perceived in the simple and complex frames up and down by the factor $\alpha$. As the scale of the instrument shrinks, the pertinent distribution of prices (which we write as $H^\alpha$ to reflect the dependence on $\alpha$) presumably becomes more concentrated near zero. Absent information about the distribution of prices encountered in real-world analogs to our experimental tasks, we assume for the purpose of the welfarist metric that $H^\alpha$ is uniform in the pertinent region, with density $\frac{h}{\alpha}$.

Defining distance metrics $d_M(r,r') \equiv\left|r-r'\right|$ and $d_W(r,r') \equiv \frac{1}{2}\left(r-r'\right)^2$, we show that $d_M\left(r^u(y_c),r^u(y_s)\right)$ provides a first-order approximation for $L_M^u\left(r^u(y_c),y_s\right)$, and that $d_W\left(r^u(y_c),r^u(y_s)\right)$ provides a second-order approximation for $L_W^u\left(r^u(y_c),y_s,H\right)$ (hence our subscript convention).\footnote{We use a second-order approximation for $L_{W}^u$ rather than a first-order approximation because the first-order term is identically equal to zero.}

In taking limits, we hold the consumer's misunderstanding of the instrument fixed and let the scale of its consequences shrink. Another possibility is to hold the scale of the instrument's consequences fixed and let the magnitude of the misunderstanding shrink. Potentially, the latter approach may provide better approximations for large instruments, but worse approximations for large misunderstandings. It may also be more suitable for settings where risk plays a central role.

As shown in Appendix \ref{MoreTheory}, this alternative approach yields similar approximation results.


\subsection{Welfare analysis when simple framing involves no biases}
\label{OneBias}

We begin by assuming that the decision maker suffers from no biases other than misapprehension of an instruments' consequences, $y_s$. In this case, $V=U$. Our first result shows that one can approximate welfare losses by measuring distances between paired valuations.

\begin{proposition} \label{Prop1-2L}

For all $y_{c}$ and $y_{s}$ satisfying $\left.\frac{dr^{V}(y_{c}^{\alpha})}{d\alpha}\right|_{\alpha=0}\neq\left.\frac{dr^{V}(y_{s}^{\alpha})}{d\alpha}\right|_{\alpha=0}$, we have

\begin{equation}
\lim_{\alpha\rightarrow0}\left(\frac{L_{M}^{V}\left(r^{V}(y_{c}^{\alpha}),y_{s}^{\alpha}\right)}{d_{M}\left(r^{V}\left(y_{c}^{\alpha}\right),r^{V}\left(y_{s}^{\alpha}\right)\right)}\right)=1\label{eq:Prop4a}
\end{equation}
and

\begin{equation}
\lim_{\alpha\rightarrow0}\left(\frac{L_{W}^{V}\left(r^{V}(y_{c}^{\alpha}),y_{s}^{\alpha},H^{\alpha}\right)}{d_{W}\left(r^{V}\left(y_{c}^{\alpha}\right),r^{V}\left(y_{s}^{\alpha}\right)\right)}\right)=h.\label{eq:Prop4b}
\end{equation}

\end{proposition}

Equation (\ref{eq:Prop4a}) provides a formal foundation for treating $d_{M}\left(r^{V}(y_{c}),r^{V}(y_{s})\right)$ as an approximate measure of the maximal money-metric welfare loss, $L_{M}^{V}\left(r^{V}(y_{c}),y_{s}\right)$. Suppose our objective is to evaluate two corrective policies, $\theta$ and $\theta'$, that lead the consumer to interpret the instrument as offering $y_{c\theta}$ and $y_{c\theta'}$, respectively. According to Proposition \ref{Prop1-2L}, $d_{M}\left(r^{V}(y_{c\theta}),r^{V}(y_{s})\right)-d_{M}\left(r^{V}(y_{c\theta'}),r^{V}(y_{s})\right)$ approximates the difference in the maximal money-metric welfare losses, $L_{M}^{V}\left(r^V(y_{c\theta}),y_s)\right)-L_{M}^{V}\left(r^V(y_{c\theta'}),y_{s}\right)$. Similarly, for the welfarist criterion, equation (\ref{eq:Prop4b}) implies that $L_{W}^{V}\left(r^V(y_{c}),y_{s},H\right)$ is approximately proportional to $d_{W}\left(r^{V}\left(y_{c}\right),r^{V}\left(y_{s}\right)\right)$, where the constant of proportionality is independent of the instrument's perceived consequences, and hence of the corrective policy. While this measure does not tell us the absolute level of the welfare loss associated with any policy, it does allow us to rank any two policies correctly, and to approximate the ratio of their associated money-metric welfare losses, $\frac{L_{W}^{V}\left(r^V(y_{c\theta}),y_{s},H\right)}{L_{W}^{V}\left(r^V(y_{c\theta'}),y_{s},H\right)}$, with $\frac{d_{W}\left(r^{V}\left(y_{c\theta}\right),r^{V}\left(y_{s}\right)\right)}{d_{W}\left(r^{V}\left(y_{c\theta'}\right),r^{V}\left(y_{s}\right)\right)}$.

Despite the fact that the expected welfare loss $L_{W}^V$ and the maximal welfare loss $L_{M}^V$ reflect different approaches to conceptualizing the loss function, they admit similar welfare approximations, inasmuch as $\big(r^V(y_c)-r^V(y_s)\big)^{2}$ is simply the square of $\left|r^V(y_c)-r^V(y_s)\right|$. While the expected loss reflects a more conventional approach, there are two reasons to prefer the maximal loss. A practical advantage is that $\left|r^V(y_c)-r^V(y_s)\right|$ is less sensitive to outliers than $\big(r^V(y_c)-r^V(y_s)\big)^{2}$. A conceptual advantage is that the maximal loss is directly interpretable as a dollar-value measure, while the expected loss is a rescaling of a dollar-value measure by the unknown density $h$. The latter remains useful, however, in that it correctly ranks policies and accurately measures the ratio of their benefits.


In Section \ref{AssessDC}, we averaged $d_M$ over a collection of instruments.
Proposition \ref{Prop1-2L} also allows us to interpret such averages. Suppose our objective is to determine $\mathcal{L}_M^V(\mathcal{I}) \equiv \sum_{I\in\mathcal{\mathfrak{I}}}\lambda_{I}L_M^V\left(r^{V}(y_c^I),y_s^I\right)$, where $\mathcal{I}$ includes the instruments of interest, and $y_f^I$ denotes the perceived outcome for instrument $I$ in frame $f$. The weight $\lambda_I$ could reflect judgments of each instrument's relevance for real-world choices, or about the relative importance of an incremental dollar in the types of circumstances that are associated with the instrument's availability. Proposition \ref{Prop1-2L} tells us that we can approximate $\mathcal{L}_M^V(\mathcal{I})$ by computing $ \sum_{I\in\mathcal{\mathfrak{I}}}\lambda_{I}d_M\left(r^{V}(y_c^I),r^{V}(y_s^I)\right)$. Likewise, the proposition tells us that we can approximate $\mathcal{L}_W^V(\mathcal{I})$ (similarly defined) up to a constant of proportionality (which allows us to rank policies and measure their relative benefits) by computing $ \sum_{I\in\mathcal{\mathfrak{I}}}h_I\lambda_{I}d_W\left(r^{V}(y_c^I),r^{V}(y_s^I)\right)$, where $h_I$ is the density of the price distribution for instrument $I$. Because the pertinent price distributions are unknown, it is reasonable to treat $h_I$ as either a constant, or at least uncorrelated with the welfare loss. In either case, computing $\sum_{I\in\mathcal{\mathfrak{I}}}\lambda_{I}d_W\left(r^{V}(y_c^I),r^{V}(y_s^I)\right)$ provides the desired approximation. Aggregation over consumers raises related issues; see Section \ref{subsec:Aggregation}.


A limitation of Proposition \ref{Prop1-2L} and other results to follow is that they pertain to binary choices rather than to selections from arbitrary menus. Because binary comparisons lie at the core of more complex choices, this feature is less of a limitation than one might think. Suppose the consumer must purchase an instrument from the set $\mathcal{M} \cup \{0\}$, where $0$ represents the status quo. As an example, different elements of this set might represent different quantities of the same instrument. Suppose further that we have assessed simply and complexly framed valuations for all members of $\mathcal{M}$. Letting $\mathcal{M}_L$ ($\mathcal{M}_H$) represent the subset of instruments for which the consumer's valuation is too low (too high), the greatest possible welfare loss is then proportional to  $\max_{I\in\mathcal{M}_H}d_M(r^V(I,c),r^V(I,s)) + \max_{I\in\mathcal{M}_L}d_M(r^V(I,c),r^V(I,s))$ (where $r^u(I,f)$ is shorthand for the valuation of instrument $I$ in frame $f$ and, by convention, $\max_{I\in\emptyset}L_M(I) \equiv 0$) -- i.e., it is the loss incurred when the consumer purchases the most overvalued instrument at the worst possible price and foregoes the most undervalued instrument at the best price consistent with purchasing the first instrument.\footnote{Suppose the consumer purchases $I$ at price $p$ when she should have purchased $I'$ at price $p'$. We then have $r^u(I,c)-p \ge \max\{0,r^u(I',c)-p'\}$ and $r^u(I',s)-p' \ge \max\{0,r^u(I,s)-p\}$. For any given $p$, the loss is maximized by setting $p'$ as low as possible, consistent with these inequalities. Ignoring the non-negativity constraints for the moment, we have $p'=r^u(I',c)-r^u(I,c)+p$. Setting $p=r^u(I,c)$ and $p'=r^u(I',c)$ satisfies non-negativity. The total loss, which we approximate as $\left[ r^u(I',s)-p' \right] + \left[ p-r^(I,s) \right]$, is then $\left[ r^u(I',s)-r^u(I',c) \right] + \left[ r^u(I,c)-r^u(I,s) \right]$. Choosing $I$ and $I'$ to be the instruments with, respectively, the greatest overvaluation and the greatest undervaluation maximizes this expression. Note that this argument subsumes the case in which either $I$ or $I'$ is $0$.} Welfarist extensions appear more challenging due to the need for assumptions concerning the joint distribution of prices, but may be possible in cases where such assumptions are natural (for example, when an instrument is available in various quantities at a fixed unit price).


\subsection{Welfare analysis when simple framing involves additional biases}

\label{ManyBiases}

\subsubsection{The general result}
\label{GeneralResult}

We now relax the assumption that simple framing, by itself, removes all pertinent biases. Formally, we extend our analysis to allow for arbitrary differences between $U$ and $V$. The main result of this section shows that our measure of deliberative competence, $d_{e}\left(r^{V}(y_{c}^{\alpha}),r^{V}(y_{s})\right)$, approximates the corresponding idealized welfare measure up to a multiplicative constant for each welfare perspective $e \in \{M,W\}$ if a certain separability assumption holds. The assumption requires that we can write the utility functions as $V\left(m,\varphi\left(y\right)\right)$ and $U\left(m,\varphi\left(y\right)\right)$ for some subutility function $\varphi$.

When $y$ is a scalar (as in our experiment), the separability assumption involves no loss of generality. When $y$ is multidimensional, it is restrictive, but it nevertheless subsumes important possibilities, including the following.

\begin{enumerate}
    \item Instruments are investments that yield financial returns in future periods and states of nature, which the decision maker misunderstands. Our objective is to evaluate policies that target those misunderstandings. A complicating factor is that the consumer may also be present-biased to an unknown degree. In that case, the utility functions may take the form  $V(m,y)=\nu(m)+\beta\varphi(y)$ and $U(m,y)=\nu(m)+\varphi(y)$, where $\varphi(y)$ is $\delta$-discounted utility and $\beta$ is the unknown present-bias parameter.
    \item Instruments are insurance contracts that yield future payments contingent on the occurrence of a particular event. The consumer misunderstands the formulas that determine those payments, and our objective is to evaluate policies that target those misunderstandings. A complicating factor is that the consumer may misjudge the probability of the event to an unknown degree. In that case, the utility functions may take the form $V(m,y)=\nu(m)+\tilde{\pi}\varphi(y)+(1-\tilde{\pi})u_0$ and $U(m,y)= \nu(m)+\pi\varphi(y)+(1-\pi)u_0$, where $\tilde{\pi}$ is the unknown perceived probability, $\pi$ is the true probability, $\varphi(y)$ is expected utility conditional on the event, and $u_0$ is expected utility conditional on the event's absence.
    \item Instruments are medications that impact various aspects of future health in ways consumers misunderstand. Our objective is to evaluate policies that target those misunderstandings. A complicating factor is that the consumer may also be imperfectly attentive to health consequences. In that case, the pertinent utility functions might take the assumed forms, with $\varphi(y)$ representing a composite health good, and with the discrepancy between $V$ and $U$ reflecting inattention.
    \item An additional complicating factor in any of the preceding examples is that the method used for eliciting valuations may activate biases. For example, multiple price lists can induce ``end-point'' effects \citep{AndersenMPL} and, more generally, framing can ``anchor'' subjects' assessments \citep[e.g.][]{ariely2003coherent}. In such cases, we might have $U(m,y)=\nu(m)+\varphi(y)$ and $V(m,y)=\mu(m)+\xi(\varphi(y))$ for functions $\mu$ and $\xi$.
\end{enumerate}

While this list is not exhaustive, it is important to understand that the separability assumption limits potential applications by ruling out biases that impact tradeoffs among the elements of the vector $y$. For example, treating departures from exponential discounting as biases, our setting accommodates quasi-hyperbolic discounting but excludes hyperbolic discounting.\footnote{In the case of dynamic choices, biases can affect a given decision either directly or indirectly through future choices that shape the consequences of the current choice. Our method is suitable for measuring the direct effects. Specifically, it allows us to gauge the magnitudes of mistakes people make in one-shot decisions, with future consequences fixed experimentally. Such decisions are the building blocks of dynamic decisions, in the sense that dynamic problems require the consumer to solve one-shot problems where the future consequences are fixed at the continuation solutions.}

The following result shows, in effect, that the idealized welfare loss, $L_M^U\left(r^{U}(y_c),y_s\right)$ or $L_W^U\left(r^{U}(y_c),y_s,H\right)$, and the corresponding measure of deliberative competence, $d_e\left(r^{V}(y_c),r^{V}(y_s)\right)$ for $e \in \{W,M\}$, are approximately proportional (in the sense that they become exactly proportional in the limit as $\alpha \rightarrow0$), where the factor of proportionality does not depend on the decision maker's misperceptions of consequences.

\begin{proposition} \label{Prop2-2L}

For each $e\in\{W,M\}$, there exists a strictly positive constant,
$K_{e}$, such that for all $y_{c}$ and $y_{s}$ satisfying $\left.\frac{dr^{u}(y_{c}^{\alpha})}{d\alpha}\right|_{\alpha=0}\neq\left.\frac{dr^{u}(y_{s}^{\alpha})}{d\alpha}\right|_{\alpha=0}$, we have

\begin{equation}
\lim_{\alpha\rightarrow0}\left(\frac{L_{M}^{U}\left(r^{U}(y_{c}^{\alpha}),y_{s}^{\alpha}\right)}{d_{e}\left(r^{V}\left(y_{c}^{\alpha}\right),r^{V}\left(y_{s}^{\alpha}\right)\right)}\right)=K_{M}\label{eq:Prop5a}
\end{equation}
\begin{equation}
\lim_{\alpha\rightarrow0}\left(\frac{L_{W}^{U}\left(r^{U}(y_{c}^{\alpha}),y_{s}^{\alpha},H^{\alpha}\right)}{d_{e}\left(r^{V}\left(y_{c}^{\alpha}\right),r^{V}\left(y_{s}^{\alpha}\right)\right)}\right)=K_{W}\label{eq:Prop5b}
\end{equation}

\end{proposition}


Proposition \ref{Prop2-2L} (for idealized welfare analysis) is weaker than Proposition \ref{Prop1-2L} (for myopic welfare analysis) in only one respect: it may be the case that $K_M \ne 1$. Even with that qualification, several important conclusions follow.

First, we can conduct useful welfare analysis in a compartmentalized fashion \emph{even when we do not have estimates of the parameters governing other biases outside the scope of the analysis}.\footnote{Naturally, if we can measure the parameters governing the other biases, we can compute the factors of proportionality, $K_e(y_s)$, and then estimate the absolute welfare effects of various corrective policies by making simple multiplicative adjustments. Such exercises would still fall within the scope of idealized welfare analysis rather than comprehensive second-best analysis, in that they would use estimates of other biases to ``correct'' the data before evaluating the policy of interest, without explicitly modeling the ancillary remedies that would achieve those corrections.} Specifically, analogously to Proposition \ref{Prop1-2L} (which concerned the myopic welfare losses $L_{M}^{V}\left(r^{V}(y_{c}),y_{s}\right)$ and $L_{W}^{V}\left(r^{V}(y_{c}),y_{s},H\right)$), even though the bias-free utility function $U$ is potentially unknown and unobservable, we can infer the ranking of corrective policies according to the idealized welfare loss, $L_{M}^{U}\left(r^{U}(y_{c}),y_{s}\right)$ or $L_{W}^{U}\left(r^{U}(y_{c}),y_{s}, H\right)$, by ranking them according to the corresponding index of deliberative competence, $d_{e}\left(r^{V}(y_{c}),r^{V}(y_{s})\right)$, which is measurable. We can even gauge the percentage differences between the dollar-equivalents of the benefits flowing from these policies. We can also aggregate over instruments, subject to the same qualifications as in Section \ref{OneBias}. Thus, for our experiment, columns 1 and 2 of Table \ref{finComp} show that the intervention in Experiment A reduces the idealized welfare loss associated with poor comprehension of compound interest by only $6.5\%$ (that is, $1 - 22.864/24.448 = 0.065$, s.e. = 0.095), while the intervention in Experiment B reduces it by $27.7\%$ (that is, $1 - 18.569/25.673 = 0.277$, s.e. = 0.060).


Second, $d_{e}\left(r^{V}(y_{c}),r^{V}(y_{s})\right)$ remains a valid approximation of idealized welfare (up to an unknown multiplicative scalar) even when the analyst is unaware that certain types of other biases exist. This robustness property is reassuring.

Third, to the extent our method is robust to certain known and unknown biases (up to a multiplicative scalar), it is also robust to normative ambiguity about such biases \citep[i.e., the possibility that two different utility functions $U$ and $U'$  may both reflect legitimate normative perspectives, as permitted in the welfare framework of][]{BernheimRangelWelfare}. Hence, we can assess the effect of a policy $\theta$ on the welfare loss from mischaracterization within the framework of idealized welfare economics without needing to take a stance on thorny issues such as the normative interpretation of present focus (that is, whether and in what sense it constitutes a bias).


\subsubsection{An example}

\label{subsec:SimpleExample}

Proposition \ref{Prop2-2L} may initially strike the reader as counterintuitive, because it allows us to rank corrective policies according to an unknown function. The following simple example makes the logic more transparent.

For the purpose of this example, we interpret $m$ as current consumption, $y$ as a vector describing state-contingent consumption in future periods, and $I$ as a financial instrument that enables the consumer to convert the former into the latter. The consumer's beliefs about the instrument are mistaken in two ways. First, due to complex framing, she thinks the instrument promises to deliver $y_{c}-y_{0}$ rather than $y_{s}-y_{0}$. Second, she entertains the objectively groundless suspicion that the instrument is a scam that will pay nothing, leaving her future outcomes unchanged with probability $1-\pi$. We also assume that
immediate and future consequences, $m$ and $y$, enter decision utility additively through separate functions, $\nu(\cdot)$ and $\varphi(\cdot)$, respectively, where $\nu(m)=m$. Finally, we assume the consumer discounts future expected utility at the rate $\beta \delta$, where $\beta$ measures the extent of present bias: $V(m,y)=m+\beta\delta\varphi(y)$, and $U(m,y)=m+\delta\varphi(y)$. Our objective is to evaluate the idealized welfare effects of policies that seek to improve the consumer's understanding of promised consequences (i.e., to reduce the discrepancy between $y_{s}$ and $y_{c}$), treating excessive skepticism and present focus as ancillary biases, of which we may or may not be aware.

In frame $f$ (with the ancillary biases present), the consumer decides whether to purchase the instrument by comparing the perceived utility derived from purchase,
$
V(m_{0}-p,y_f) = m_{0} -p +\pi \beta \delta \varphi(y_{f}) + (1-\pi) \beta \delta \varphi(y_{0}),
$
with the perceived utility derived from no purchase,
$
V(m_{0},y_0) = m_{0} + \beta \delta \varphi(y_{0})
$.
The consumer's reservation price, $r^V(y_f)$, is the value of $p$ that equates these expressions:
$
r^{V}(y_f) = \pi\beta\delta \left(\varphi(y_{f})-\varphi(y_{0})\right).
$
Similarly,
$
r^{U}(y_f) = \delta\left(\varphi(y_{f})-\varphi(y_{0})\right).
$
In the case where $\varphi(y_s)>\varphi(y_c)$, we then have
\[
L_M^U\left(r^U(y_c),r^U(y_s)\right)=U\left(m_0-r^U(y_c),\varphi(y_s)\right)-U\left(m_0,\varphi(y_0)\right)
\]
\[
= \left[ m_0 - r^U(y_c) + \delta \varphi(y_s) \right]- \left[ m_0 + \delta \varphi(y_0) \right] = \delta \left[\varphi(y_s) - \varphi(y_c)\right]
\]
We also have
\[
d_M\left(r^V(y_c),r^V(y_s)\right) = r^V(y_s)-r^V(y_c) = \pi \beta \delta \left[\varphi(y_s) - \varphi(y_c)\right]
\]
Putting these observations together yields
\[
L_M^U\left(r^U(y_c),y_s\right)=\frac{1}{\pi\beta}d_M\left(r^V(y_c),r^V(y_s)\right)
\]
A similar argument applies when $\varphi(y_s)<\varphi(y_c)$. Thus, $K_M(y_s)=\frac{1}{\pi\beta}$. If $H$ is uniform within the relevant interval, we can likewise show that $L_W^U\left(r^U(y_c),y_s,H\right)$ is exactly proportional to $d_W\left(r^V(y_c),r^V(y_s)\right)$.

The implications of this analysis merit emphasis. Suppose an analyst evaluates educational interventions based on $d_M$ or $d_W$, but is unaware that consumers suffer from either present bias or false beliefs about experimenter reliability. The analysis would still yield valid measures of idealized welfare effects, up to an unknown constant of proportionality. Comparisons of relative benefits across policies would remain valid.

A notable feature of this example is that the relationship between the idealized welfare loss and our measure of deliberative competence is exact. Accordingly, the example shows that our approximations can yield accurate answers even when consequences and misunderstandings are both large.

Within the idealized welfare framework, these interpretations follow under the assumption that issues of present bias and excessive suspicion about experimenter reliability will be fully resolved through other corrective policies. In settings where this assumption is unacceptable, and in which there are no biases other than these two, narrow second-best welfare analysis is more appropriate. According to the logic of that framework, it is typically optimal for the consumer to entertain misunderstandings that counteract other biases. In the current example, the consumer incurs  a welfare loss of $
L_M^U\left(r^V(y_s), y_s\right) = \left| (\beta \pi - 1) \delta \big( \varphi(y_s) - \varphi(y_0)\big) \right|$ when she understands the instrument's consequences perfectly. As $\beta\pi$ shrinks, the potential gains from counteracting the associated bias grow, and consequently correcting the consumer's misunderstanding of $y_s$ becomes less beneficial.


\subsection{Policy-induced confounds}

\label{subsec:ConfoundingFraming}

So far, we have assumed that neither $V$ nor $U$ depends directly on the policies we seek to compare (e.g., $\theta$ and $\theta'$)---in other words, that there are no policy-induced confounds. This assumption is responsible for the invariance of the multiplicative constant, $K_e$, across policies, which these comparisons exploit.  Yet, it is possible that an intervention changes $V$, causing the consumer's simply framed valuations to vary with the policy. For example, if an educational intervention targeting the mechanics of interest compounding exhorts people to be patient, simply framed valuations for delayed payments might increase.

\begin{table}[htbp!]
\caption{Valuations in simply framed tasks}
\label{simpleFrame} \label{finCompBR}
\vspace{-0.4cm}
\begin{centering}

\begin{tabular}{lcccc}
\multicolumn{5}{c}{\textbf{}}\\
\midrule
 & (1)  & (2)  & (3)  & (4) \\
Delay in Days  & \multicolumn{2}{c}{72} & \multicolumn{2}{c}{36}\\
\cmidrule(r){2-3}\cmidrule(r){4-5} Experiment  & A  & B  & A  & B \\
\midrule
Levels  &  &  &  & \\
\hspace{0.4cm} \textit{Control}  & 68.637*** & 69.698*** & 75.874*** & 75.343*** \\
 & (2.246) & (1.766) & (2.015) & (1.628) \\
\hspace{0.4cm} \textit{Treatment}  & 69.513*** & 77.908*** & 75.801*** & 82.791*** \\
  & (2.265) & (1.707) & (2.076) & (1.451) \\
Difference  & 0.876 & 8.210*** & -0.073 & 7.448*** \\
 & (3.190) & (2.456) & (2.893) & (2.181) \\
 &  &  &  & \\
Observations  & 1,075 & 1,740 & 1,075 & 1,740 \\
Subjects  & 215 & 348 & 215    & 348 \\
\bottomrule
\end{tabular}
\par\end{centering}
\footnotesize
\textbf{\textit{Notes:}} Valuations in simply framed tasks, $r_{s}^{i,R,t}$, rescaled to range from 0 to 100 (percentage points). Each column displays the coefficients of a separate OLS regression. Standard errors in parentheses, clustered
by subject. {*}{*}{*}$p<0.01,${*}{*}{*}$p<0.05,${*}$p<0.1$.
\end{table}

This concern arises in our experiment. Table \ref{simpleFrame} presents regressions describing valuations in the simple frame, separately for each timeframe. Columns 1 and 3 show that, in Experiment A, there are no treatment effects on these valuations. In each condition, subjects value a dollar received with a 72-day delay at roughly 70 cents, and a dollar received with a 32-day delay at roughly 75 cents.\footnote{These magnitudes are typical for studies that elicit time preferences over short horizons \citep{Fredericketal.Review}. Not only do subjects discount the future heavily, but the discount function decreases much less steeply for longer delays than for short delay. Part of the explanation may involve perceptions of experimenter reliability.} In contrast, a treatment effect is present in Experiment B. The valuations of subjects in the Control condition are similar to those elicited from subjects in Experiment A, as columns 2 and 4 show. Average simply framed valuations in the Treatment condition, however, are higher by 8.2 and 7.4 cents for the 72-day and 36-day timeframes, respectively.

How should an analyst proceed if the data reveal these types of confounds? As we show next, a simple adjustment to our measure of deliberative competence preserves all the implications of Proposition \ref{Prop2-2L}.


Formally, we allow the indirect utility function $V$ to depend on policies $\theta$ directly rather than only through the subjects' misperception of the instrument's consequences, $y$. That is, we write $V\left(m,\varphi\left(y\right),\theta\right)$. Because $\theta$ influences behavior through valuation biases, we assume it does not directly affect the bias-free utility function, $U\left(m,\varphi\left(y\right)\right)$. Notationally, we include $\theta$ as an argument of $r^{V}$ to accommodate the confounding framing effects.


Our next proposition identifies the adjustment to measured deliberative competence required to restore comparability across policies.

\begin{proposition} \label{Prop3-2L}

There exist a strictly positive function, $\kappa(y_{s})$, such that
for all values of $y_{c\theta}$ and $y_{s}$ satisfying $\left.\frac{dr^{u}(y_{c\theta}^{\alpha})}{d\alpha}\right|_{\alpha=0}\neq\left.\frac{dr^{u}(y_{s}^{\alpha})}{d\alpha}\right|_{\alpha=0}$, and all policies $\theta$, we have

\begin{equation}
\lim_{\alpha\rightarrow0}\left[\left(\frac{L_{M}^{U}\left(r^{U}(y_{c\theta}^{\alpha}),y_{s}^{\alpha}\right)}{d_{M}\left(r^{V}\left(y_{c\theta}^{\alpha},\theta\right),r^{V}\left(y_{s}^{\alpha},\theta\right)\right)}\right)-\frac{\alpha\kappa(y_{s})}{r^{V}(y_{s}^{\alpha},\theta)}\right]=0\label{eq:Prop6a}
\end{equation}
and
\begin{equation}
\lim_{\alpha\rightarrow0}\left[\left(\frac{L_{W}^{U}\left(r^{U}(y_{c\theta}^{\alpha}),y_{s}^{\alpha},H^{\alpha}\right)}{d_{W}\left(r^{V}\left(y_{c\theta}^{\alpha},\theta\right),r^{V}\left(y_{s}^{\alpha},\theta\right)\right)}\right)-h\left(\frac{\alpha\kappa(y_{s})}{r^{V}(y_{s}^{\alpha},\theta)}\right)^{2}\right]=0\label{eq:Prop6b}
\end{equation}
Furthermore, when $y$ is a scalar, $\kappa(y_s)=(y_s-y_0)\kappa^*$ for some strictly positive constant $\kappa^*$.

\end{proposition}

According to this proposition, we can address the confound simply by rescaling our measures of deliberative competence. Specifically, using the proposition to generate approximations for the case of $\alpha=1$, we conclude that $L_{M}^{U}\left(r^{U}(y_{c\theta}),y_{s}\right)$ is proportional to $\frac{d_{M}\left(r^{V}\left(y_{c\theta},\theta\right),r^{V}\left(y_{s},\theta\right)\right)}{r^{V}(y_{s},\theta)}$, and that $L_{W}^{U}\left(r^{U}(y_{c\theta}),y_{s},H\right)$ is proportional to $\frac{d_{W}\left(r^{V}\left(y_{c\theta},\theta\right),r^{V}\left(y_{s},\theta\right)\right)}{\left(r^{V}(y_{s},\theta)\right)^2}$, where the constants of proportionality are independent of $y_c$, and consequently of the policy $\theta$.

The fact that the constants of proportionality depend on $y_s$ (through $\kappa(y_s)$) introduces a qualification when aggregating over instruments. However, for the case in which $y_s$ is a scalar, the proposition implies that $L_{M}^{U}\left(r^{U}(y_{c\theta}),y_{s}\right)$ is approximately proportional to $\frac{d_{M}\left(r^{V}\left(y_{c\theta},\theta\right),r^{V}\left(y_{s},\theta\right)\right)}{r^{V}(y_{s},\theta)/(y_s-y_0)}$, and that $L_{W}^{U}\left(r^{U}(y_{c\theta}),y_{s},H\right)$ is approximately proportional to $\frac{d_{W}\left(r^{V}\left(y_{c\theta},\theta\right),r^{V}\left(y_{s},\theta\right)\right)}{\left(r^{V}(y_{s},\theta)/(y_s-y_0)\right)^2}$, where the constants of proportionality are also independent of $y_s$ (apart from the possibility that $y_s$ is related to $h$). In other words, to restore all the implications of Proposition \ref{Prop2-2L}, we simply rescale the distance metrics by $r^{V}(y_{s},\theta)/(y_s-y_0)$ and  $\left(r^{V}(y_{s},\theta)/(y_s-y_0)\right)^2$, respectively.


We use this proposition to reevaluate the data from our experiment.\footnote{
Recall that our empirical analysis involves normalized valuations, which equal the raw valuations divided by the future payments. To mitigate noise in the simply framed elicitations, we average over each subject's simply framed valuations for each timeframe and use the resulting mean to correct complexly framed choices by that subject in that timeframe. Note that the normalized future payoff, $y_s(I)-y_0$, is 1. Consequently, when we rescale the distance metrics by $r^{V}(y_{s},\theta)$ and  $\left(r^{V}(y_{s},\theta)\right)^2$, respectively, we are also rescaling by $r^{V}(y_{s},\theta)/(y_s-y_0)$ and  $\left(r^{V}(y_{s},\theta)/(y_s-y_0)\right)^2$, which gives us comparability across instruments (as noted above).}  The correction effectively changes the units of measurement: the differences in Table \ref{finComp} reflect subjects' valuations of future prospects in terms of immediate payments, while division by simply framed valuations reflects subjects valuations of future prospects in terms of future payments.

Table \ref{finCompCorrection} reproduces the results of Table \ref{finComp} with the corrected measure of financial competence. Columns 1 and 2 average across both timeframes; the remaining columns display estimates separately within each timeframe. We continue to observe large and highly statistically significant improvements in deliberative competence in Experiment B; the estimated relative improvement is now $1 - 25.491/37.541 = 0.321$ (s.e. 0.058). Treatment effects in Experiment A remain insubstantial and statistically distinguishable from zero.


\begin{table}[htbp!]
\setlength{\tabcolsep}{3pt}


\begin{centering}
\caption{Deliberative Competence corrected for policy-induced framing effects. \label{finCompCorrection} }
\begin{tabular}{lcccccc}
\midrule
 & (1)  & (2)  & (3)  & (4)  & (5)  & (6) \\

Delay in days  & 72 and 36  & 72 and 36  & 72  & 72  & 36  & 36 \\
\cmidrule(r){2-3} \cmidrule(r){4-5} \cmidrule(r){6-7} Experiment  & A  & B  & A  & B  & A  & B \\
\midrule
 &  &  &  &  &  & \\
Levels  &  &  &  &  &  & \\
 \quad \textit{Control} & -34.639*** & -37.541*** & -36.246*** & -39.326*** & -33.032*** & -35.756*** \\
 & (2.115) & (1.906) & (2.535) & (2.127) & (2.053) & (2.001) \\
\quad \textit{Treatment}  & -37.204*** & -25.491*** & -38.363*** & -26.293*** & -36.044*** & -24.689*** \\
 & (3.787) & (1.775) & (3.882) & (1.865) & (4.202) & (1.882) \\
\quad \textit{Substance-Only}  & -37.606*** &  & -40.610*** &  & -34.602*** &  \\
 & (4.710) &  & (6.550) &  & (3.210) &  \\
\quad \textit{Rhetoric-Only}  & -29.728*** &  & -32.242*** &  & -27.214*** &  \\
 & (2.427) &  & (2.808) &  & (2.337) &  \\
 &  &  &  &  &  &  \\
$p$-value of difference \\
to Control \\
\quad \textit{Treatment}  & 0.555 & 0.000 & 0.648 & 0.000 & 0.520 & 0.000 \\
\quad \textit{Substance-Only} & 0.566 &  & 0.535 &  & 0.680 &  \\
 \quad \textit{Rhetoric-Only} & 0.128 &  & 0.290 &  & 0.062 &  \\
Observations & 4,550 & 3,470 & 2,275 & 1,735 & 2,275 & 1,735 \\
Subjects & 455 & 347 & 455 & 347 & 455 & 347 \\
\bottomrule
\end{tabular}
\par\end{centering}
\footnotesize
\textbf{\textit{Notes:}} Each column displays the coefficients of a separate OLS regression of deliberative competence $d_M^{i,R,t}=-|r_{c}^{i,R,t}-r_{s}^{i,R,t}|$, corrected for the change in simply framed valuations, on treatment indicators. One subject in experiment B revealed valuations of 0 in each simply framed task. Because the correction entails dividing by the within-subject mean of simply framed valuations, we exclude this subject from the analysis. ***$p<0.01,$***$p<0.05,$*$p<0.1$.
\end{table}


\subsection{Interpersonal aggregation}

\label{subsec:Aggregation}

So far, we have focused on evaluating the welfare effects of opportunity-neutral interventions on a single individual. How can an analyst assess the aggregate welfare effect of such interventions on a heterogeneous population? Equipped with a set of welfare weights that reflect the relative social value of an extra dollar received by each individual, one can meaningfully aggregate money-metric welfare losses over members of a population. That standard procedure becomes more challenging in the current context because, unless we adopt the maximal loss criterion and assume away biases outside the scope of the analysis, we measure dollar-equivalent welfare losses up to a constant of proportionality that can potentially vary from one person to the next. To focus on that issue, we will assume for simplicity that the social value of an extra dollar is the same for everyone.

Proposition \ref{Prop2-2L} tells us that, as an approximation, the welfare loss for individual $j$ given perspective $e\in \{W,M\}$ (for some specified instrument and policy), abbreviated $L_e^{U}(j)$, is approximately equal to measured deliberative competence for individual $j$, abbreviated $d_e^V(j)$, times a constant $K_e(j)$. It follows that the aggregate (average) welfare loss is
$
\overline{L}_e^{U}=\overline{d}_e\overline{K}_e+\mathrm{cov}\left(d_e,K_e\right),
$
where we use bars to denote means across individuals for each variable.

Our objective is to compare policies according to their average idealized welfare losses, $\overline{L}_{e}^{U}$. Assume for the moment that decision-making defects within the scope of the analysis are uncorrelated with defects outside the scope of the analysis --  in other words, $\mathrm{cov}\left(d_{e}^{V},K_e\right)=0$. In that case, the invariance of each $K_e(j)$, and hence of $\overline{K}_e$, across corrective policies implies that the change in the aggregate welfare loss ($\overline{L}_e^{U}$) is a fixed multiple of the change in average deliberative competence ($\overline{d}_e$), which we can measure.

Unfortunately, biases tend to be correlated across domains \citep{DeanOrtoleva,StangoZinman.Bcounts,Econographics}.\footnote{In the case of $e=W$, the constant of proportionality $K_W(j)$ incorporates the density parameter $h\left(r^V(y_s)\right)$ for individual $j$. There is no particular reason to think this term is correlated with either $d_{W}^{V}(j)$ or with the other components of $K_W(j)$. To the extent these correlations are zero, heterogeneity in the density term does not impact $\bar{L}_W^U$.} The correlation is also unlikely to be stable across policies or to vary in proportion to $\overline{d}_e$. To illustrate, suppose present bias is associated with greater underestimation (less overestimation) of compound interest. A policy that shifts the entire population from underestimation to overestimation will reverse the correlation between $d_e^V(j)$ and the ancillary bias.


One strategy for addressing potentially problematic correlations between the biases of interest and other decision-making defects is to disaggregate into more homogeneous subsamples, and to evaluate whether results are robust with respect to subsample weights. While we obviously cannot disaggregate based on the magnitudes of unobserved biases, we can exploit the fact that such biases tend to impact simply framed valuations. For example, in our experiment, considerations such as present bias and false beliefs about experimenter reliability would suppress valuations of future rewards.


Accordingly, we sort subjects into quartiles based on their mean valuations of simply framed prospects (separately for each treatment group to account for the fact that the Treatment intervention in Experiment B affected those valuations). For each experiment, we then regress measured deliberative competence on a Treatment dummy, an indicator for each quartile of simply framed valuations, and an interaction between those indicators and the Treatment dummy. We cluster standard errors on the subject level.


\begin{table}[h]
\caption{Deliberative Competence by quartiles of simply framed valuations}
\label{finCompAggregation}
\setstretch{1}
\begin{centering}

\begin{tabular}{lcccc}
\multicolumn{5}{c}{\textbf{Experiment A}}\\
\toprule
VARIABLE & \multicolumn{4}{c}{Deliberative Competence}\\
\midrule
 & \multicolumn{4}{c}{Quartile simply framed valuation}\\
\cmidrule(r){2-5}  & Bottom  & 2nd from bottom  & 2nd from top  & Top \\
Levels  &  &  &  & \\
\hspace{0.4cm}\textit{Control}  & -17.007{*}{*}{*}  & -21.693{*}{*}{*}  & -26.589{*}{*}{*}  & -32.216{*}{*}{*} \\
 & (2.037)  & (2.596)  & (3.410)  & (3.893) \\
\hspace{0.4cm}\textit{Treatment}  & -24.967{*}{*}{*}  & -22.345{*}{*}{*}  & -22.855{*}{*}{*}  & -21.367{*}{*}{*} \\
 & (4.484)  & (2.779)  & (2.559)  & (3.772) \\
Effect  & -7.961  & -0.652  & 3.734  & 10.849{*}{*} \\
 & (4.925)  & (3.803)  & (4.263)  & (5.421) \\
\cmidrule(r){2-5} Observations  & \multicolumn{4}{c}{2,150}\\
Subjects  & \multicolumn{4}{c}{215}\\
\bottomrule
 &  &  &  & \\
\multicolumn{5}{c}{\textbf{Experiment B}}\\
\toprule
VARIABLE & \multicolumn{4}{c}{Deliberative Competence}\\
\midrule
 & \multicolumn{4}{c}{Quartile simply framed valuation}\\
\cmidrule(r){2-5}  & Bottom  & 2nd from bottom  & 2nd from top  & Top \\
Levels  &  &  &  & \\
\hspace{0.4cm}\textit{Control}  & -17.734{*}{*}{*}  & -25.920{*}{*}{*}  & -25.706{*}{*}{*}  & -33.156{*}{*}{*} \\
 & (1.893)  & (2.073)  & (2.414)  & (3.605) \\
\hspace{0.4cm}\textit{Treatment}  & -17.818{*}{*}{*}  & -21.239{*}{*}{*}  & -17.215{*}{*}{*}  & -18.018{*}{*}{*} \\
 & (2.155)  & (2.392)  & (2.254)  & (2.547) \\
Effect  & -0.085  & 4.681  & 8.492{*}{*}  & 15.138{*}{*}{*} \\
 & (2.869)  & (3.165)  & (3.302)  & (4.414) \\
\cmidrule(r){2-5} Observations  & \multicolumn{4}{c}{3,480}\\
Subjects  & \multicolumn{4}{c}{348}\\
\bottomrule
\end{tabular}
\par\end{centering}
\footnotesize
\textbf{\textit{Notes:}}
Effect of treatments on deliberative competence,  $d_M^{i,R,t}=-|r_{c}^{i,R,t}-r_{s}^{i,R,t}|$, by quartiles of simply framed valuations, rescaled to range from 0 to 100 (percentage points). Each panel presents the
output of a single OLS regression. Standard errors in parentheses,
clustered by subject. {*}{*}{*}$p<0.01,${*}{*}{*}$p<0.05,${*}$p<0.1$.
\end{table}


Table \ref{finCompAggregation} displays the results. Within each quartile, the Treatment effect is larger in Experiment B than in Experiment A by several percentage points. For Experiment B, the benefit of the intervention is roughly zero for subjects in the bottom quartile, positive for all other quartiles, and statistically significant for the top two quartiles. For Experiment A, the benefit is negative and substantial (though not statistically significant at conventional levels) for those in the lowest quartile, slightly negative for those in the second quartile, and positive for the top two quartiles, but statistically significant only for the top quartile. We conclude that the Treatment intervention in Experiment B has a (weakly) positive effect on deliberative competence regardless of the welfare weights assigned to different subjects based on a proxy for biases outside the scope of the analysis, whereas the Treatment intervention in Experiment A has mixed effects, clearly benefiting only those with the highest simply framed valuations. Because the Treatment intervention in Experiment B has a larger positive effect within each quartile than the intervention in Experiment A, it is unlikely that our conclusions concerning the benefits of practice and individualized feedback are driven by a spurious relationship between welfare weights and extraneous biases.


\section{Conclusion}

\label{conclusion}


We have developed a simple method for evaluating opportunity-neutral policies that seek to improve the quality of decision making. It involves simple transformations of the price-metric bias, which are interpretable robustly as measures of the welfare loss resulting from poor comprehension of consequences. We illustrated its advantages over conventional evaluative metrics in an application involving financial education.

The method lends itself to many other applications. Maintaining a focus on financial education, one could use it to evaluate training pertaining to loans and leases, rent-to-own agreements, portfolio diversification, contracts with multi-part tariffs, and insurance products. One could also evaluate alternatives to education, such as peer advice \citep[as we show in][]{BLB}, professional guidance, disclosure rules, and informational labels.


Outside the financial domain, our method offers a potential avenue for evaluating a variety of opportunity-neutral policies that target specific biases. As in other recent work in Behavioral Public Economics \citep[reviewed in][]{BernheimTaubinsky}, the main requirement is that one can contrive versions of valuation tasks that plausibly remove the bias of interest. This requirement is most easily satisfied when the bias pertains to a specific misconception about the consequences of particular actions, and where one can describe the options in terms of their consequences without triggering the misconception. For example, to evaluate the benefits of ``traffic light'' labels concerning energy efficiency for household appliances, ``simple framing'' might entail full and correct information concerning energy costs \citep[as in][]{AllcottTaubinsky.Lightbulbs}. One can imagine similar applications involving nutritional labels and other health advisories. Similarly,  using the type of data gathered in \citet{Akessonetal2022}, one could measure the welfare benefits of interventions that aim to help consumers identify fake reviews on online platforms.

Naturally, the approach has limitations, particularly regarding robustness with respect to biases outside the scope of the analysis, which we highlighted in our theoretical discussion. Idealized welfare analysis overcomes the Lipsey-Lancaster critique of ``piecemeal'' welfare analysis by making a couple of conceptual compromises, which we have stated explicitly. It also involves a separability assumption that may or may not be plausible in the context of any given application. A practical limitation is that implementation requires the analyst to observe an individual's simply and complexly framed valuations for the same instrument. Because naturally occurring processes do not generally produce such data, potential applications are limited to experiments. While the current investigation involves a laboratory experiment with stylized tasks, applications involving field experiments with more naturalistic tasks are also feasible.


\newpage{}

{\bibsep}{0pt plus 0.3ex}


\begin{thebibliography}{99}
\bibitem{replicationData} Sandro Ambuehl; B. Douglas Bernheim; Annamaria Lusardi. Data and Code for: Evaluating Deliberative Competence: A Simple Method with an Application to Financial Choice. American Economic Association. 2022.
\bibitem{Akessonetal2022} Jesper Akesson; Robert W. Hahn; Robert D. Metcalfe; Manuel Monti-Nussbaum. The Impact of Fake Reviews on Demand and Welfare. \textit{Unpublished manuscript}. 2022.
\bibitem{Oster} Oster, Emily. Health recommendations and selection in health behaviors. \textit{American Economic Review: Insights}. vol. 2. no. 2. pp. 143--60. 2020.
\bibitem{Meade} Meade, James Edward. The theory of customs unions. North-Holland Publishing. 1955.
\bibitem{ariely2003coherent} Ariely, Dan; Loewenstein, George; Prelec, Drazen. “Coherent arbitrariness”: Stable demand curves without stable preferences. \textit{The Quarterly Journal of Economics}. MIT Press. vol. 118. no. 1. pp. 73--106. 2003.
\bibitem{bernheim2021defense} Bernheim, B Douglas. In defense of behavioral welfare economics. \textit{Journal of Economic Methodology}. Taylor \& Francis. vol. 28. no. 4. pp. 385--400. 2021.
\bibitem{SaezStantcheva} Saez, Emmanuel; Stantcheva, Stefanie. Generalized social marginal welfare weights for optimal tax theory. \textit{American Economic Review}. vol. 106. no. 1. pp. 24--45. 2016.
\bibitem{KoszegiRabin} K{\H{o}}szegi, Botond; Rabin, Matthew. A model of reference-dependent preferences. \textit{The Quarterly Journal of Economics}. MIT Press. vol. 121. no. 4. pp. 1133--1165. 2006.
\bibitem{BDM} Becker, Gordon M; DeGroot, Morris H; Marschak, Jacob. Measuring utility by a single-response sequential method. \textit{Behavioral science}. Wiley Online Library. vol. 9. no. 3. pp. 226--232. 1964.
\bibitem{PlottZeiler} Plott, Charles R; Zeiler, Kathryn. The willingness to pay-willingness to accept gap, the" endowment effect," subject misconceptions, and experimental procedures for eliciting valuations. \textit{American Economic Review}. vol. 95. no. 3. pp. 530--545. 2005.
\bibitem{McMahonWiel} McMahon, James E; Wiel, Stephen. Energy-efficiency labels and standards: A guidebook for appliances, equipment and lighting. 2005.
\bibitem{CFPBprivate} CFPB. Navigating the Market: A comparison of spending on financial education and financial marketing. \textit{Washington, DC: Consumer Financial Protection Bureau}. 2013.
\bibitem{OECDpolicyHandbook} OECD. National Strategies for Financial Education. 2015.
\bibitem{OECDfinLitEU} OECD. Launch of the EC/OECD-INFE project to develop a financial competence framework for the EU. 2021.
\bibitem{EUactionPlan} European Commission. A Capital Markets Union for people and businesses-new action plan. vol. Document 52020DC0590. 2020.
\bibitem{Willis2021} Willis, Lauren E. Alternatives to Financial Education. \textit{Loyola Law School, Los Angeles Legal Studies Research Paper}. no. 2021-11. 2021.
\bibitem{Milkmanetal.Debiasing} Soll, Jack B; Milkman, Katherine L; Payne, John W. A User's Guide to Debiasing. \textit{The Wiley Blackwell Handbook of Judgment and Decision Making}. Wiley Online Library. vol. 2. pp. 924--951. 2015.
\bibitem{BhargavaLoewenstein} Bhargava, Saurabh; Loewenstein, George. Behavioral economics and public policy 102: Beyond nudging. \textit{American Economic Review}. vol. 105. no. 5. pp. 396--401. 2015.
\bibitem{Bushongetal.} Bushong, Benjamin; Rabin, Matthew; Schwartzstein, Joshua. A model of relative thinking. \textit{The Review of Economic Studies}. Oxford University Press. vol. 88. no. 1. pp. 162--191. 2021.
\bibitem{KoszegiSzeidl} K{\H{o}}szegi, Botond; Szeidl, Adam. A model of focusing in economic choice. \textit{The Quarterly Journal of Economics}. Oxford University Press. vol. 128. no. 1. pp. 53--104. 2013.
\bibitem{RizzoWhitman.Hayekian} Rizzo, Mario J; Whitman, Glen. The unsolved Hayekian knowledge problem in behavioral economics. \textit{Behavioural Public Policy}. Cambridge University Press. pp. 1--13. 2021.
\bibitem{BhargavaConellPrice} Bhargava, Saurabh; Conell-Price, Lynn. Serenity Now, Save Later? Evidence on Retirement Savings Puzzles from a 401 (k) Field Experiment. \textit{Unpublished}. 2020.
\bibitem{DellaVigna.StructuralBehavioral} DellaVigna, Stefano. Structural behavioral economics. In Handbook of Behavioral Economics: Applications and Foundations 1. Elsevier. vol. 1. pp. 613--723. 2018.
\bibitem{GoldinReck} Goldin, Jacob; Reck, Daniel. Revealed-Preference Analysis with Framing Effects. \textit{Journal of Political Economy}. The University of Chicago Press Chicago, IL. vol. 128. no. 7. pp. 2759--2795. 2020.
\bibitem{Bernheim.UnsophisticatedPlans} Bernheim, B. Douglas. Personal Saving, Information, and Economic Literacy: New Directions for Public Policy. In Tax Policy for Economic Growth in the 1990s. American Council for Capital Formation. pp. 53-78. 1994.
\bibitem{Rasmussen} Rasmussen, Christopher James. Effective cost-sharing models in higher education: Insights from low-income students in Australian Universities. \textit{Higher Education}. Springer. vol. 51. no. 1. pp. 1--25. 2006.
\bibitem{CallenderJackson} Callender, Claire; Jackson, Jonathan. Does the fear of debt deter students from higher education?. \textit{Journal of social policy}. Cambridge University Press. vol. 34. no. 4. pp. 509--540. 2005.
\bibitem{Schleichetal.} Schleich, Joachim; Faure, Corinne; Meissner, Thomas. Adoption of retrofit measures among homeowners in {EU} countries: The effects of access to capital and debt aversion. \textit{Energy Policy}. Elsevier. pp. 112025. 2020.
\bibitem{Allcottetal.RamseyStrikesBack} Allcott, Hunt; Lockwood, Benjamin; Taubinsky, Dmitry. Ramsey Strikes Back: Optimal Commodity Tax and Redistribution in the Presence of Salience Effects. In AEA Papers and Proceedings. vol. 108. pp. 88--92. 2018.
\bibitem{NudgeBook} Thaler, Richard H; Sunstein, Cass R. Nudge: Improving decisions about health, wealth, and happiness. Penguin. 2009.
\bibitem{DeJansetal.} De Jans, Steffi; Hudders, Liselot; Cauberghe, Veroline. Advertising literacy training: The immediate versus delayed effects on children's responses to product placement.. \textit{European Journal of Marketing}. Emerald Publishing. 2017.
\bibitem{Nelson} Nelson, Michelle R. Developing persuasion knowledge by teaching advertising literacy in primary school. \textit{Journal of Advertising}. Taylor \& Francis. vol. 45. no. 2. pp. 169--182. 2016.
\bibitem{NelsonKehr} Nelson, Michelle R; Kehr, Dale P. Food-focused advertising literacy can increase nutrition knowledge in elementary school students. \textit{Journal of nutrition education and behavior}. Elsevier. vol. 48. no. 10. pp. 749--751. 2016.
\bibitem{Hwangetal.} Hwang, Yoori; Yum, Jung-Yoon; Jeong, Se-Hoon. What Components Should Be Included in Advertising Literacy Education? Effect of Component Types and the Moderating Role of Age. \textit{Journal of Advertising}. Taylor \& Francis. vol. 47. no. 4. pp. 347--361. 2018.
\bibitem{Rozendaaletal.} Rozendaal, Esther; Lapierre, Matthew A; Van Reijmersdal, Eva A; Buijzen, Moniek. Reconsidering advertising literacy as a defense against advertising effects. \textit{Media Psychology}. Taylor \& Francis. vol. 14. no. 4. pp. 333--354. 2011.
\bibitem{Liaoetal.} Liao, Li-Ling; Lai, I-Ju; Chang, Li-Chun; Lee, Chia-Kuei. Effects of a food advertising literacy intervention on Taiwanese children's food purchasing behaviors. \textit{Health education research}. Oxford University Press. vol. 31. no. 4. pp. 509--520. 2016.
\bibitem{Haetal.} Ha, Oh-Ryeong; Killian, Haley; Bruce, Jared M; Lim, Seung-Lark; Bruce, Amanda S. Food Advertising Literacy Training Reduces the Importance of Taste in Children's Food Decision-Making: A Pilot Study. \textit{Frontiers in Psychology}. Frontiers. vol. 9. pp. 1293. 2018.
\bibitem{Huddersetal.} Hudders, Liselot; Cauberghe, Veroline; Panic, Katarina. How advertising literacy training affect children's responses to television commercials versus advergames. \textit{International Journal of Advertising}. Taylor \& Francis. vol. 35. no. 6. pp. 909--931. 2016.
\bibitem{Blaschetal.} Blasch, Julia; Filippini, Massimo; Kumar, Nilkanth; Martinez-Cruz, Adan L. Narrowing the energy efficiency gap: The impact of educational programs, online support tools and energy-related investment literacy. \textit{Online Support Tools and Energy-Related Investment Literacy (September 21, 2017)}. 2017.
\bibitem{Karpudewan} Karpudewan, Mageswary; Ponniah, Jamunah; Zain, Ahmad Nurulazam Md. Project-based learning: An approach to promote energy literacy among secondary school students. \textit{The Asia-Pacific Education Researcher}. Springer. vol. 25. no. 2. pp. 229--237. 2016.
\bibitem{Dwyer} Dwyer, Catherine; others. The relationship between energy literacy and environmental sustainability. \textit{Low Carbon Economy}. Scientific Research Publishing. vol. 2. no. 03. pp. 123. 2011.
\bibitem{Schwartzetal.} Schwartz, Tobias; Denef, Sebastian; Stevens, Gunnar; Ramirez, Leonardo; Wulf, Volker. Cultivating energy literacy: results from a longitudinal living lab study of a home energy management system. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 1193--1202. 2013.
\bibitem{Bodzinetal.} Bodzin, Alec M; Fu, Qiong; Peffer, Tamara E; Kulo, Violet. Developing energy literacy in US middle-level students using the geospatial curriculum approach. \textit{International Journal of Science Education}. Taylor \& Francis. vol. 35. no. 9. pp. 1561--1589. 2013.
\bibitem{DeWatersPowers} DeWaters, Jan E; Powers, Susan E. Energy literacy of secondary students in New York State (USA): A measure of knowledge, affect, and behavior. \textit{Energy policy}. Elsevier. vol. 39. no. 3. pp. 1699--1710. 2011.
\bibitem{Brounenetal.} Brounen, Dirk; Kok, Nils; Quigley, John M. Residential energy use and conservation: Economics and demographics. \textit{European Economic Review}. Elsevier. vol. 56. no. 5. pp. 931--945. 2012.
\bibitem{Benjaminetal.Reconsidering} Benjamin, Daniel; Fontana, Alan; Kimball, Miles. Reconsidering risk aversion. \textit{Presentation: Economic Science Association North American Meeting}. 2019.
\bibitem{BenkertNetzer} Benkert, Jean-Michel; Netzer, Nick. Informational requirements of nudging. \textit{Journal of Political Economy}. University of Chicago Press Chicago, IL. vol. 126. no. 6. pp. 2323--2355. 2018.
\bibitem{DellaVignaMalmendier.ContractDesign} DellaVigna, Stefano; Malmendier, Ulrike. Contract design and self-control: Theory and evidence. \textit{The Quarterly Journal of Economics}. MIT Press. vol. 119. no. 2. pp. 353--402. 2004.
\bibitem{GruberKoszegi.TaxIncidence} Gruber, Jonathan; K{\H{o}}szegi, Botond. Tax incidence when individuals are time-inconsistent: the case of cigarette excise taxes. \textit{Journal of Public Economics}. Elsevier. vol. 88. no. 9-10. pp. 1959--1987. 2004.
\bibitem{GruberKoszegi.AddictionRational} Gruber, Jonathan; K{\H{o}}szegi, Botond. Is addiction ``rational''? Theory and evidence. \textit{The Quarterly Journal of Economics}. MIT Press. vol. 116. no. 4. pp. 1261--1303. 2001.
\bibitem{Econographics} Chapman, Jonathan; Dean, Mark; Ortoleva, Pietro; Snowberg, Erik; Camerer, Colin. Econographics. \textit{NBER Working Paper}. 2018.
\bibitem{StangoZinman.Bcounts} Stango, Victor; Zinman, Jonathan. We are all behavioral, more or less: Measuring and using consumer-level behavioral sufficient statistics. \textit{NBER Working Paper}. 2019.
\bibitem{DeanOrtoleva} Dean, Mark; Ortoleva, Pietro. The empirical relationship between nonstandard economic behaviors. \textit{Proceedings of the National Academy of Sciences}. National Acad Sciences. vol. 116. no. 33. pp. 16262--16267. 2019.
\bibitem{Benjaminetal.WhoIsBehavioral} Benjamin, Daniel J; Brown, Sebastian A; Shapiro, Jesse M. Who is `behavioral'? Cognitive ability and anomalous preferences. \textit{Journal of the European Economic Association}. Oxford University Press. vol. 11. no. 6. pp. 1231--1255. 2013.
\bibitem{Sunstein.Maximin} Sunstein, Cass R. Maximin. \textit{Yale Journal on Regulation}. 2020.
\bibitem{Perryetal.} Perry, Yael; Petrie, Katherine; Buckley, Hannah; Cavanagh, Lindy; Clarke, Deborah; Winslade, Matthew; Hadzi-Pavlovic, Dusan; Manicavasagar, Vijaya; Christensen, Helen. Effects of a classroom-based educational resource on adolescent mental health literacy: A cluster randomised controlled trial. \textit{Journal of adolescence}. Elsevier. vol. 37. no. 7. pp. 1143--1151. 2014.
\bibitem{Makiabadietal.} Makiabadi, Elham; Kaveh, Mohammad Hossein; Mahmoodi, Mohammad Reza; Asadollahi, Abdolrahim; Salehi, Mousa. Enhancing Nutrition-Related Literacy, Knowledge and Behavior among University Students: A Randomized Controlled Trial. \textit{International Journal of Nutrition Sciences}. Shiraz University of Medical Sciences. vol. 4. no. 3. pp. 122--129. 2019.
\bibitem{Warren} Warren, Cynthia. Addressing nutritional literacy in preschool learning environments. \textit{Journal of Family Strengths}. vol. 17. no. 1. pp. 6. 2017.
\bibitem{Liuetal.} Liu, Peggy J; Roberto, Christina A; Liu, Linda J; Brownell, Kelly D. A test of different menu labeling presentations. \textit{Appetite}. Elsevier. vol. 59. no. 3. pp. 770--777. 2012.
\bibitem{VariyamCawley} Variyam, Jayachandran N; Cawley, John. Nutrition labels and obesity. \textit{NBER Working Paper}. 2006.
\bibitem{Kimetal.} Kim, Sung-Yong; Nayga Jr, Rodolfo M; Capps Jr, Oral. The effect of food label use on nutrient intakes: an endogenous switching regression analysis. \textit{Journal of Agricultural and resource Economics}. JSTOR. pp. 215--231. 2000.
\bibitem{Elbeletal.} Elbel, Brian; Gyamfi, Joyce; Kersh, Rogan. Child and adolescent fast-food choice and the influence of calorie labeling: a natural experiment. \textit{International journal of obesity}. Nature Publishing Group. vol. 35. no. 4. pp. 493--500. 2011.
\bibitem{Levyetal.Cafeteria} Levy, Douglas E; Riis, Jason; Sonnenberg, Lillian M; Barraclough, Susan J; Thorndike, Anne N. Food choices of minority and low-income employees: a cafeteria intervention. \textit{American journal of preventive medicine}. Elsevier. vol. 43. no. 3. pp. 240--248. 2012.
\bibitem{BlumenthalVolpp} Blumenthal, Karen; Volpp, Kevin G. Enhancing the effectiveness of food labeling in restaurants. \textit{JAMA}. American Medical Association. vol. 303. no. 6. pp. 553--554. 2010.
\bibitem{Downsetal.AERPP} Downs, Julie S; Loewenstein, George; Wisdom, Jessica. Strategies for promoting healthier food choices. \textit{American Economic Review}. vol. 99. no. 2. pp. 159--64. 2009.
\bibitem{Downsetal.AJPH} Downs, Julie S; Wisdom, Jessica; Wansink, Brian; Loewenstein, George. Supplementing menu labeling with calorie recommendations to test for facilitation effects. \textit{American Journal of Public Health}. American Public Health Association. vol. 103. no. 9. pp. 1604--1609. 2013.
\bibitem{Wisdometal.AEJapplied} Wisdom, Jessica; Downs, Julie S; Loewenstein, George. Promoting healthy choices: Information versus convenience. \textit{American Economic Journal: Applied Economics}. vol. 2. no. 2. pp. 164--78. 2010.
\bibitem{CaplinMartinWelfare} Caplin, Andrew; Martin, Daniel J. Framing, Information, and Welfare. \textit{NBER Working Paper}. 2020.
\bibitem{FarhiGabaix} Farhi, Emmanuel; Gabaix, Xavier. Optimal taxation with behavioral agents. \textit{American Economic Review}. vol. 110. no. 1. pp. 298--336. 2020.
\bibitem{Afendulisetal.} Afendulis, Christopher C; Sinaiko, Anna D; Frank, Richard G. Dominated choices and Medicare Advantage enrollment. \textit{Journal of economic behavior \& organization}. Elsevier. vol. 119. pp. 72--83. 2015.
\bibitem{SinaikoHirth} Sinaiko, Anna D; Hirth, Richard A. Consumers, health insurance and dominated choices. \textit{Journal of Health Economics}. Elsevier. vol. 30. no. 2. pp. 450--457. 2011.
\bibitem{Mirrlees} Mirrlees, James A. An exploration in the theory of optimum income taxation. \textit{The Review of Economic Studies}. JSTOR. vol. 38. no. 2. pp. 175--208. 1971.
\bibitem{Harberger.Triangle} Harberger, Arnold C. The measurement of waste. \textit{The American Economic Review}. JSTOR. vol. 54. no. 3. pp. 58--76. 1964.
\bibitem{bernheimFinancialIlliteracy} Bernheim, Douglas. Financial illiteracy, education and retirement saving. \textit{Living with defined contribution pensions}. vol. 3868. 1998.
\bibitem{taubinskyReesJones} Taubinsky, Dmitry; Rees-Jones, Alex. Attention Variation and Welfare: Theory and Evidence from a Tax Salience Experiment. \textit{The Review of Economic Studies}. Oxford University Press. vol. 85. no. 4. pp. 2462--2496. 2018.
\bibitem{LipseyLancaster} Lipsey, Richard G; Lancaster, Kelvin. The general theory of second best. \textit{The Review of Economic Studies}. JSTOR. vol. 24. no. 1. pp. 11--32. 1956.
\bibitem{Cordesetal.EGB} Cordes, Henning; Foltice, Bryan; Langer, Thomas. Misperception of Exponential Growth: Are People Aware of Their Errors?. \textit{Decision Analysis}. INFORMS. vol. 16. no. 4. pp. 261--280. 2019.
\bibitem{Kaiseretal.FinEdReview} Kaiser, Tim; Lusardi, Annamaria; Menkhoff, Lukas; Urban, Carly. Financial education affects financial knowledge and downstream behaviors. \textit{Journal of Financial Economics}. Elsevier. 2021.
\bibitem{CarvalhoSilverman} Carvalho, Leandro; Silverman, Dan. Complexity and Sophistication. \textit{NBER Working Paper}. 2019.
\bibitem{Banksetal.} Banks, James; Carvalho, Leandro S; Perez-Arce, Francisco. Education, Decision Making, and Economic Rationality. \textit{Review of Economics and Statistics}. MIT Press. vol. 101. no. 3. pp. 428--441. 2019.
\bibitem{Beshearsetal.Review} Beshears, John; Choi, James J; Laibson, David; Madrian, Brigitte C. Behavioral household finance. In Handbook of Behavioral Economics: Applications and Foundations 1. Elsevier. vol. 1. pp. 177--276. 2018.
\bibitem{BernheimTaubinsky} Bernheim, B Douglas; Taubinsky, Dmitry. Behavioral public economics. In Handbook of Behavioral Economics: Applications and Foundations 1. Elsevier. vol. 1. pp. 381--516. 2018.
\bibitem{Luhrmannetal.2} Luhrmann, Melanie; Serra-Garcia, Marta; Winter, Joachim. The impact of financial education on adolescents' intertemporal choices. \textit{American Economic Journal: Economic Policy}. vol. 10. no. 3. pp. 309--32. 2018.
\bibitem{Campbell.Review} Campbell, John Y. Household finance. \textit{The journal of finance}. Wiley Online Library. vol. 61. no. 4. pp. 1553--1604. 2006.
\bibitem{Bruhnetal.Mexico} Bruhn, Miriam; Ibarra, Gabriel Lara; McKenzie, David. The minimal impact of a large-scale financial education program in Mexico City. \textit{Journal of Development Economics}. Elsevier. vol. 108. pp. 184--189. 2014.
\bibitem{Enkeetal.Stakes} Enke, Benjamin; Gneezy, Uri; Hall, Brian; Martin, David; Nelidov, Vadim; Offerman, Theo; van de Ven, Jeroen. Cognitive Biases: Mistakes or Missing Stakes?. \textit{The Review of Economics and Statistics}. pp. 1--45. 2021.
\bibitem{vNM.gameTheory} Morgenstern, Oskar; Von Neumann, John. Theory of games and economic behavior. Princeton university press. 1953.
\bibitem{MachinaSiniscalchi.AmbiguityReview} Machina, Mark J; Siniscalchi, Marciano. Ambiguity and ambiguity aversion. In Handbook of the Economics of Risk and Uncertainty. Elsevier. vol. 1. pp. 729--807. 2014.
\bibitem{Carroll.Review} Carroll, Gabriel. Robustness in mechanism design and contracting. \textit{Annual Review of Economics}. Annual Reviews. vol. 11. pp. 139--166. 2019.
\bibitem{Harrisonetal.IndexInsurance} Harrison, Glenn; Morsink, Karlijn; Schneider, Mark. Do No Harm? The Welfare Consequences of Behavioral Interventions. \textit{Unpublished}. 2020.
\bibitem{Sutteretal.FinLit} Matthias Sutter; Michael Weyland; Anna Untertrifaller; Manuel Froitzheim. Teaching financial literacy has an impact on risk and time preferences---Insights from a field experiment. \textit{Unpublished}. 2020.
\bibitem{Linnainmaaetal.FinancialAdvisersRiskTaking} Linnainmaa, Juhani; Melzer, Brian; Previtero, Alessandro; Foerster, Stephen. Financial advisors and risk-taking. \textit{Unpublished manuscript}. 2018.
\bibitem{Vanguard.AdvisorsAlpha} Bennyhoff, Donald G; Kinniry Jr, Francis M. Advisor's Alpha. \textit{Vanguard research paper (December). https://advisors. vanguard. com/iwe/pdf/ICRAA. pdf}. 2011.
\bibitem{oDonoghueRabin.SinTaxes} O'Donoghue, Ted; Rabin, Matthew. Optimal sin taxes. \textit{Journal of Public Economics}. Elsevier. vol. 90. no. 10-11. pp. 1825--1849. 2006.
\bibitem{GruberKoszegi} Gruber, Jonathan; K{\"o}szegi, Botond. Is addiction ``rational''? Theory and evidence. \textit{The Quarterly Journal of Economics}. MIT Press. vol. 116. no. 4. pp. 1261--1303. 2001.
\bibitem{wikiBiases} Wikipedia. List of cognitive biases. 2019.
\bibitem{Sydnor} Sydnor, Justin. (Over) insuring modest risks. \textit{American Economic Journal: Applied Economics}. vol. 2. no. 4. pp. 177--99. 2010.
\bibitem{AtanasovBaker} Atanasov, Pavel; Baker, Tom. Putting health back into health insurance choice. \textit{Medical Care Research and Review}. Sage Publications Sage CA: Los Angeles, CA. vol. 71. no. 4. pp. 337--355. 2014.
\bibitem{GrubbOsborne} Grubb, Michael D; Osborne, Matthew. Cellular service demand: Biased beliefs, learning, and bill shock. \textit{American Economic Review}. vol. 105. no. 1. pp. 234--71. 2015.
\bibitem{AbaluckGruber} Abaluck, Jason; Gruber, Jonathan. Choice inconsistencies among the elderly: evidence from plan choice in the Medicare Part D program. \textit{American Economic Review}. vol. 101. no. 4. pp. 1180--1210. 2011.
\bibitem{Klingetal.} Kling, Jeffrey R; Mullainathan, Sendhil; Shafir, Eldar; Vermeulen, Lee C; Wrobel, Marian V. Comparison friction: Experimental evidence from Medicare drug plans. \textit{The Quarterly Journal of Economics}. MIT Press. vol. 127. no. 1. pp. 199--235. 2012.
\bibitem{AllcottMullainathanTaubinsky} Allcott, Hunt; Mullainathan, Sendhil; Taubinsky, Dmitry. Energy policy with externalities and internalities. \textit{Journal of Public Economics}. Elsevier. vol. 112. pp. 72--88. 2014.
\bibitem{AllcottSunstein} Allcott, Hunt; Sunstein, Cass R. Regulating internalities. \textit{NBER working paper}. vol. 21187. 2015.
\bibitem{Arrondel} Arrondel, Luc; Calvo-Pardoz, Hector; Giannitsarou, Chryssi; Haliassos, Michael. Informative Social Interactions. \textit{Working Paper}. 2018.
\bibitem{Mugermanetal} Mugerman, Yevgeny; Sade, Orly; Shayo, Moses. Long term savings decisions: Financial reform, peer effects and ethnicity. \textit{Journal of Economic Behavior \& Organization}. vol. 106. pp. 235--253. 2014.
\bibitem{Frydman} Frydman, Cary. Relative Wealth Concerns in Portfolio Choice: Neural and Behavioral Evidence. \textit{Working Paper}. 2015.
\bibitem{Cohen1982} Cohen, Peter A.; Kulik, James A.; Kulik, Chen-Lin. Educational Outcomes of Tutoring: A Meta-analysis of Findings. \textit{American Educational Research Journal}. vol. 19. no. 2. pp. 237-248. 1982.
\bibitem{IyengarShotter} Iyengar, Raghuram; Shotter, Andrew. Learning under supervision: an experimental study. \textit{Experimental Economics}. vol. 11. pp. 154--173. 2008.
\bibitem{Shotter} Shotter, Andrew. Decision Making with Naive Advice. \textit{American Economic Review: Paper and Proceedings}. vol. 93. no. 2. pp. 196-201. 2003.
\bibitem{Bhargava.MetalLabelsACAplan} Bhargava, Saurabh; Loewenstein, George; Benartzi, Shlomo. The costs of poor health (plan choices) \& prescriptions for reform. \textit{Behavioral Science \& Policy}. Brookings Institution Press. vol. 3. no. 1. pp. 1--12. 2017.
\bibitem{Bhargavaetal.DominatedHealthcarePlans} Bhargava, Saurabh; Loewenstein, George; Sydnor, Justin. Choose to lose: Health plan choices from a menu with dominated option. \textit{The Quarterly Journal of Economics}. Oxford University Press. vol. 132. no. 3. pp. 1319--1372. 2017.
\bibitem{GigerenzerHoffrage} Gigerenzer, Gerd; Hoffrage, Ulrich. How to improve Bayesian reasoning without instruction: frequency formats.. \textit{Psychological review}. American Psychological Association. vol. 102. no. 4. pp. 684. 1995.
\bibitem{Johnsonetal.HealthNudges} Johnson, Eric J; Hassin, Ran; Baker, Tom; Bajger, Allison T; Treuer, Galen. Can consumers make affordable care affordable? The value of choice architecture. \textit{PloS one}. Public Library of Science. vol. 8. no. 12. pp. e81521. 2013.
\bibitem{Jinetal.NoNews} Jin, Ginger Zhe; Luca, Michael; Martin, Daniel. Is no news (perceived as) bad news? An experimental investigation of information disclosure. 2015.
\bibitem{ReactionaryTrading} Sandro Ambuehl; B Douglas Bernheim; Tingyan Jia; Constantine Yannelis; Andrew Y Park. Reactionary Trading: An experiment with high net worth individuals. \textit{in progress}. 2019.
\bibitem{BLB} Sandro Ambuehl; B Douglas Bernheim; Fulya Ersoy; Donna Harris. Peer Advice on Financial Decisions: A case of the blind leading the blind?. \textit{NBER Working Paper}. vol. w25034. 2022.
\bibitem{portfolioProject} Sandro Ambuehl; B. Douglas Bernheim; Fulya Ersoy; Annamaria Lusardi; David Zuckerman. Does financial education improve portfolio choices or merely change them?. \textit{in progress}. 2019.
\bibitem{Loewensteinetal.HealthInsurance} Loewenstein, George; Friedman, Joelle Y; McGill, Barbara; Ahmad, Sarah; Linck, Suzanne; Sinkula, Stacey; Beshears, John; Choi, James J; Kolstad, Jonathan; Laibson, David; others. Consumers' misunderstanding of health insurance. \textit{Journal of Health Economics}. Elsevier. vol. 32. no. 5. pp. 850--862. 2013.
\bibitem{Cuevasetal.} Cuevas, Conrado; Bernhardt, Dan; Sanclemente, Mario; Sanclemente, Mario. The Pied Piper of Pensioners. \textit{Unpublished Manuscript}. 2018.
\bibitem{Malkiel.RandomWalk} Malkiel, Burton Gordon; McCue, Kerin. A random walk down Wall Street. Norton New York. 1985.
\bibitem{Andreonietal.Money} Andreoni, James; Gravert, Christina; Kuhn, Michael A; Saccardo, Silvia; Yang, Yang. Arbitrage Or Narrow Bracketing? On Using Money to Measure Intertemporal Preferences. \textit{NBER working paper}. 2018.
\bibitem{EricsonLaibson} Ericson, Keith Marzilli; Laibson, David. Intertemporal Choice. \textit{NBER working paper}. 2018.
\bibitem{YakoboskiDickemper} Yakoboski, Paul; Dickemper, Jennifer. Increased saving but little planning: Results of the 1997 Retirement Confidence Survey. \textit{EBRI Issue Brief}. vol. 191. pp. 1--21. 1997.
\bibitem{MadrianShea} Madrian, Brigitte C; Shea, Dennis F. The power of suggestion: Inertia in 401 (k) participation and savings behavior. \textit{The Quarterly Journal of Economics}. MIT Press. vol. 116. no. 4. pp. 1149--1187. 2001.
\bibitem{Choietal.SP500} Choi, James J; Laibson, David; Madrian, Brigitte C. Why does the law of one price fail? An experiment on index mutual funds. \textit{The Review of Financial Studies}. Oxford University Press. vol. 23. no. 4. pp. 1405--1432. 2009.
\bibitem{BenartziThaler.RetirementSavings} Benartzi, Shlomo; Thaler, Richard H. Risk aversion or myopia? Choices in repeated gambles and retirement investments. \textit{Management science}. INFORMS. vol. 45. no. 3. pp. 364--381. 1999.
\bibitem{SkimmyhornBootcamp} Skimmyhorn, William. Assessing financial education: Evidence from boot camp. \textit{American Economic Journal: Economic Policy}. vol. 8. no. 2. pp. 322--43. 2016.
\bibitem{Urbanetal.} Urban, Carly; Schmeiser, Maximilian; Collins, J Michael; Brown, Alexandra. The effects of high school personal financial education policies on financial behavior. \textit{Economics of Education Review}. Elsevier. 2018.
\bibitem{AllcottTaubinsky.Lightbulbs} Allcott, Hunt; Taubinsky, Dmitry. Evaluating behaviorally motivated policy: Experimental evidence from the lightbulb market. \textit{American Economic Review}. vol. 105. no. 8. pp. 2501--38. 2015.
\bibitem{ChettyAnnualReviews} Chetty, Raj. Sufficient statistics for welfare analysis: A bridge between structural and reduced-form methods. \textit{Annu. Rev. Econ.}. Annual Reviews. vol. 1. no. 1. pp. 451--488. 2009.
\bibitem{Kahnemanetal.BackToBentham} Kahneman, Daniel; Wakker, Peter P; Sarin, Rakesh. Back to Bentham? Explorations of experienced utility. \textit{The quarterly journal of economics}. MIT Press. vol. 112. no. 2. pp. 375--406. 1997.
\bibitem{GabaixLaibsonShrouded} Gabaix, Xavier; Laibson, David. Shrouded attributes, consumer myopia, and information suppression in competitive markets. \textit{The Quarterly Journal of Economics}. MIT Press. vol. 121. no. 2. pp. 505--540. 2006.
\bibitem{oDonoghueRabinNowOrLater} O'Donoghue, Ted; Rabin, Matthew. Doing it now or later. \textit{American Economic Review}. vol. 89. no. 1. pp. 103--124. 1999.
\bibitem{GulPesendorferTemptation} Gul, Faruk; Pesendorfer, Wolfgang. Temptation and self-control. \textit{Econometrica}. Wiley Online Library. vol. 69. no. 6. pp. 1403--1435. 2001.
\bibitem{AndreoniSprengerUncertaintyEquivalents} Andreoni, James; Sprenger, Charles. Uncertainty equivalents: Testing the limits of the independence axiom. \textit{NBER working paper}. 2011.
\bibitem{HalevyStrotzMeetsAllais} Halevy, Yoram. Strotz meets Allais: Diminishing impatience and the certainty effect. \textit{American Economic Review}. vol. 98. no. 3. pp. 1145--62. 2008.
\bibitem{Ortolevaetal.CertaintyEffect} Cerreia-Vioglio, Simone; Dillenberger, David; Ortoleva, Pietro. Cautious expected utility and the certainty effect. \textit{Econometrica}. Wiley Online Library. vol. 83. no. 2. pp. 693--728. 2015.
\bibitem{Andersenetal.MagnitudeEffect} Andersen, Steffen; Harrison, Glenn W; Lau, Morten I; Rutstr{\"o}m, E Elisabet. Discounting behaviour and the magnitude effect: Evidence from a field experiment in Denmark. \textit{Economica}. Wiley Online Library. vol. 80. no. 320. pp. 670--697. 2013.
\bibitem{Mullainathan.ReducedForm} Mullainathan, Sendhil; Schwartzstein, Joshua; Congdon, William J. A Reduced-Form Approach to Behavioral Public Finance. \textit{Annu. Rev. Econ.}. Annual Reviews. vol. 4. no. 1. pp. 511--540. 2012.
\bibitem{Buhrmeisteretal.} Buhrmester, Michael D; Talaifar, Sanaz; Gosling, Samuel D. An Evaluation of Amazon's Mechanical Turk, Its Rapid Rise, and Its Effective Use. \textit{Perspectives on Psychological Science}. SAGE Publications Sage CA: Los Angeles, CA. vol. 13. no. 2. pp. 149--154. 2018.
\bibitem{Haraetal.TurkEarnings} Hara, Kotaro; Adams, Abigail; Milland, Kristy; Savage, Saiph; Callison-Burch, Chris; Bigham, Jeffrey P. A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. pp. 449. 2018.
\bibitem{RiegerWang} Rieger, Marc Oliver; Wang, Mei. Prospect theory for continuous distributions. \textit{Journal of Risk and Uncertainty}. Springer. vol. 36. no. 1. pp. 83--102. 2008.
\bibitem{MeasureTheory} Vladimir I. Bogachev. Measure Theory. Springer, Berlin. 2007.
\bibitem{MatejkaMcKay} Mat{\v{e}}jka, Filip; McKay, Alisdair. Rational inattention to discrete choices: A new foundation for the multinomial logit model. \textit{American Economic Review}. vol. 105. no. 1. pp. 272--98. 2015.
\bibitem{RubinsteinSalant} Salant, Yuval; Rubinstein, Ariel. (A, f): choice with frames. \textit{The Review of Economic Studies}. Wiley-Blackwell. vol. 75. no. 4. pp. 1287--1296. 2008.
\bibitem{Edmondson} Edmondson, Amy. Psychological safety and learning behavior in work teams. \textit{Administrative Science Quarterly}. Sage Publications. vol. 44. no. 2. pp. 350--383. 1999.
\bibitem{UKcurriculum} Department for Education. The national curriculum in England. Key stages 3 and 4 framework document. 2014.
\bibitem{UKfinEdPolicy} House of Commons Library. Financial and enterprise education in schools. 2016.
\bibitem{KoPirinsky} Ko, K Jeremy; Pirinsky, Christo. Did Social Interactions Fuel or Suppress the US Housing Bubble?. \textit{Unpublished Manuscript}. 2017.
\bibitem{HvideOstberg} Hvide, Hans K; {\"O}stberg, Per. Social interaction at work. \textit{Journal of Financial Economics}. Elsevier. vol. 117. no. 3. pp. 628--652. 2015.
\bibitem{Lyle} Lyle, David S. Estimating and interpreting peer and role model effects from randomly assigned social groups at West Point. \textit{The Review of Economics and Statistics}. MIT Press. vol. 89. no. 2. pp. 289--299. 2007.
\bibitem{Carrelletal.Military} Carrell, Scott E; Sacerdote, Bruce I; West, James E. From natural variation to optimal policy? The importance of endogenous peer group formation. \textit{Econometrica}. Wiley Online Library. vol. 81. no. 3. pp. 855--882. 2013.
\bibitem{Coller} Coller, Maribeth; Williams, Melonie B.. Eliciting Individual Discount Rates. \textit{Experimental Economics}. vol. 2. pp. 107-127. 1999.
\bibitem{Harrison} Harrison, Glenn W; Lau, Morten Igel; Williams, Melonie B.. Estimating Individual Discount Rates in Denmark: A Field Experiment. \textit{American Economic Review}. vol. 92. no. 5. pp. 1606-1617. 2002.
\bibitem{Andersen2014} Andersen, Steffen; Harrison, Glenn W; Lau, Morten Igel; Rutstr{\"o}m, E Elisabet. Discounting behavior:A reconsideration. \textit{European Economic Review}. vol. 71. pp. 15-33. 2014.
\bibitem{Bruhnetal.} Bruhn, Miriam; Le{\~a}o, Luciana de Souza; Legovini, Arianna; Marchetti, Rogelio; Zia, Bilal. The impact of high school financial education: Evidence from a large-scale evaluation in Brazil. \textit{American Economic Journal: Applied Economics}. vol. 8. no. 4. pp. 256--95. 2016.
\bibitem{Shue.ExecutiveNetworks} Shue, Kelly. Executive networks and firm policies: Evidence from the random assignment of MBA peers. \textit{The Review of Financial Studies}. Oxford University Press. vol. 26. no. 6. pp. 1401--1442. 2013.
\bibitem{MalmendierLerner} Lerner, Josh; Malmendier, Ulrike. With a little help from my (random) friends: Success and failure in post-business school entrepreneurship. \textit{The Review of Financial Studies}. Society for Financial Studies. vol. 26. no. 10. pp. 2411--2452. 2013.
\bibitem{CohenFrazziniMalloy} Cohen, Lauren; Frazzini, Andrea; Malloy, Christopher. The small world of investing: Board connections and mutual fund returns. \textit{Journal of Political Economy}. The University of Chicago Press. vol. 116. no. 5. pp. 951--979. 2008.
\bibitem{OuimetTate} Ouimet, Paige; Tate, Geoffrey. Learning from Coworkers: Peer Effects on Individual Investment Decisions. \textit{NBER working paper}. vol. 24085. 2017.
\bibitem{Kimbroughetal.PeerTeaching} Kimbrough, Erik O; McGee, Andrew D; Shigeoka, Hitoshi. How Do Peers Impact Learning? An Experimental Investigation of Peer-to-Peer Teaching and Ability Tracking. \textit{NBER working paper}. vol. 23439. 2017.
\bibitem{McCartneyShah} W. Benedict McCartney; Avni Shah. `I'll Have What She's Having': Identifying' Social Influence in Household Mortgage Decisions. \textit{Unpublished Manuscript}. 2017.
\bibitem{AndreoniSprengerRisk} Andreoni, James; Sprenger, Charles. Risk preferences are not time preferences. \textit{The American Economic Review}. American Economic Association. vol. 102. no. 7. pp. 3357--3376. 2012.
\bibitem{PT} Kahneman, Daniel; Tversky, Amos. Prospect theory: An analysis of decision under risk. \textit{Econometrica: Journal of the econometric society}. JSTOR. pp. 263--291. 1979.
\bibitem{CPT} Tversky, Amos; Kahneman, Daniel. Advances in prospect theory: Cumulative representation of uncertainty. \textit{Journal of Risk and Uncertainty}. Springer. vol. 5. no. 4. pp. 297--323. 1992.
\bibitem{HarrisonRutstrom} Glenn W. Harrison; E. Elisabet Rutstr{\"o}m. Risk Aversion in the Laboratory. Emerald Group Publishing Limited. pp. 41-196. 2008.
\bibitem{Noussairetal} Charles N. Noussair; Stefan T. Trautmann; Gijs van de Kuilen. Higher Order Risk Attitudes, Demographics, and Financial Decisions. \textit{The Review of Economic Studies}. vol. 81. no. 1. pp. 325-55. 2014.
\bibitem{Metcalfe} Janet Metcalfe. Evolution of Metacognition. \textit{Handbook of Metamemory and Memory}. pp. 29-46. 2008.
\bibitem{KondorKoszegi} P{\'e}ter Kondor; Botond K{\"o}szegi. Cursed Financial Innovation. \textit{WZB Discussion Paper}. vol. SP II 2015--306. 2015.
\bibitem{FinLitAroundTheWorld} Leora Klapper; Annamaria Lusardi; Peter van Oudheusden. Financial Literacy around the World: Insights from the Standard \& Poor's Rating Services Global Financial Literacy Survey. 2015.
\bibitem{EysterWeizsacker} Erik Eyster; Georg Weizs{\"a}cker. Correlation Neglect in Portfolio Choice: Lab Evidence. \textit{Unpublished Manuscript}. 2016.
\bibitem{CharnessGneezy} Gary Charness; Uri Gneezy. Portfolio Choice and Risk Attitudes: An Experiment. \textit{Economic Inquiry}. vol. 48. no. 1. pp. 133-46. 2010.
\bibitem{Beshearsetal.AggReturnsDisc} John Beshears; James J. Choi; David Laibson; Brigitte C. Madrian. Does Aggregated Returns Disclosure Increase Portfolio Risk Taking?. \textit{The Review of Financial Studies}. vol. 30. no. 6. pp. 1971-2005. 2017.
\bibitem{BeauchampCesariniJohannesson} Beauchamp, Jonathan; Cesarini, David; Johannesson, Magnus. The Psychometric and Empirical Properties of Measures of Risk Preferences. \textit{Unpublished Manuscript}. 2015.
\bibitem{Beauchampetal} Jonathan P. Beauchamp; Daniel J. Benjamin; Christopher F. Chabris; David I. Laibson. Controlling for the Compromise Effect Debiases Estimates of Risk Preference Parameters. \textit{NBER Working Paper}. vol. 21792. 2016.
\bibitem{Ambrus} Ambrus, Attila; Greiner, Ben; Pathak, Parag. Group versus Individual Decision-Making: Is There a Shift?. \textit{Working Paper}. 2009.
\bibitem{Battaglini} Battaglini, Marco; B{\'e}nabou, Roland; Tirole, Jean. Self-Control in Peer Groups. \textit{Journal of Economic Theory}. vol. 123. pp. 105-34. 2005.
\bibitem{Michaeli} Michaeli, Moti; Spiro, Daniel. From Peer Pressure to Biased Norms. \textit{American Economic Journal: Microeconomics}. vol. 9. no. 1. pp. 152-216. 2017.
\bibitem{BernheimExley} Bernheim, B. Douglas; Exley, Christine L.. Understanding Conformity: An Experimental Investigation. \textit{HBS Working Paper 16-070}. 2015.
\bibitem{BrownIvkovic} Brown, Jeffrey R.; Ivkovi{\'c}, Zoran; Smith, Paul A.; Weisbenner, Scott. Neighbors Matter: Causal Community Effects and Stock Market Participation. \textit{The Journal Of Finance}. vol. 63. no. 3. pp. 1509-531. 2008.
\bibitem{Miller} Miller, Margaret; Reichelstein, Julia; Salas, Christian; Zia, Bilal. Can You Help Someone Become Financially Capable? A Meta-Analysis of the Literature. \textit{World Bank Policy Research Working Paper}. vol. 6745. 2014.
\bibitem{ExpensiveIndexFunds} David Randall. Analysis: High-Priced Index Funds? The Worst Deal for Investors. \textit{Reuters}. 2014.
\bibitem{KrugerDunning} Kruger, Justin; Dunning, David. Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments. \textit{Journal of Personality and Social Psychology}. American Psychological Association. vol. 77. no. 6. pp. 1121-134. 1999.
\bibitem{LieberSkimmyhorn} Lieber, Ethan MJ; Skimmyhorn, William. Peer Effects in Financial Decision-Making: Social Spending but Private Saving. \textit{Unpublished Manuscript}. 2017.
\bibitem{Frechette.ProfessionalsVsStudents} Fr{\'e}chette, Guillaume R. Handbook of Experimental Economic Methodology. Oxford University Press. 2015.
\bibitem{Sacerdote.AnnualReviews} Sacerdote, Bruce. Experimental and Quasi-Experimental Analysis of Peer Effects: Two Steps Forward?. \textit{Annual Review of Economics}. Annual Reviews. vol. 6. no. 1. pp. 253-72. 2014.
\bibitem{Manski.Reflection} Manski, Charles F. Identification of Endogenous Social Effects: The Reflection Problem. \textit{The Review of Economic Studies}. Oxford University Press. vol. 60. no. 3. pp. 531-42. 1993.
\bibitem{Cohenetal.MELreview} Cohen, Jonathan D; Ericson, Keith Marzilli; Laibson, David; White, John Myles. Measuring Time Preferences. \textit{NBER working paper}. vol. 22455. 2016.
\bibitem{HoxbyWeingarth} Hoxby, Caroline M; Weingarth, Gretchen. Taking Race Out of the Equation: School Reassignment and the Structure of Peer Effects. \textit{Unpublished Manuscript}. 2005.
\bibitem{Dufloetal.PeerEffects} Duflo, Esther; Dupas, Pascaline; Kremer, Michael. Peer Effects, Teacher Incentives, and the Impact of Tracking: Evidence from a Randomized Evaluation in Kenya. \textit{American Economic Review}. American Economic Association. vol. 101. no. 5. pp. 1739-774. 2011.
\bibitem{Greenetal.Mediation} Green, Donald P; Ha, Shang E; Bullock, John G. Enough Already about ``Black Box'' Experiments: Studying Mediation Is More Difficult than Most Scholars Suppose. \textit{The Annals of the American Academy of Political and Social Science}. Sage Publications. vol. 628. no. 1. pp. 200-08. 2010.
\bibitem{Haliassosetal.} Haliassos, Michael; Jansson, Thomas; Karabulut, Yigitcan. Financial Literacy Externalities. \textit{Sveriges Riksbank Working Paper Series}. vol. 333. 2017.
\bibitem{Georgarakosetal.} Georgarakos, Dimitris; Haliassos, Michael; Pasini, Giacomo. Household Debt and Social Interactions. \textit{The Review of Financial Studies}. Soc Financial Studies. vol. 27. no. 5. pp. 1404-433. 2014.
\bibitem{HortacsuSyverson} Horta{\c{c}}su, Ali; Syverson, Chad. Product Differentiation, Search Costs, and Competition in the Mutual Fund Industry: A Case Study of S\&P 500 Index Funds. \textit{Quarterly Journal of Economics}. vol. 119. no. 2. pp. 403-56. 2004.
\bibitem{Gillenetal.} Ben Gillen; Erik Snowberg; Leeat Yariv. Experimenting with Measurement Error: Techniques with Applications to the {C}altech Cohort Study. \textit{Unpublished Manuscript}. 2016.
\bibitem{BernheimConformity} B. Douglas Bernheim. A Theory of Conformity. \textit{Journal of Political Economy}. vol. 102. no. 5. pp. 841-77. 1994.
\bibitem{Battacharyaetal.UnbiasedAdvise} Utpal Bhattacharya; Andreas Hackethal; Simon Kaesler; Benjamin Loos; Steffen Meyer. Is Unbiased Financial Advice to Retail Investors Sufficient? {A}nswers from a Large Field Study. \textit{The Review of Financial Studies}. vol. 25. no. 4. pp. 975-1032. 2012.
\bibitem{GoereeYariv} Jacob K. Goeree; Leeat Yariv. Conformity in the Lab. \textit{Journal of the Economic Science Association}. vol. 1. pp. 15-28. 2015.
\bibitem{Chenetal.MovieLens} Yan Chen; F. Maxwell Harper; Joseph Konstan; Sherry Xin Li. Social Comparisons and Contributions to Online Communities: A Field Experiment on MovieLens. \textit{American Economic Review}. vol. 100. no. 4. pp. 1358-398. 2010.
\bibitem{JacksonBruegmann} Jackson, C Kirabo; Bruegmann, Elias. Teaching Students and Teaching Each Other: The Importance of Peer Learning for Teachers. \textit{American Economic Journal: Applied Economics}. American Economic Association. vol. 1. no. 4. pp. 85-108. 2009.
\bibitem{CaiChenFang} Cai, Hongbin; Chen, Yuyu; Fang, Hanming. Observational Learning: Evidence from a Randomized Natural Field Experiment. \textit{American Economic Review}. American Economic Association. vol. 99. no. 3. pp. 864-82. 2009.
\bibitem{YanivKleinberger} Yaniv, Ilan; Kleinberger, Eli. Advice Taking in Decision Making: Egocentric Discounting and Reputation Formation. \textit{Organizational Behavior and Human Decision Processes}. vol. 83. no. 2. pp. 260-81. 2000.
\bibitem{Imasetal.WaitingToChoose} Alex Imas; Michael Kuhn; Vera Mironova. Waiting to Choose. \textit{CESifo Working Paper Series}. vol. 6162. 2016.
\bibitem{BlanesIVidalNossol} Blanes i Vidal, Jordi; Nossol, Mareike. Tournaments without Prizes: Evidence from Personnel Records. \textit{Management Science}. INFORMS. vol. 57. no. 10. pp. 1721-736. 2011.
\bibitem{KuhnenTymula} Kuhnen, Camelia M; Tymula, Agnieszka. Feedback, Self-Esteem, and Performance in Organizations. \textit{Management Science}. INFORMS. vol. 58. no. 1. pp. 94-113. 2012.
\bibitem{Halevy.MoneyPrimary} Yoram Halevy. Some Comments on the Use of Monetary and Primary Rewards in the Measurement of Time Preferences. \textit{Unpublished Manuscript}. 2014.
\bibitem{Booijetal.} Booij, Adam S; Leuven, Edwin; Oosterbeek, Hessel. Ability Peer Effects in University: Evidence from a Randomized Experiment. \textit{The Review of Economic Studies}. vol. 84. no. 2. pp. 547-78. 2016.
\bibitem{LahnoSerraGarcia} Lahno, Amrei M; Serra-Garcia, Marta. Peer Effects in Risk Taking: Envy or Conformity?. \textit{Journal of Risk and Uncertainty}. Springer. vol. 50. no. 1. pp. 73-95. 2015.
\bibitem{IkovicWeisbenner} Ivkovi{\'c}, Zoran; Weisbenner, Scott. Information Diffusion Effects in Individual Investors' Common Stock Purchases: Covet Thy Neighbors' Investment Choices. \textit{The Review of Financial Studies}. Soc Financial Studies. vol. 20. no. 4. pp. 1327-357. 2007.
\bibitem{Kastetal.} Kast, Felipe; Meier, Stephan; Pomeranz, Dina. Saving More in Groups: Field Experimental Evidence from {C}hile. \textit{HBS Working Paper 12-060}. 2016.
\bibitem{ConformityQuestionnaire} Mehrabian, Albert; Stefl, Catherine A. Basic Temperament Components of Loneliness, Shyness, and Conformity. \textit{Social Behavior and Personality: An International Journal}. Scientific Journal Publishers. vol. 23. no. 3. pp. 253-63. 1995.
\bibitem{BigFiveShort} Rammstedt, Beatrice; John, Oliver P. Measuring Personality in One Minute or Less: A 10-Item Short Version of the Big Five Inventory in {E}nglish and {G}erman. \textit{Journal of Research in Personality}. Elsevier. vol. 41. no. 1. pp. 203-12. 2007.
\bibitem{Hongetal.2005} Hong, Harrison; Kubik, Jeffrey D; Stein, Jeremy C. Thy Neighbor's Portfolio: Word-of-Mouth Effects in the Holdings and Trades of Money Managers. \textit{The Journal of Finance}. Wiley Online Library. vol. 60. no. 6. pp. 2801-824. 2005.
\bibitem{Hongetal.2004} Hong, Harrison; Kubik, Jeffrey D; Stein, Jeremy C. Social Interaction and Stock-Market Participation. \textit{The Journal of Finance}. Wiley Online Library. vol. 59. no. 1. pp. 137-63. 2004.
\bibitem{MobiusRosenblat} Mobius, Markus; Rosenblat, Tanya. Social Learning in Economics. \textit{Annual Review of Economics}. Annual Reviews. vol. 6. no. 1. pp. 827-47. 2014.
\bibitem{KerrTindale} Kerr, Norbert L; Tindale, R Scott. Group Performance and Decision Making. \textit{Annual Review of Psychology}. Annual Reviews. vol. 55. pp. 623-55. 2004.
\bibitem{Lusardi2008} Lusardi, Annamaria. Household Saving Behavior: The Role of Financial Literacy, Information, and Financial Education. \textit{NBER Working Paper}. vol. 13824. 2008.
\bibitem{Lusardi2003} Lusardi, Annamaria. Planning and Saving for Retirement. \textit{Unpublished Manuscript}. 2003.
\bibitem{BoudreauMcCubbins} Boudreau, Cheryl; McCubbins, Mathew D. The Blind Leading the Blind: Who Gets Polling Information and Does it Improve Decisions?. \textit{The Journal of Politics}. Cambridge Univ Press. vol. 72. no. 2. pp. 513-27. 2010.
\bibitem{Michaelsenetal.} Michaelsen, Larry K; Watson, Warren E; Black, Robert H. A Realistic Test of Individual versus Group Consensus Decision Making. \textit{Journal of Applied Psychology}. American Psychological Association. vol. 74. no. 5. pp. 834-39. 1989.
\bibitem{CharnessSutter} Charness, Gary; Sutter, Matthias. Groups Make Better Self-Interested Decisions. \textit{Journal of Economic Perspectives}. American Economic Association. vol. 26. no. 3. pp. 157-76. 2012.
\bibitem{BernheimInMitchellSchieber} B. Douglas Bernheim. Financial Illiteracy, Education, and Retirement Saving. In Living with Defined Contribution Pensions. Remaking Responsibility for Retirement.. University of Pennsylvania Press. 1998.
\bibitem{Melzeretal.} Linnainmaa, Juhani T; Melzer, Brian T; Previtero, Alessandro. The Misguided Beliefs of Financial Advisors. \textit{Unpublished Manuscript}. 2016.
\bibitem{FeldZolitz} Feld, Jan; Z{\"o}litz, Ulf. Understanding Peer Effects: On the Nature, Estimation and Channels of Peer Effects. \textit{Journal of Labor Economics}. vol. 35. no. 2. pp. 387-428. 2017.
\bibitem{Caietal.Networks} Cai, Jing; De Janvry, Alain; Sadoulet, Elisabeth. Social Networks and the Decision to Insure. \textit{American Economic Journal: Applied Economics}. American Economic Association. vol. 7. no. 2. pp. 81-108. 2015.
\bibitem{Bursztynetal.PeerEffects} Bursztyn, Leonardo; Ederer, Florian; Ferman, Bruno; Yuchtman, Noam. Understanding Mechanisms Underlying Peer Effects: Evidence from a Field Experiment on Financial Decisions. \textit{Econometrica}. Wiley Online Library. vol. 82. no. 4. pp. 1273-301. 2014.
\bibitem{Beshearsetal.PeerInfoSavings} Beshears, John; Choi, James J; Laibson, David; Madrian, Brigitte C; Milkman, Katherine L. The Effect of Providing Peer Information on Retirement Savings Decisions. \textit{The Journal of Finance}. Wiley Online Library. vol. 70. no. 3. pp. 1161-201. 2015.
\bibitem{CRT} Frederick, Shane. Cognitive reflection and decision making. \textit{Journal of Economic Perspectives}. JSTOR. vol. 19. no. 4. pp. 25-42. 2005.
\bibitem{Brownetal.} Brown, Alexandra; Collins, J Michael; Schmeiser, Maximilian; Urban, Carly. State Mandated Financial Education and the Credit Behavior of Young Adults. \textit{Divisions of Research \& Statistics and Monetary Affairs Federal Reserve Board, Washington, D.C., Finance and Economics Discussion Series}. vol. 2014-68. 2014.
\bibitem{enkeZimmermann} Enke, Benjamin; Zimmermann, Florian. Correlation neglect in belief formation. \textit{The Review of Economic Studies}. Oxford University Press. vol. 86. no. 1. pp. 313--332. 2019.
\bibitem{Kunda} Kunda, Ziva. The Case for Motivated Reasoning. \textit{Psychological bulletin}. American Psychological Association. vol. 108. no. 3. pp. 480-98. 1990.
\bibitem{Aufenageretal.} Tobias Aufenanger; Friedemann Richter; Matthias Wrede. Measuring Decision-Making Ability in the Evaluation of Financial Literacy Education Programs. \textit{Unpublished Manuscript}. 2016.
\bibitem{oDonoghueRabin} Ted O'Donoghue; Matthew Rabin. Doing It Now or Later. \textit{American Economic Review}. vol. 89. no. 1. pp. 103-24. 1999.
\bibitem{CaplinMartin} Caplin, Andrew; Martin, Daniel. A Testable Theory of Imperfect Perception. \textit{The Economic Journal}. Wiley Online Library. vol. 125. pp. 184-202. 2015.
\bibitem{CaplinDean.AER15} Andrew Caplin; Mark Dean. Revealed Preference, Rational Inattention, and Costly Information Acquisition. \textit{American Economic Review}. vol. 105. no. 7. pp. 2183-203. 2015.
\bibitem{AonHewitt2013} Rob Austin; Winfield Evens. 2013 Trends \& Experience in Defined Contribution Plans. \textit{Aon Hewitt}. 2013.
\bibitem{CoffmanNiehaus} Lucas Coffman; Paul Niehaus. Pathways of Persuasion. \textit{Unpublished Manuscript}. 2016.
\bibitem{AmbuehlBernheimLusardi} Sandro Ambuehl; B. Douglas Bernheim; Annamaria Lusardi. Evaluating Financial Competence. \textit{NBER Working Paper}. vol. 20618. 2018.
\bibitem{Harrisonetal.} Harrison, Glenn W; Lau, Morten Igel; Rutstr{\"o}m, E Elisabet; Sullivan, Melonie B. Eliciting Risk and Time Preferences Using Field Experiments: Some Methodological Issues. \textit{Field Experiments in Economics}. Greenwich, CT: JAI Press, Research in Experimental Economics. vol. 10. pp. 125-218. 2005.
\bibitem{Sprenger.EndowmentEffect} Sprenger, Charles. An Endowment Effect for Risk: Experimental Tests of Stochastic Reference Points. \textit{Journal of Political Economy}. University of Chicago Press. vol. 123. no. 6. pp. 1456-499. 2015.
\bibitem{Theodosetal.} Theodos, Brett; Simms, Margaret; Treskon, Mark; Stacy, Christina; Brash, Rachel; Emam, Dina; Daniels, Rebecca; Collazos, Juan. An Evaluation of the Impacts and Implementation Approaches of Financial Coaching Programs. \textit{Urban Institute}. 2015.
\bibitem{BernheimWelfareReview2} B. Douglas Bernheim. The Good, the Bad, and the Ugly: A Unified Approach to Behavioral Welfare Economics. \textit{Journal of Benefit-Cost Analysis}. vol. 7. no. 1. pp. 12-68. 2016.
\bibitem{GodaManchesterSojourner} Goda, Gopi Shah; Manchester, Colleen Flaherty; Sojourner, Aaron J. What Will My Account Really Be Worth? Experimental Evidence on How Retirement Income Projections Affect Saving. \textit{Journal of Public Economics}. Elsevier. vol. 119. pp. 80-92. 2014.
\bibitem{Godaetal.EGB} Goda, Gopi Shah; Levy, Matthew; Manchester, Colleen Flaherty; Sojourner, Aaron; Tasoff, Joshua. Predicting retirement savings using survey measures of exponential-growth bias and present bias. \textit{Economic Inquiry}. Wiley Online Library. vol. 57. no. 3. pp. 1636--1658. 2019.
\bibitem{BinswangerCarman} Binswanger, Johannes; Carman, Katherine Grace. How Real People Make Long-Term Decisions: The Case of Retirement Preparation. \textit{Journal of Economic Behavior \&amp; Organization}. Elsevier. vol. 81. no. 1. pp. 39-60. 2012.
\bibitem{Weberetal.CommunicatingRisk} Weber, Elke U; Siebenmorgen, Niklas; Weber, Martin. Communicating Asset Risk: How Name Recognition and the Format of Historic Volatility Information Affect Risk Perception and Investment Decisions. \textit{Risk Analysis}. Wiley Online Library. vol. 25. no. 3. pp. 597-609. 2005.
\bibitem{AnagolGamble} Anagol, Santosh; Gamble, Keith Jacks. Does Presenting Investment Results Asset by Asset Lower Risk Taking?. \textit{Journal of Behavioral Finance}. vol. 14. no. 4. pp. 276-300. 2013.
\bibitem{Batemanetal.RiskPresentation} Bateman, Hazel; Eckert, Christine; Geweke, John; Louviere, Jordan; Satchell, Stephen; Thorp, Susan. Risk Presentation and Portfolio Choice. \textit{Review of Finance}. Oxford University Press. pp. 201-29. 2016.
\bibitem{vonGaudecker} Hans-Martin} {von Gaudecker. How Does Household Portfolio Diversification Vary with Financial Literacy and Financial Advice?. \textit{The Journal of Finance}. Wiley Online Library. vol. 70. no. 2. pp. 489-507. 2015.
\bibitem{Eganetal.Misconduct} Egan, Mark; Matvos, Gregor; Seru, Amit. The Market for Financial Adviser Misconduct. \textit{NBER working paper}. vol. 22050. 2016.
\bibitem{MooreHealy} Moore, Don A; Healy, Paul J. The Trouble with Overconfidence. \textit{Psychological Review}. American Psychological Association. vol. 115. no. 2. pp. 502-17. 2008.
\bibitem{GrubbOverconfidence} Grubb, Michael D. Overconfident Consumers in the Marketplace. \textit{Journal of Economic Perspectives}. JSTOR. vol. 29. no. 4. pp. 9-35. 2015.
\bibitem{PuriRobinson} Puri, Manju; Robinson, David T. Optimism and Economic Choice. \textit{Journal of Financial Economics}. Elsevier. vol. 86. no. 1. pp. 71-99. 2007.
\bibitem{Grahametal.} Graham, John R; Harvey, Campbell R; Huang, Hai. Investor Competence, Trading Frequency, and Home Bias. \textit{Management Science}. INFORMS. vol. 55. no. 7. pp. 1094-106. 2009.
\bibitem{Hadaretal.} Hadar, Liat; Sood, Sanjay; Fox, Craig R. Subjective Knowledge in Consumer Financial Decisions. \textit{Journal of Marketing Research}. American Marketing Association. vol. 50. no. 3. pp. 303-16. 2013.
\bibitem{LevyTasoffMeta} Matthew R. Levy; Joshua Tasoff. Exponential-Growth Bias and Overconfidence. \textit{Journal of Economic Psychology}. vol. 58. pp. 1-14. 2017.
\bibitem{LusardiMichaudMitchell} Lusardi, Annamaria; Michaud, Pierre-Carl; Mitchell, Olivia S. Optimal Financial Knowledge and Wealth Inequality. \textit{NBER Working Paper}. vol. 18669. 2013.
\bibitem{Fernandesetal.Meta} Fernandes, Daniel; Lynch Jr., John G; Netemeyer, Richard G. Financial Literacy, Financial Education, and Downstream Financial Behaviors. \textit{Management Science}. INFORMS. vol. 60. no. 8. pp. 1861-883. 2014.
\bibitem{Skimmyhorn2015} William Skimmyhorn. Assessing Financial Education: Promising Evidence From Boot Camp. \textit{American Economic Journal: Economic Policy}. vol. 8. no. 2. pp. 322-43. 2016.
\bibitem{BarnettCeci} Barnett, Susan M; Ceci, Stephen J. When and Where Do We Apply What We Learn? A Taxonomy for Far Transfer. \textit{Psychological Bulletin}. American Psychological Association. vol. 128. no. 4. pp. 612-37. 2002.
\bibitem{Tverskyetal.PrefReversals} Amos Tversky; Paul Slovic; Daniel Kahneman. The Causes of Preference Reversal. \textit{American Economic Review}. vol. 80. no. 1. pp. 204-17. 1990.
\bibitem{GretherPlott} David M. Grether; Charles R. Plott. Economic Theory of Choice and the Preference Reversal Phenomenon. \textit{American Economic Review}. vol. 69. no. 4. pp. 623-38. 1979.
\bibitem{Bordaloetal.} Pedro Bordalo; Nicola Gennaioli; Andrei Shleifer. Salience Theory of Choice under Risk. \textit{Quarterly Journal of Economics}. pp. 1243-285. 2012.
\bibitem{Smith} Vernon L. Smith. Experimental Economics: Induced Value Theory. \textit{American Economic Review}. vol. 66. no. 2. pp. 274-79. 1976.
\bibitem{Kapoor} Jack Kapoor; {Les} Dlabay; Robert J. Hughes. Personal Finance. McGraw-Hill / Irwin. 2014.
\bibitem{Madura} Jeff Madura. Personal Finance. Prentice Hall. 2013.
\bibitem{Keown} Arthur J. Keown. Personal Finance: Turning Money Into Wealth. Prentice Hall. 2012.
\bibitem{PadoaSchioppaAssad} Camillo Padoa-Schioppa; John A. Assad. Neurons in the Orbitofrontal Cortex Encode Economic Value. \textit{Nature}. vol. 441. pp. 223-26. 2006.
\bibitem{Wallisetal.} Jonathan D. Wallis. Orbitofrontal Cortex and Its Contribution to Decision-Making. \textit{Annual Review of Neuroscience}. vol. 30. pp. 31-56. 2007.
\bibitem{Rangeletal.} Antonio Rangel; Colin Camerer; P. Read Montague. A Framework for Studying the Neurobiology of Value-Based Decision Making. \textit{Nature Reviews Neuroscience}. vol. 9. pp. 545-56. 2008.
\bibitem{Krajbichetal.1} Ian Krajbich; Carrie Armel; Antonio Rangel. Visual Fixations and the Computation and Comparison of Value in Simple Choice. \textit{Nature Neuroscience}. vol. 13. no. 10. pp. 1292-298. 2010.
\bibitem{ArrowWARP} Kenneth J. Arrow. Rational Choice Functions and Orderings. \textit{Economica}. vol. 26. no. 102. pp. 121-27. 1959.
\bibitem{Peysakhovichetal.} Alexander Peysakhovich; Martin A. Nowak; David G. Rand. Humans Display a 'Cooperative Phenotype' That Is Domain General and Temporally Stable. \textit{Nature Communications}. vol. 5. 2014.
\bibitem{Hortonetal.} John J. Horton; David G. Rand; Richard J. Zeckhauser. The Online Laboratory: Conducting Experiments in a Real Labor Market. \textit{Experimental Economics}. vol. 14. pp. 399-425. 2011.
\bibitem{Fredericketal.Review} Shane Frederick; George Loewenstein; Ted O'Donoghue. Time Discounting and Time Preference: A Critical Review. \textit{Journal of Economic Literature}. vol. 40. no. 2. pp. 351-401. 2002.
\bibitem{LusardiMitchellOUP} Annamaria Lusardi; Olivia Mitchell. Financial Literacy and Planning: Implications for Retirement Well-being. In Financial Literacy. Implications for Retirement Security and the Financial Marketplace. Oxford University Press. pp. 17-39. 2011.
\bibitem{BertrandMorse} Marianne Bertrand; Adair Morse. Information Disclosure, Cognitive Biases, and Payday Borrowing. \textit{The Journal of Finance}. vol. 66. no. 6. pp. 1865-993. 2011.
\bibitem{Augenblicketal.} Ned Augenblick; Muriel Niederle; Charles Sprenger. Working over Time: Dynamic Inconsistency in Real Effort Tasks. \textit{Unpublished Manuscript}. 2014.
\bibitem{MandellKlein} Lewis Mandell; Linda Schmid Klein. The Impact of Financial Literacy Education on Subsequent Financial Behavior. \textit{Journal of Financial Counseling and Planning}. vol. 20. no. 1. pp. 15-24. 2009.
\bibitem{CalvetCampbellSodiniAER} Laurent E. Calvet; John Y. Campbell; Paolo Sodini. Measuring the Financial Sophistication of Households. \textit{American Economic Review}. vol. 99. no. 2. pp. 393-98. 2009.
\bibitem{ErnstFarrisKing} Keith Ernst; John Farris; Uriah King. Quantifying the Economic Cost of Predatory Payday Lending. 2004.
\bibitem{Carlinetal.} Bruce I. Carlin; Li Jiang; Stephen A. Spiller. Learning Millennial-Style. \textit{NBER Working Paper}. vol. 20268. 2014.
\bibitem{Collins} J.M. Collins. The Impacts of Mandatory Financial Education: Evidence from a Randomized Field Study. \textit{Journal of Economic Behavior \& Organization}. vol. 95. pp. 146-58. 2013.
\bibitem{ServonKaestner} Lisa J. Servon; Robert Kaestner. Consumer Financial Literacy and the Impact of Online Banking on the Financial Behavior of Lower-Income Bank Customers. \textit{Journal of Consumer Affairs}. vol. 42. no. 2. pp. 271-305. 2008.
\bibitem{GartnerTodd} Kimberly Gartner; Richard M. Todd. Effectiveness of Online Early Intervention Financial Education Programs for Credit-Card Holders. \textit{Working Paper, Federal Reserve Bank Chicago}. 2005.
\bibitem{Skimmyhorn} William Skimmyhorn. Essays in Behavioral Household Finance. 2012.
\bibitem{ColeSampsonZia} Shawn Cole; Thomas Sampson; Bilal Zia. Prices or Knowledge? What Drives Demand for Financial Services in Emerging Markets?. \textit{The Journal of Finance}. vol. 66. no. 6. pp. 1933-967. 2011.
\bibitem{Mandell} Lewis Mandell. The Financial Literacy of Young American Adults: Results of the 2008 National Jump\$tart Coalition Survey of High School Seniors and College Students. \textit{Jump\$tart Coalition}. vol. Washington, D.C.. 2009.
\bibitem{JumpStart} Jump\$tart Coalition for Personal Financial Literacy. Financial Literacy Shows Slight Improvement among Nation's High School Students. \textit{Press Release}. vol. Washington, D.C.. 2006.
\bibitem{BernheimWelfareReview} B. Douglas Bernheim. Behavioral Welfare Economics. \textit{Journal of the European Economic Association}. vol. 7. no. 2-3. pp. 267-319. 2009.
\bibitem{WagenaarSagaria} William M. Wagenaar; Sabato D. Sagaria. Misperception of Exponential Growth. \textit{Perception and Psychophysics}. vol. 18. no. 6. pp. 416-22. 1975.
\bibitem{Afriat72} Sidney N. Afriat. Efficiency Estimation of Production Functions. \textit{International Economic Review}. vol. 13. no. 3. pp. 568-98. 1972.
\bibitem{HastingsTejedaAshton} Justine S. Hastings; Lydia Tejeda-Ashton. Financial Literacy, Information, and Demand Elasticity: Survey and Experimental Evidence from Mexico. \textit{NBER Working Paper}. vol. 14538. 2008.
\bibitem{Drexleretal.} Alejandro Drexler; Greg Fischer; Antoinette Schoar. Keeping It Simple: Financial Literacy and Rules of Thumb. \textit{American Economic Journal: Applied Economics}. vol. 6. no. 2. pp. 1-31. 2014.
\bibitem{Agarwaletal.} Sumit Agarwal; John C. Driscoll; Xavier Gabaix; David Laibson. The Age of Reason: Financial Decisions over the Life Cycle and Implications for Regulation. \textit{Brookings Papers on Economic Activity}. vol. Fall. pp. 51-101. 2009.
\bibitem{Choietal.100Bills} James J. Choi; David Laibson; Brigitte C. Madrian. \$100 Bills on the Sidewalk: Suboptimal Investment in 401(k) Plans. \textit{Review of Economics and Statistics}. vol. 93. no. 3. pp. 748-63. 2011.
\bibitem{WillisAgainstFinEd} Lauren E. Willis. Against Financial Literacy Education. \textit{University of Pennsylvania Law School Public Law and Legal Theory Research Paper}. no. 8-10. 2010.
\bibitem{IyengarKamenica} Sheena S. Iyengar,; Emir Kamenica. Choice Proliferation, Simplicity Seeking, and Asset Allocation. \textit{Journal of Public Economics}. vol. 94. pp. 530-39. 2010.
\bibitem{Carpenaetal.} Fenella Carpena; Shawn Cole; Jeremy Shapiro; Bilal Zia. Unpacking the Causal Chain of Financial Literacy. \textit{The World Bank Policy Research Working Paper}. vol. 5798. 2011.
\bibitem{BernheimRangelAddiction} B. Douglas Bernheim; Antonio Rangel. Addiction and Cue-Triggered Decision Processes. \textit{American Economic Review}. vol. 94. no. 5. pp. 1558-590. 2004.
\bibitem{BernheimRangelWelfare} B. Douglas Bernheim; Antonio Rangel. Beyond Revealed Preference: Choice-Theoretic Foundations for Behavioral Welfare Economics. \textit{Quarterly Journal of Economics}. vol. 124. no. 1. pp. 51-104. 2009.
\bibitem{BernheimPopovFradkin} B. Douglas Bernheim; Igor Popov; Andrey Fradkin. The Welfare Economics of Default Options in 401(k) Plans. \textit{American Economic Review}. vol. 105. no. 9. pp. 2798-837. 2015.
\bibitem{McKenzieLiersch} Craig R. M. Mckenzie; Michael J. Liersch. Misunderstanding Savings Growth: Implications for Retirement Savings Behavior. \textit{Journal of Marketing Research}. vol. 48. pp. 1-13. 2011.
\bibitem{ChettyLooneyKroft} Raj Chetty; Adam Looney; Kory Kroft. Salience and Taxation: Theory and Evidence. \textit{American Economic Review}. vol. 99. no. 4. pp. 1145-177. 2009.
\bibitem{EcheniqueLeeShum} Federico Echenique; Sangmok Lee; Matthew Shum. The Money Pump as a Measure of Revealed Preference Violations. \textit{Journal of Political Economy}. vol. 119. no. 6. pp. 1201-223. 2011.
\bibitem{ColeShastry} Shawn Cole; Gauri Kartini Shastry. Is High School the Right Time to Teach Self-control? The Effect of Financial Education and Mathematics Courses on Savings Behavior. \textit{Unpublished Manuscript}. 2010.
\bibitem{CalvetCampbellSodiniJPE} Laurent E. Calvet; John Y. Campbell; Paolo Sodini. Down or Out: Assessing the Welfare Costs of Household Investment Mistakes. \textit{Journal of Political Economy}. vol. 115. no. 5. pp. 707-47. 2007.
\bibitem{Karivetal.} Syngjoo Choi; Shachar Kariv; Wieland M{\"u}ller; Dan Silverman. Who is (More) Rational?. \textit{American Economic Review}. vol. 104. no. 6. pp. 1518-550. 2014.
\bibitem{AlmenbergGerdes} Johan Almenberg; Christer Gerdes. Exponential Growth Bias and Financial Literacy. \textit{Applied Economics Letters}. vol. 19. no. 17. pp. 1693-696. 2012.
\bibitem{FEDbulletin} Jesse Bricker; Arthur B. Kennickell; Kevin B. Moore; John Sabelhaus. Changes in U.S. Family Finances from 2007 to 2010: Evidence from the Survey of Consumer Finances. \textit{Federal Reserve Bulletin}. vol. 98. no. 2. 2012.
\bibitem{LusardiRepresentative} Annamaria Lusardi. Americans' Financial Capability. \textit{NBER Working Paper}. vol. 17103. 2011.
\bibitem{KalaciSerraGarcia} Kenan Kalayci; Marta Serra-Garcia. Complexity and Biases. \textit{Experimental Economics}. vol. 19. no. 1. pp. 31-50. 2016.
\bibitem{AbelerJaeger} Johannes Abeler; Simon J{\"a}ger. Complex Tax Incentives. \textit{American Economic Journal: Economic Policy}. vol. 7. no. 3. pp. 1-28. 2015.
\bibitem{EisensteinHoch} Eric M. Eisenstein; Stephen J. Hoch. Intuitive Compounding: Framing, Temporal Perspective, and Expertise. \textit{Unpublished Manuscript}. 2007.
\bibitem{Song} Changcheng Song. Financial Illiteracy and Pension Contributions: A Field Experiment on Compound Interest in China. \textit{Unpublished Manuscript}. 2015.
\bibitem{StangoZinman} Victor Stango; Jonathan Zinman. Exponential Growth Bias and Household Finance. \textit{The Journal of Finance}. vol. 64. no. 6. pp. 2807-849. 2009.
\bibitem{LevyTasoff} Matthew Levy; Joshua Tasoff. Exponential Growth Bias and Lifecycle Consumption. \textit{Journal of the European Economic Association}. vol. 14. no. 3. pp. 545-83. 2016.
\bibitem{Hungetal.2009} Angela A. Hung,; Andrew M. Parker; Joanne K. Yoong. Defining and Measuring Financial Literacy. \textit{RAND working paper}. vol. WR-708. 2009.
\bibitem{MasonSuri} Winter Mason; Siddarth Suri. Conducting Behavioral Research on Amazon's Mechanical Turk. \textit{Behavior Research Methods}. vol. 44. no. 1. pp. 1-23. 2012.
\bibitem{AndreoniSprengerTime} James Andreoni; Charles Sprenger. Estimating Time Preferences from Convex Budgets. \textit{American Economic Review}. vol. 102. no. 7. pp. 3333-356. 2012.
\bibitem{DufloSaezSocialInteraction} Esther Duflo; Emmanuel Saez. The Role of Information and Social Interactions in Retirement Plan Decisions: Evidence from a Randomized Experiment. \textit{Quarterly Journal of Economics}. vol. 118. no. 3. pp. 815-42. 2003.
\bibitem{BernheimGarrettMaki} B. Douglas Bernheim; Daniel M. Garrett; Dean M. Maki. Education and Saving: The Long-Term Effects of High School Financial Curriculum Mandates. \textit{Journal of Public Economics}. vol. 80. pp. 435-65. 2001.
\bibitem{BernheimGarrett} B. Douglas Bernheim; Daniel M. Garrett. The Effects of Financial Education in the Workplace: Evidence from a Survey of Households. \textit{Journal of Public Economics}. vol. 87. pp. 1487-519. 2003.
\bibitem{Kline} Paul Kline. Handbook of Psychological Testing. Routledge. 1999.
\bibitem{Bayeretal.} Patrick J. Bayer; B. Douglas Bernheim; John Karl Scholz. The Effects of Financial Education in the Workplace: Evidence from a Survey of Employers. \textit{Economic Inquiry}. vol. 47. no. 4. pp. 605-24. 2009.
\bibitem{LaibsonGoldenEggs} David Laibson. Golden Eggs and Hyperbolic Discounting. \textit{Quarterly Journal of Economics}. vol. 112. no. 2. pp. 443-77. 1997.
\bibitem{Hastingsetal.Review} Justine S. Hastings; Brigitte C. Madrian; William L. Skimmyhorn. Financial Literacy, Financial Education, and Economic Outcomes. \textit{Annual Review of Economics}. vol. 5. pp. 347-73. 2013.
\bibitem{LusardiMitchellReview} Annamaria Lusardi; Olivia Mitchell. The Economic Importance of Financial Literacy: Theory and Evidence. \textit{Journal of Economic Literature}. vol. 52. no. 1. pp. 1-44. 2014.
\bibitem{FinancingYourFuture} Council for Economic Education. Financing Your Future (DVD). 2006.
\bibitem{Walstadetal.} William B. Walstad; Ken Rebeck; Richard A. MacDonald. The Effects of Financial Education on the Financial Knowledge of High School Students. \textit{Journal of Consumer Affairs}. vol. 44. no. 2. pp. 336-57. 2010.
\bibitem{Lusardietal.VisualTools} Annamaria Lusardi; Anya Samek; Arie Kapteyn; Lewis Glinert; Angela Hung; Aileen Heinberg. Visual Tools and Narratives: New Ways to Improve Financial Literacy. \textit{Journal of Pension Economics and Finance}. pp. 1-27. 2015.
\bibitem{Luhrmannetal.} Melanie L{\"u}hrmann; Marta Serra-Garcia; Joachim Winter. Teaching Teenagers in Finance: Does It Work?. \textit{Journal of Banking \& Finance}. vol. 54. pp. 160-74. 2015.
\bibitem{Heinbergetal.FiveSteps} Aileen Heinberg; Angela Hung; Arie Kapteyn; Annamaria Lusardi; Anya Savikhin Samek; Joanne Yoong. Five Steps to Planning Success: Experimental Evidence from U.S. Households. \textit{Oxford Review of Economic Policy}. vol. 30. no. 4. pp. 697-724. 2014.
\bibitem{MalkielEllis} Burt G. Malkiel; Charles D. Ellis. The Elements of Investing: Easy Lessons for Every Investor. Wiley. 2013.
\bibitem{LusardiMitchellOrdinaryConsumers} Annamaria Lusardi; Olivia S Mitchell. How ordinary consumers make complex economic decisions: Financial literacy and retirement readiness. \textit{Quarterly Journal of Finance}. World Scientific. vol. 7. no. 03. pp. 1750008. 2017.
\bibitem{AndersenMPL} Steffen Andersen; Glenn W. Harrison; Morten I. Lau; E. Elisabet Rutstrom. Elicitation using Multiple Price List Formats. \textit{Experimental Economics}. vol. 9. pp. 383-405. 2006.
\bibitem{HoltLaury} Charles A. Holt; Susan K. Laury. Risk Aversion and Incentive Effects. \textit{American Economic Review}. vol. 92. no. 5. pp. 1644-655. 2002.
\bibitem{Levyetal.} Haim Levy; Moshe Levy; Natalie Alisof. Homemade Leverage: Theory versus Experimental Evidence. \textit{Journal of Portfolio Management}. vol. 31. pp. 84-93. 2004.
\bibitem{WillisAER} Lauren E. Willis. The Financial Education Fallacy. \textit{American Economic Review}. vol. 101. no. 3. pp. 429-34. 2011.
\bibitem{vanRooijetal.} Maarten van Rooij; Annamaria Lusardi; Rob Alessie. Financial Literacy and Stock Market Participation. \textit{Journal of Financial Economics}. vol. 101. no. 2. pp. 449-72. 2011.
\bibitem{Kroll3} Yoram Kroll; Haim Levy. Further Tests of the Separation Theorem and the Capital Asset PricingModel. \textit{American Economic Review}. vol. 82. no. 3. pp. 664-70. 1992.
\bibitem{Kroll1} Yoram Kroll; Haim Levy; Amnon Rapoport. Experimental Tests of the Separation Theorem and the Capital Asset Pricing Model. \textit{American Economic Review}. vol. 78. no. 3. pp. 500-19. 1988.
\bibitem{Kroll2} Yoram Kroll; Haim Levy; Amnon Rapoport. Experimental Tests of the Mean-Variance Model for Portfolio Selection. \textit{Organizational Behavior and Human Decision Processes}. vol. 42. pp. 388-410. 1988.
\bibitem{BaltussenPost} Guido Baltussen; Gerrit T. Post. Irrational Diversification: An Examination of Individual Portfolio Choice. \textit{Journal of Financial and Quantitative Analysis}. vol. 5. pp. 1463-491. 2011.
\bibitem{HeathTversky} Chip Heath; Amos Tversky. Preference and belief: Ambiguity and competence in choice under uncertainty.. \textit{Journal of risk and uncertainty}. vol. 4(1). pp. 5-28. 1991.
\bibitem{GneezyPotters} Uri Gneezy; Jan Potters. An experiment on risk taking and evaluation periods. \textit{The Quaterly Journal of Economics}. vol. 112. no. 2. pp. 631-645. 1997.
\bibitem{Thaleretal.} Richard Thaler; Amost Tversky; Daniel Kahneman; Alan Schwartz. The effect of myopia and loss aversion on risk taking: An experimental test.. \textit{The Quarterly Journal of Economics}. vol. 112. no. 2. pp. 647-661. 1997.
\bibitem{GigerenzerTodd} Gerd Gigerenzer; Peter M. Todd; the ABC Research Group. Simple heuristics that make us smart. Oxford University Press. 1999.
\bibitem{BenartziThaler2001} Shlomo Benartzi; Richard H. Thaler. Naive diversification strategies in defined contribution saving plans.. \textit{American Economic Review}. pp. 79-98. 2001.
\bibitem{FoxLanger} Craig R. Fox; Thomas Langer. Biases in allocation under risk and uncertainty: Partition dependence, unit dependence, and proecedure dependence.
\end{thebibliography}


\newpage{}

\newpage{}


\begin{appendix}

\renewcommand\thetable{\thesection.\arabic{table}}
\renewcommand\thefigure{\thesection.\arabic{figure}}


\setcounter{table}{0}
\renewcommand{\thetable}{A\arabic{table}}
\setcounter{figure}{0}
\renewcommand{\thefigure}{A\arabic{figure}}

\setcounter{footnote}{0}


\renewcommand{\thesection}{\Alph{section}}
\renewcommand{\thesubsection}{\Alph{section}.\arabic{subsection}}
\renewcommand{\thesubsubsection}{\Alph{section}.\arabic{subsection}.\arabic{subsubsection}}

\makeatletter    \def\p@subsection{}   \makeatother
\makeatletter    \def\p@subsubsection{}   \makeatother

 \setcounter{page}{0}
\begin{center}
\textbf{\Huge{}{}ONLINE-APPENDIX}{\Huge{}}\\
{\Huge{} \vspace{0.5cm}
 }\textbf{\Huge{}{}NOT FOR PUBLICATION}{\Huge{} }\\
{\Huge{} \vspace{0.5cm}}{\Large{}{}Evaluating Deliberative Competence: \\
A Simple Method with an Application to Financial Choice} \\
{\Huge{} \vspace{1cm}}{\Large{}{}Sandro Ambuehl, B. Douglas Bernheim, Annamaria Lusardi}
\par\end{center}

\setstretch{1.5}


\thispagestyle{empty}


\part{}

\parttoc


\newpage{}


\section{Proofs of Propositions 1-3}

\label{proofs}

\subsection{Proof of Proposition \ref{Prop1-2L}}


In proving this proposition, we write the evaluation standard as $u \in {V,U}$, rather than as $V$, so that we can invoke this result for both $V$ and $U$ in the proof of Proposition \ref{Prop2-2L}.\

\textbf{Step 1:} Proof of equation (\ref{eq:Prop4a}).

For $u\in\{U,V\}$, we can rewrite equation (\ref{eq:Prop4a}) as
follows:
\begin{equation}
\lim_{\alpha\rightarrow0}\left(\frac{l_{M}^{u}\left(r^{u}(y_{c}^{\alpha}),y_{s}^{\alpha}\right)}{u_{m}(0,y_{0})\left|r^{u}\left(y_{c}^{\alpha}\right)-r^{u}\left(y_{s}^{\alpha}\right)\right|}\right)=1.\label{eq:mod_L_R_M}
\end{equation}
The numerator and denominator both converge to zero as $\alpha\rightarrow0$,
so we apply L'H\^opital's rule to the terms $\frac{l_{M}^{u}\left(r^{u}(y_{c}^{\alpha}),y_{s}^{\alpha}\right)}{u_{m}(0,y_{0})}$
and $\left|r^{u}\left(y_{c}^{\alpha}\right)-r^{u}\left(y_{s}^{\alpha}\right)\right|$.

Suppose first that $\left.\frac{\partial r^{u}\left(y_{s}^{\alpha}\right)}{\partial\alpha}\right|_{\alpha=0}-\left.\frac{\partial r^{u}\left(y_{c}^{\alpha}\right)}{\partial\alpha}\right|_{\alpha=0}>0$,
so that $r^{u}\left(y_{s}^{\alpha}\right)>r^{u}\left(y_{c}^{\alpha}\right)$
for small $\alpha$. In that case we can replace $\left|r^{u}\left(y_{c}^{\alpha}\right)-r^{u}\left(y_{s}^{\alpha}\right)\right|$
in the denominator with $r^{u}\left(y_{s}^{\alpha}\right)-r^{u}\left(y_{c}^{\alpha}\right)$.

By definition,

\[
l_{M}^{u}\left(r^{u}(y_{c}^{\alpha}),y_{s}^{\alpha}\right)=u\left(-r^{u}\left(y_{c}^{\alpha}\right),y_{s}^{\alpha}\right)-u\left(0,y_{0}\right).
\]
It follows that
\[
\frac{d}{d\alpha}\left(\frac{l_{M}^{u}\left(r^{u}(y_{c}^{\alpha}),y_{s}^{\alpha}\right)}{u_{m}(0,y_{0})}\right)=-\frac{u_{m}\left(-r^{u}\left(y_{c}^{\alpha}\right),y_{s}^{\alpha}\right)}{u_{m}(0,y_{0})}\frac{\partial r^{u}\left(y_{c}^{\alpha}\right)}{\partial\alpha}+\frac{u_{y}\left(-r^{u}\left(y_{c}^{\alpha}\right),y_{s}^{\alpha}\right)\left(y_{s}-y_{0}\right)}{u_{m}(0,y_{0})}
\]
(Note that the gradient of $u$ with respect to $y$, denoted $u_y$, is a row vector and $(y_s - y_0)$ is a column vector).
Using the fact that
\[
\frac{dr^{u}\left(y_{s}^{\alpha}\right)}{d\alpha}=\frac{u_{y}\left(-r^{u}\left(y_{s}^{\alpha}\right),y_{s}^{\alpha}\right)\left(y_{s}-y_{0}\right)}{u_{m}\left(-r^{u}\left(y_{s}^{\alpha}\right),y_{s}^{\alpha}\right)},
\]
we have
\[
\frac{d}{d\alpha}\left(\frac{l_{M}^{u}\left(r^{u}(y_{c}^{\alpha}),y_{s}^{\alpha}\right)}{u_{m}(0,y_{0})}\right)=-\frac{u_{m}\left(-r^{u}\left(y_{c}^{\alpha}\right),y_{s}^{\alpha}\right)}{u_{m}\left(0,y_{0}\right)}\frac{\partial r^{u}\left(y_{c}^{\alpha}\right)}{\partial\alpha}
\]

\[
+\frac{u_{y}\left(-r^{u}\left(y_{c}^{\alpha}\right),y_{s}^{\alpha}\right)\left(y_{s}-y_{0}\right)}{u_{y}\left(-r^{u}\left(y_{s}^{\alpha}\right),y_{s}^{\alpha}\right)\left(y_{s}-y_{0}\right)}
\frac{ u_{m}\left(-r^{u}\left(y_{s}^{\alpha}\right),y_{s}^{\alpha}\right)}{u_{m}(0,y_{0})}
\frac{\partial r^{u}\left(y_{s}^{\alpha}\right)}{\partial\alpha}
\]
Taking limits, we have

\[
\lim_{\alpha\rightarrow 0}
\frac{d}{d\alpha}\left(\frac{l_{M}^{u}\left(r^{u}(y_{c}^{\alpha}),y_{s}^{\alpha})\right)}{u_{m}(0,y_{0})}\right)=\left.\frac{\partial r^{u}\left(y_{s}^{\alpha}\right)}{\partial\alpha}\right|_{\alpha = 0}-\left.\frac{\partial r^{u}\left(y_{c}^{\alpha}\right)}{\partial\alpha}\right|_{\alpha = 0}
\]
The desired conclusion then follows directly from l'H\^opital's rule.

An analogous argument for the case of $\left.\frac{\partial r^{u}\left(y_{s}^{\alpha}\right)}{\partial\alpha}\right|_{\alpha=0}-\left.\frac{\partial r^{u}\left(y_{c}^{\alpha}\right)}{\partial\alpha}\right|_{\alpha=0}<0$,
which implies $r^{u}\left(y_{s}^{\alpha}\right)>r^{u}\left(y_{c}^{\alpha}\right)$
for small $\alpha$, completes the proof of equation (\ref{eq:mod_L_R_M}).

\textbf{Step 2:} Proof of equation (\ref{eq:Prop4b}).\

For $u\in\{U,V\}$, we can rewrite equation (\ref{eq:Prop4b}) as
follows:

\begin{equation}
\lim_{\alpha\rightarrow0}\left(\frac{\alpha l_{W}^{u}\left(r^{u}(y_{c}^{\alpha}),y_{s}^{\alpha},H^{\alpha}\right)}{\frac{1}{2}\left(r^{u}\left(y_{c}^{\alpha}\right)-r^{u}\left(y_{s}^{\alpha}\right)\right)^{2}hu_{m}(0,y_{0})}\right)=1.\label{eq:mod_L_R_W}
\end{equation}
The numerator and denominator both converge to zero as $\alpha\rightarrow0$,
so we apply L'H\^opital's rule to the terms $\frac{\alpha l_{W}^{u}\left(r^{u}(y_{c}^{\alpha}),y_{s}^{\alpha},H^{\alpha}\right)}{hu_{m}(0,y_{0})}$
and $\frac{1}{2}\left(r^{u}\left(y_{c}^{\alpha}\right)-r^{u}\left(y_{s}^{\alpha}\right)\right)^{2}$.
As we show below, the derivatives of both terms also converge to zero
as $\alpha\rightarrow0$, so two applications of the rule are required.

We begin with $\frac{1}{2}\left(r^{u}\left(y_{c}^{\alpha}\right)-r^{u}\left(y_{s}^{\alpha}\right)\right)^{2}$.
We have:
\[
\frac{d}{d\alpha}\frac{1}{2}\left(r^{u}\left(y_{c}^{\alpha}\right)-r^{u}\left(y_{s}^{\alpha}\right)\right)^{2}=\left(r^{u}\left(y_{c}^{\alpha}\right)-r^{u}\left(y_{s}^{\alpha}\right)\right)\left(\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}-\frac{dr^{u}\left(y_{s}^{\alpha}\right)}{d\alpha}\right),
\]
Given our assumptions concerning bounds on the derivatives of the utility functions, $\frac{dr^{u}\left(y_{f}^{\alpha}\right)}{d\alpha}$ is bounded. Consequently, this expression converges to zero as $\alpha\rightarrow0$ (because both valuations
converge to zero). So we consider the second derivative:
\[
\frac{d^{2}}{d\alpha^{2}}\frac{1}{2}\left(r^{u}\left(y_{c}^{\alpha}\right)-r^{u}\left(y_{s}^{\alpha}\right)\right)^{2}=\left(\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}-\frac{dr^{u}\left(y_{s}^{\alpha}\right)}{d\alpha}\right)^{2}+\left(r^{u}\left(y_{s}^{\alpha}\right)-r^{u}\left(y_{c}^{\alpha}\right)\right)\left(\frac{d^{2}r^{u}\left(y_{c}^{\alpha}\right)}{d\alpha^{2}}-\frac{d^{2}r^{u}\left(y_{s}^{\alpha}\right)}{d\alpha^{2}}\right)
\]
It is straightforward to verify that $\frac{d^{2}r^{u}\left(y_{f}^{\alpha}\right)}{d\alpha^{2}}$ is bounded under our assumptions. It follows that
\begin{equation}
\lim_{\alpha\rightarrow0}\frac{1}{2}\frac{d^{2}}{d\alpha^{2}}\left[\left(r^{u}\left(y_{s}^{\alpha}\right)\right)^{2}-\left(r^{u}\left(y_{c}^{\alpha}\right)\right)^{2}\right]=\left(\lim_{\alpha\rightarrow0}\frac{dr^{u}\left(y_{s}^{\alpha}\right)}{d\alpha}-\lim_{\alpha\rightarrow0}\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}\right)^{2},\label{eq:r2}
\end{equation}
which is non-zero by assumption.

Next consider $\frac{\alpha l_{W}^{u}\left(r^{u}(y_{c}^{\alpha}),y_{s}^{\alpha},H^{\alpha}\right)}{hu_{m}(0,y_{0})}$.
Suppose first that $\left.\frac{dr^{u}\left(y_{s}^{\alpha}\right)}{d\alpha}\right|_{\alpha=0}-\left.\frac{\partial r^{u}\left(y_{c}^{\alpha}\right)}{\partial\alpha}\right|_{\alpha=0}>0$,
so that $r^{u}\left(y_{s}^{\alpha}\right)>r^{u}\left(y_{c}^{\alpha}\right)$
for small $\alpha$. By definition,
\[
\frac{\alpha}{h}l_{W}^{u}\left(r^{u}(y_{c}^{\alpha}),y_{s}^{\alpha},H^{\alpha}\right)=\int_{r^{u}\left(y_{c}^{\alpha}\right)}^{r^{u}\left(y_{s}^{\alpha}\right)}\left[u\left(-p,y_{s}^{\alpha}\right)-u\left(0,y_{0}\right)\right]dp
\]

Noting that the integrand is zero at the upper limit of integration,
it follows that
\[
\frac{d}{d\alpha}\left[\frac{\alpha}{h}l_{W}^{u}\left(r^{u}(y_{c}^{\alpha}),y_{s}^{\alpha},H^{\alpha}\right)\right]=-\left[u\left(-r^{u}\left(y_{c}^{\alpha}\right),y_{s}^{\alpha}\right)-u\left(0,y_{0}\right)\right]\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}+\int_{r^{u}\left(y_{c}^{\alpha}\right)}^{r^{u}\left(y_{s}^{\alpha}\right)}\left[u_{y}\left(-p,y_{s}^{\alpha}\right)(y_s - y_0)\right]dp
\]
Given our assumptions concerning bounds on the derivatives of the utility functions, $\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}$ and $u_{y}\left(-p,y_{s}^{\alpha}\right)(y_s - y_0)$ are bounded. Consequently, both terms of this expression converge to zero as $\alpha\rightarrow0$. We therefore take the second derivative:
\[
\begin{array}{rcl}
\frac{d^{2}}{d\alpha^{2}}\left[\frac{\alpha}{h}l_{W}^{u}\left(r^{u}(y_{c}^{\alpha}),y_{s}^{\alpha},H^{\alpha}\right)\right]
& = & -\left[u\left(-r^{u}\left(y_{c}^{\alpha}\right),y_{s}^{\alpha}\right)-u\left(0,y_{0}\right)\right]\frac{d^{2}r^{u}\left(y_{c}^{\alpha}\right)}{d\alpha^{2}}+u_{m}\left(-r^{u}\left(y_{c}^{\alpha}\right),y_{s}^{\alpha}\right)\left(\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}\right)^{2}\\
\\
 &  & -2u_{y}\left(-r^{u}\left(y_{c}^{\alpha}\right),y_{s}^{\alpha}\right)\left(y_s - y_0 \right) \frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}+u_{y}\left(-r^{u}\left(y_{s}^{\alpha}\right),y_{s}^{\alpha}\right)(y_s - y_0)\frac{dr^{u}\left(y_{s}^{\alpha}\right)}{d\alpha}\\
\\
 &  & +\int_{r^{u}\left(y_{c}^{\alpha}\right)}^{r^{u}\left(y_{s}^{\alpha}\right)}\frac{d}{d\alpha}\left[u_{y}\left(-p,y_{s}^{\alpha}\right)(y_s-y_0)\right]dp
\end{array}
\]
Recall that
\[
\frac{dr^{u}\left(y_{s}^{\alpha}\right)}{d\alpha}=\frac{u_{y}\left(-r^{u}\left(y_{s}^{\alpha}\right),y_{s}^{\alpha}\right)\left(y_{s}-y_{0}\right)}{u_{m}\left(-r^{u}\left(y_{s}^{\alpha}\right),y_{s}^{\alpha}\right)},
\]
We use this expression to substitute for $u_{y}\left(-r^{u}\left(y_{s}^{\alpha}\right),y_{s}^{\alpha}\right)\left(y_{s}-y_{0}\right)$ in the fourth term. We also multiply the third term by $\frac{dr^{u}\left(y_{s}^{\alpha}\right)}{d\alpha}\frac{u_{m}\left(-r^{u}\left(y_{s}^{\alpha}\right),y_{s}^{\alpha}\right)}{u_{y}\left(-r^{u}\left(y_{s}^{\alpha}\right),y_{s}^{\alpha}\right)\left(y_{s}-y_{0}\right)}=1$. Then we take limits as $\alpha\rightarrow0$. The boundedness of $\frac{d^{2}r^{u}\left(y_{f}^{\alpha}\right)}{d\alpha^{2}}$ (noted above) guarantees that the first term vanishes. The boundedness of the first and second derivatives of the utility functions guarantees that the last term vanishes. We thus obtain:
\begin{eqnarray} \nonumber
\lim_{\alpha\rightarrow 0} \frac{d^{2}}{d\alpha^{2}}\left[\frac{\alpha}{h}l_{W}^{u}\left(r^{u}(y_{c}^{\alpha}),y_{s}^{\alpha},H^{\alpha}\right)\right] \nonumber
& = & u_{m}\left(0,y_0\right) \left[\left(\lim_{\alpha\rightarrow 0}\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}\right)^{2}
- 2
\left(\lim_{\alpha\rightarrow 0}\frac{dr^{u}\left(y_{s}^{\alpha}\right)}{d\alpha}\right)
 \left(\lim_{\alpha\rightarrow 0}\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}\right)
+\left(\lim_{\alpha\rightarrow 0}\frac{dr^{u}\left(y_{s}^{\alpha}\right)}{d\alpha}\right)^2\right] \\ \nonumber \\
&=&  u_{m}\left(0,y_0\right) \left[\lim_{\alpha\rightarrow 0}\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha} - \lim_{\alpha\rightarrow 0}\frac{dr^{u}\left(y_{s}^{\alpha}\right)}{d\alpha}\right]^2 \label{eq:l2}
\end{eqnarray}

Equation (\ref{eq:mod_L_R_W}) follows immediately from equations
(\ref{eq:r2}) and (\ref{eq:l2}).


\hfill $\blacksquare$

\subsection{Proof of Proposition \ref{Prop2-2L}}


\textbf{Step 1.}
We claim that there exists a constant $K$ such that, for all $y_{c}$
and $y_{s}$,
\begin{equation}
\lim_{\alpha\rightarrow0}\left(\frac{r^{U}\left(y_{c}^{\alpha}\right)-r^{U}\left(y_{s}^{\alpha}\right)}{r^{V}\left(y_{c}^{\alpha}\right)-r^{V}\left(y_{s}^{\alpha}\right)}\right)=K.\label{eq:rU_rV}
\end{equation}
Because the numerator and denominator both converge to zero, we apply
L'Hopital's rule:
\[
\lim_{\alpha\rightarrow0}\left(\frac{r^{U}\left(y_{c}^{\alpha}\right)-r^{U}\left(y_{s}^{\alpha}\right)}{r^{V}\left(y_{c}^{\alpha}\right)-r^{V}\left(y_{s}^{\alpha}\right)}\right)=\frac{\lim_{\alpha\rightarrow0}\frac{dr^{U}\left(y_{c}^{\alpha}\right)}{d\alpha}-\lim_{\alpha\rightarrow0}\frac{dr^{U}\left(y_{s}^{\alpha}\right)}{d\alpha}}{\lim_{\alpha\rightarrow0}\frac{dr^{V}\left(y_{c}^{\alpha}\right)}{d\alpha}-\lim_{\alpha\rightarrow0}\frac{dr^{V}\left(y_{s}^{\alpha}\right)}{d\alpha}}
\]
Recall that $r^{u}\left(y\right)$
is defined by the following equation:
\[
u\left(-r^{u}\left(y\right),\varphi\left(y\right)\right)=u\left(0,\varphi\left(y_{0}\right)\right)
\]
Differentiating implicitly with respect to $\alpha$ and evaluating at $y_f^\alpha$ , we obtain:
\[
\frac{d r^{u}\left(y_{f}^{\alpha}\right)}{d\alpha}=\frac{u_{\varphi}\left(-r^{u}\left(y_{f}^{\alpha}\right),\varphi\left(y_{f}^{\alpha}\right)\right)}{u_{m}\left(-r^{u}\left(y_{f}^{\alpha}\right),\varphi\left(y_{f}^{\alpha}\right)\right)}\left(\nabla\varphi\left(y_{f}^{\alpha}\right)\cdot\left(y_{f}-y_{0}\right)\right)
\]
Using the fact that $\lim_{\alpha\rightarrow0}r^{u}\left(y_{f}^{\alpha}\right)=0$,
it follows that
\[
\lim_{\alpha\rightarrow0}\frac{\partial r^{u}\left(y_{f}^{\alpha}\right)}{\partial\alpha}=\frac{u_{\varphi}\left(0,\varphi\left(y_{0}\right)\right)}{u_{m}\left(0,\varphi\left(y_{0}\right)\right)}\left(\nabla\varphi\left(y_{0}\right)\cdot\left(y_{f}-y_{0}\right)\right)
\]
Therefore,

\[
\lim_{\alpha\rightarrow0}\frac{\partial r^{u}\left(y_{c}^{\alpha}\right)}{\partial\alpha}-\lim_{\alpha\rightarrow0}\frac{\partial r^{u}\left(y_{s}^{\alpha}\right)}{\partial\alpha}=\frac{u_{\varphi}\left(0,\varphi\left(y_{0}\right)\right)}{u_{m}\left(0,\varphi\left(y_{0}\right)\right)}\left(\nabla\varphi\left(y_{0}\right)\cdot\left(y_{c}-y_{s}\right)\right)
\]
Accordingly, we have
\[
\lim_{\alpha\rightarrow0}\left(\frac{r^{U}\left(y_{c}^{\alpha}\right)-r^{U}\left(y_{s}^{\alpha}\right)}{r^{V}\left(y_{c}^{\alpha}\right)-r^{V}\left(y_{s}^{\alpha}\right)}\right)=\frac{U_{\varphi}\left(0,\varphi\left(y_{0}\right)\right)V_{m}\left(0,\varphi\left(y_{0}\right)\right)}{U_{m}\left(0,\varphi\left(y_{0}\right)\right)V_{\varphi}\left(0,\varphi\left(y_{0}\right)\right)}\equiv K>0
\]
As claimed, $K$ does not depend on $y_{s}$ or $y_{c}$ (even though
the limits of the numerator and denominator do).

\textbf{Step 2. }
Proof of equations (\ref{eq:Prop5a}) and (\ref{eq:Prop5b}).

First consider $e=M$. We have
\[
\lim_{\alpha\rightarrow0}\left(\frac{L_{M}^{U}\left(r^{U}(y_{c}^{\alpha}),y_{s}^{\alpha}\right)}{d_{M}\left(r^{V}\left(y_{c}^{\alpha}\right),r^{V}\left(y_{s}^{\alpha}\right)\right)}\right)=\lim_{\alpha\rightarrow0}\left(\frac{L_{M}^{U}\left(r^{U}(y_{c}^{\alpha}),y_{s}^{\alpha}\right)}{d_{M}\left(r^{U}\left(y_{c}^{\alpha}\right),r^{U}\left(y_{s}^{\alpha}\right)\right)}\right)\lim_{\alpha\rightarrow0}\frac{d_{M}\left(r_{c}^{U}\left(y_{c}^{\alpha}\right),r_{s}^{U}\left(y_{s}^{\alpha}\right)\right)}{d_{M}\left(r_{c}^{V}\left(y_{c}^{\alpha}\right),r_{s}^{V}\left(y_{s}^{\alpha}\right)\right)}
\]
From Proposition \ref{Prop1-2L}, we know that the first term after
the equals sign is unity. For the second term, we have
\[
\lim_{\alpha\rightarrow0}\frac{d_{M}\left(r_{c}^{U}\left(y_{c}^{\alpha}\right),r_{s}^{U}\left(y_{s}^{\alpha}\right)\right)}{d_{M}\left(r_{c}^{V}\left(y_{c}^{\alpha}\right),r_{s}^{V}\left(y_{s}^{\alpha}\right)\right)}=\lim_{\alpha\rightarrow0}\left|\frac{r_{c}^{U}\left(y_{c}^{\alpha}\right)-r_{s}^{U}\left(y_{s}^{\alpha}\right)}{r_{c}^{V}\left(y_{c}^{\alpha}\right)-r_{s}^{V}\left(y_{s}^{\alpha}\right)}\right|=\left|\lim_{\alpha\rightarrow0}\frac{r_{c}^{U}\left(y_{c}^{\alpha}\right)-r_{s}^{U}\left(y_{s}^{\alpha}\right)}{r_{c}^{V}\left(y_{c}^{\alpha}\right)-r_{s}^{V}\left(y_{s}^{\alpha}\right)}\right|=K
\]
Consequently, equation (\ref{eq:Prop5a}) holds for $K_{M}=K$.

Now consider $e=W$. We have
\[
\lim_{\alpha\rightarrow0}\left(\frac{L_{W}^{U}\left(r^{U}(y_{c}^{\alpha}),y_{s}^{\alpha},H^{\alpha}\right)}{d_{W}\left(r^{V}\left(y_{c}^{\alpha}\right),r^{V}\left(y_{s}^{\alpha}\right)\right)}\right)=\lim_{\alpha\rightarrow0}\left(\frac{L_{W}^{U}\left(r^{U}(y_{c}^{\alpha}),y_{s}^{\alpha},H^{\alpha}\right)}{d_{W}\left(r^{U}\left(y_{c}^{\alpha}\right),r^{U}\left(y_{s}^{\alpha}\right)\right)}\right)\lim_{\alpha\rightarrow0}\frac{d_{W}\left(r_{c}^{U}\left(y_{c}^{\alpha}\right),r_{s}^{U}\left(y_{s}^{\alpha}\right)\right)}{d_{W}\left(r_{c}^{V}\left(y_{c}^{\alpha}\right),r_{s}^{V}\left(y_{s}^{\alpha}\right)\right)}
\]
From Proposition \ref{Prop1-2L}, we know that the first term
after the equals sign is $h$. For the second term, we have
\[
\lim_{\alpha\rightarrow0}\frac{d_{W}\left(r_{c}^{U}\left(y_{c}^{\alpha}\right),r_{s}^{U}\left(y_{s}^{\alpha}\right)\right)}{d_{W}\left(r_{c}^{V}\left(y_{c}^{\alpha}\right),r_{s}^{V}\left(y_{s}^{\alpha}\right)\right)}=\lim_{\alpha\rightarrow0}\left(\frac{r_{c}^{U}\left(y_{c}^{\alpha}\right)-r_{s}^{U}\left(y_{s}^{\alpha}\right)}{r_{c}^{V}\left(y_{c}^{\alpha}\right)-r_{s}^{V}\left(y_{s}^{\alpha}\right)}\right)^{2}=\left(\lim_{\alpha\rightarrow0}\frac{r_{c}^{U}\left(y_{c}^{\alpha}\right)-r_{s}^{U}\left(y_{s}^{\alpha}\right)}{r_{c}^{V}\left(y_{c}^{\alpha}\right)-r_{s}^{V}\left(y_{s}^{\alpha}\right)}\right)^{2}=K^{2}
\]
Consequently, equation (\ref{eq:Prop5b}) holds for $K_{W}=hK^{2}$.

\hfill $\blacksquare$


\subsection{Proof of Proposition \ref{Prop3-2L}}


Define

\[
\kappa(y_{s})\equiv\left(\frac{U_{\varphi}\left(0,\varphi\left(y_{0}\right)\right)\nabla\varphi\left(y_{0}\right)\cdot\left(y_{s}-y_{0}\right)}{U_{m}\left(0,\varphi\left(y_{0}\right)\right)}\right)
\]

First consider $e=M$. Proceeding as in the proof of Proposition \ref{Prop2-2L},
we have

\[
\lim_{\alpha\rightarrow0}\left(\frac{L_{M}^{U}\left(r^{U}(y_{c\theta}^{\alpha}),y_{s}^{\alpha}\right)}{d_{M}\left(r^{V}\left(y_{c\theta}^{\alpha},\theta\right),r^{V}\left(y_{s}^{\alpha},\theta\right)\right)}\right)=\frac{U_{\varphi}\left(0,\varphi\left(y_{0}\right)\right)V_{m}\left(0,\varphi\left(y_{0}\right),\theta\right)}{U_{m}\left(0,\varphi\left(y_{0}\right)\right)V_{\varphi}\left(0,\varphi\left(y_{0}\right),\theta\right)}
\]
Furthermore,

\[
\lim_{\alpha\rightarrow0}\left(\frac{\alpha\kappa(y_{s})}{r^{V}(y_{s}^{\alpha},\theta)}\right)=\lim_{\alpha\rightarrow0}\left(\frac{r^{V}(y_{s}^{\alpha},\theta)}{\alpha}\right)^{-1}\kappa(y_{s})=\left(\left.\frac{dr^{V}(y_{s}^{\alpha},\theta)}{d\alpha}\right|_{\alpha=0}\right)^{-1}\kappa(y_{s})
\]

\[
=\left(\frac{V_{m}\left(0,\varphi\left(y_{0}\right),\theta\right)}{V_{\varphi}\left(0,\varphi\left(y_{0}\right),\theta\right)\nabla\varphi\left(y_{0}\right)\cdot\left(y_{s}-y_{0}\right)}\right)\kappa(y_{s})=\frac{U_{\varphi}\left(0,\varphi\left(y_{0}\right)\right)V_{m}\left(0,\varphi\left(y_{0}\right),\theta\right)}{U_{m}\left(0,\varphi\left(y_{0}\right)\right)V_{\varphi}\left(0,\varphi\left(y_{0}\right),\theta\right)}
\]
Equation (\ref{eq:Prop6a}) is thereby verified.

Now consider $e=W$. Proceeding as in the proof of Proposition \ref{Prop2-2L}, we
have

\[
\lim_{\alpha\rightarrow0}\left(\frac{L_{W}^{U}\left(r^{U}(y_{c\theta}^{\alpha}),y_{s}^{\alpha},H^{\alpha}\right)}{d_{W}\left(r^{V}\left(y_{c\theta}^{\alpha},\theta\right),r^{V}\left(y_{s}^{\alpha},\theta\right)\right)}\right)=h\left(\frac{U_{\varphi}\left(0,\varphi\left(y_{0}\right)\right)V_{m}\left(0,\varphi\left(y_{0}\right),\theta\right)}{U_{m}\left(0,\varphi\left(y_{0}\right)\right)V_{\varphi}\left(0,\varphi\left(y_{0}\right),\theta\right)}\right)^{2}
\]
Furthermore,

\[
\lim_{\alpha\rightarrow0}h\left(\frac{\alpha\kappa(y_{s})}{r^{V}(y_{s}^{\alpha},\theta)}\right)^{2}=\lim_{\alpha\rightarrow0}h\left(\frac{r^{V}(y_{s}^{\alpha},\theta)}{\alpha}\right)^{-2}\left(\kappa(y_{s})\right)^{2}=\left(\left.\frac{dr^{V}(y_{s}^{\alpha},\theta)}{d\alpha}\right|_{\alpha=0}\right)^{-2}\left(\kappa(y_{s})\right)^{2}
\]

\[
=h\left(\frac{V_{m}\left(0,\varphi\left(y_{0}\right),\theta\right)\kappa(y_{s})}{V_{\varphi}\left(0,\varphi\left(y_{0}\right),\theta\right)\nabla\varphi\left(y_{0}\right)\cdot\left(y_{s}-y_{0}\right)}\right)^{2}=h\left(\frac{U_{\varphi}\left(0,\varphi\left(y_{0}\right)\right)V_{m}\left(0,\varphi\left(y_{0}\right),\theta\right)}{U_{m}\left(0,\varphi\left(y_{0}\right)\right)V_{\varphi}\left(0,\varphi\left(y_{0}\right),\theta\right)}\right)^{2}
\]
Equation (\ref{eq:Prop6b}) is thereby verified.

Finally, note that when $y$ is a scalar, we have $\kappa\left(y_s\right)=\left(y_s-y_0\right)\kappa^*$, where
\[
\kappa^*\equiv\left(\frac{U_{\varphi}\left(0,\varphi\left(y_{0}\right)\right)\varphi'\left(y_{0}\right)}{U_{m}\left(0,\varphi\left(y_{0}\right)\right)}\right).
\]


\hfill $\blacksquare$


\section{Additional theoretical results} \label{MoreTheory}


For the approximations in the main test, we held the the magnitude of the misunderstanding fixed and let the consequences of the instrument shrink. Another possibility is to hold the scale of the instrument constant and let the magnitude of the misunderstanding shrink. As we noted in the text, the latter approach may provide better approximations for large instruments (which may be particularly useful in settings where risk preferences play a central role), but worse approximations for large misunderstandings. This section develops this alternative approach.

Formally, we define $y_c^\alpha = \alpha y_c + (1-\alpha) y_s$, and study the relationship between the welfare loss and the valuations $r^V\left(y_c^\alpha \right)$ and $r^V(y_s)$ as $\alpha$ shrinks to zero. Because the scale of the instrument remains fixed as we vary $\alpha$, we take the distribution of prices, $H$, to be fixed as well. We also assume that the density, $h(p)$, is bounded and differentiable with a bounded derivative. This alternative formulation requires a slight modification of the welfarist criterion. As before, we convert $l_e^u$ to money-metric utility (for $e\in\{M, W\}$), but in this case we divide by $u_m\left(-r^u(y_s),y_s\right)$.


We now state and discuss counterparts for the three propositions in the main text. Proofs follow.

\begin{proposition}\label{propInterpretation} For all $y_{c}$ satisfying $\left.\frac{dr^V\left(y_c^\alpha \right)}{d\alpha} \right|_{\alpha=0} \neq 0$, we have
\begin{equation}
\label{Prop1M}
\lim_{\alpha\rightarrow0}\left(\frac{L_{M}^{V}\left(r^V(y_{c}^{\alpha}),y_{s}\right)}{d_{M}\left(r^{V}(y_{c}^{\alpha}),r^{V}(y_{s})\right)}\right)=1
\end{equation}
and
\begin{equation}
\label{Prop1W}
\lim_{\alpha\rightarrow0}\left(\frac{L_{W}^{V}\left(r^V(y_{c}^{\alpha}),y_{s},H\right)}{d_{W}\left(r^{V}(y_{c}^{\alpha}),r^{V}(y_{s})\right)}\right)=h\left(r^{V}(y_{s})\right)
\end{equation}

\end{proposition}

Two distinctions between Propositions \ref{Prop1-2L} and \ref{propInterpretation} merit discussion. First, the constant of proportionality in equation (\ref{Prop1W}) involves $h\left(r^V(y_s)\right)$ rather than the scalar $h$. In either case, to aggregate over instruments, one must assume that the pertinent density is uncorrelated with measured deliberative competence. Second, for Proposition \ref{propInterpretation}, the money-metric scaling factor $u_m\left(-r^u(y_s),y_s\right)$ also depends on the instrument through $y_s$. Aggregation over instruments therefore requires either that the differences in the marginal utility of income are small (e.g., because curvature is modest over the relevant range), or that measured deliberative competence is uncorrelated with simply framed valuations across instruments.


\begin{proposition}\label{IdealizedWelfare} For each $e\in\{W, M\}$, there exists a strictly positive function $K_e(y_s)$, such that for all $y_{c}$ satisfying $\left.\frac{dr^V\left(y_c^\alpha \right)}{d\alpha} \right|_{\alpha=0} \neq 0$, we have
\begin{equation}
\label{Prop2M}
\lim_{\alpha\rightarrow0}\left(\frac{L_{M}^{U}\left(r^{U}\left(y_{c}^{\alpha}\right),y_{s}\right)}{d_{M}\left(r^{V}\left(y_{c}^{\alpha}\right),r^{V}\left(y_{s}\right)\right)}\right)=K_{M}\left(y_{s}\right)
\end{equation}
\begin{equation}
\label{Prop2W}
\lim_{\alpha\rightarrow0}\left(\frac{L_{W}^{U}\left(r^{U}\left(y_{c}^{\alpha}\right),y_{s},H\right)}{d_{W}\left(r^{V}\left(y_{c}^{\alpha}\right),r^{V}\left(y_{s}\right)\right)}\right)=K_{W}\left(y_{s}\right)
\end{equation}

\end{proposition}

An important difference between Propositions \ref{Prop2-2L} and \ref{IdealizedWelfare} is that, for the latter, the constants of proportionality depend on the instrument (even with no differences in the density term across instruments for the welfarist criterion). Once again, this feature introduces some qualifications with respect to aggregating over instruments for the alternative approach.

Our next proposition concerns potential policy-induced confounds. It identifies an adjustment to measured deliberative competence required to restore comparability across policies when they potentially impact simply framed valuations. It involves a function $\rho^{V}\left(y_{s},\theta\right)$, defined as follows. Let $y_s^\beta = \beta y_s + (1-\beta)y_0$, where $\beta$ is a scalar. Then $\rho^{V}\left(y_{s},\theta\right) = \left. \frac{dr^V(y_s^\beta,\theta)}{d\beta}\right|_{\beta=1}$. In words, $\rho^{V}\left(y_{s},\theta\right)$ is the amount by which the simply framed valuation changes as we scale up the instrument's consequences. Critically, the value of this term is easily inferred from the type of data we collected in our experiment.

\begin{proposition}\label{Correction} For each $e\in\{W, M\}$, there exists a strictly positive function $\kappa_e(y_s)$  such that, for all $y_{c\theta}$ satisfying $\left.\frac{dr^V\left(y_c^\alpha,\theta \right)}{d\alpha} \right|_{\alpha=0} \neq 0$ and policies $\theta$, we have
\begin{equation}
\label{Prop3M}
\lim_{\alpha\rightarrow0}\left(\frac{L_{M}^{U}\left(r^U(y_{c}^{\alpha}),y_{s}\right)}{d_{M}^{V}\left(r^{V}\left(y_{c\theta}^{\alpha},\theta\right),r^{V}\left(y_{s},\theta\right)\right)}\right)=\frac{\kappa_M(y_{s})}{\rho^{V}\left(y_{s},\theta\right)}
\end{equation}
and
\begin{equation}
\label{Prop3W}
\lim_{\alpha\rightarrow0}\left(\frac{L_{W}^{U}\left(r^U(y_{c}^{\alpha}),y_{s},H\right)}{d_{W}^{V}\left(r^{V}\left(y_{c\theta}^{\alpha},\theta\right),r^{V}\left(y_{s},\theta\right)\right)}\right)=\frac{\kappa_W(y_{s})}{\left[\rho^{V}\left(y_{s},\theta\right)\right]^2}
\end{equation}

\end{proposition}


According to this proposition, we can address the confound simply by rescaling our measures of deliberative competence. In particular, once we divide $d_{M}^{V}\left(r^{V}\left(y_{c\theta},\theta\right),r^{V}\left(y_{s},\theta\right)\right)$ by $\rho^{V}\left(y_{s},\theta\right)$, and
$d_{W}^{V}\left(r^{V}\left(y_{c\theta},\theta\right),\right.$ $\left.r^{V}\left(y_{s},\theta\right)\right)$ by $\left[\rho^{V}\left(y_{s},\theta\right)\right]^2$, the main implications of Proposition \ref{IdealizedWelfare} follow.

Section \ref{resultsCorrectionAlt} uses Proposition \ref{Correction} to correct for policy-induced framing effects and shows that our empirical results are qualitatively unchanged.


\subsection{Proof of Proposition \ref{propInterpretation}}


In proving this proposition, we write the evaluation standard as $u\in\{V,U\}$, rather than as $V$, so that we can invoke this result for both  $V$ and $U$ in the proof of Proposition \ref{IdealizedWelfare}.

The proof involves two steps.
\

\textbf{Step 1:} Proof of equation (\ref{Prop1M}).

Suppose first that $\left.\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}\right|_{\alpha=0}<0$. Hence, for all sufficiently small $\alpha$ we have that  $r^{u}\left(y_{s}\right)>r^{u}\left(y_{c}^{\alpha}\right)$
and that $r^{u}\left(y_{c}^{\alpha}\right)$ is strictly increasing.
In this  case the denominator of the expression
in equation (\ref{Prop1M}) is $r^{u}\left(y_{s}\right)-r^{u}\left(y_{c}^{\alpha}\right)$.
Because $L_{M}^{u}\left(r^u(y_{c}^{\alpha}),y_{s}\right)$ (the numerator)
and $r^{u}\left(y_{s}\right)-r^{u}\left(y_{c}^{\alpha}\right)$ both
converge to zero as $\alpha\rightarrow0$, we apply l'Hopital's rule.

For the numerator, the facts that $u$ is strictly increasing in its
first component, and that $r^{u}\left(y_{c}^{\alpha}\right)$ is strictly increasing, imply
\[
L_{M}^u\left(r^u(y_{c}^{\alpha}),y_{s}\right)=\max_{p\in\left[r^{u}\left(y_{c}^{\alpha}\right),r^{u}\left(y_{s}\right)\right]}\left[\frac{u\left(-p,y_{s}\right)-u\left(0,y_{0}\right)}{u_{m}(-r^{u}(y_{s}),y_{s})}\right]=\frac{u\left(-r^{u}\left(y_{c}^{\alpha}\right),y_{s}\right)-u\left(0,y_{0}\right)}{u_{m}(-r^{u}(y_{s}),y_{s})}
\]
It follows that
\[
\frac{dL_{M}^u\left(r^u(y_{c}^{\alpha}),y_{s}\right)}{d\alpha}=-\frac{u_{m}\left(-r^{u}\left(y_{c}^{\alpha}\right),y_{s}\right)}{u_{m}(-r^{u}(y_{s}),y_{s})}\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha},
\]
and accordingly that
\[
\lim_{\alpha\rightarrow0}\frac{dL_{M}^u\left(r^u(y_{c}^{\alpha}),y_{s}\right)}{d\alpha}=-\lim_{\alpha\rightarrow0}\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}
\]

For the denominator, the derivative is $-\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}$,
so the limit is the same as for the numerator. Equation (\ref{Prop1M}) follows (in this case) immediately from L'Hopital's
rule.

Now suppose $\left.\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}\right|_{\alpha=0}>0$,
so that $r^{u}\left(y_{s}\right)>r^{u}\left(y_{c}^{\alpha}\right)$
for small $\alpha$, in which case the denominator of the expression
in equation (\ref{Prop1M}) is $r^{u}\left(y_{c}^{\alpha}\right)-r^{u}\left(y_{s}\right)$.
In this case, for the numerator, we have

\[
L_{M}^u\left(r^u(y_{c}^{\alpha}),y_{s}\right)=\max_{p\in\left[r^{u}\left(y_{s}\right),r^{u}\left(y_{c}^{\alpha}\right)\right]}\left[\frac{u\left(0,y_{0}\right)-u\left(-p,y_{s}\right)}{u_{m}(-r^{u}(y_{s}),y_{s})}\right]=\frac{u\left(0,y_{0}\right)-u\left(-r^{u}\left(y_{c}^{\alpha}\right),y_{s}\right)}{u_{m}(-r^{u}(y_{s}),y_{s})}
\]
The remainder of the argument is the same, except that the signs of
the numerator and denominator are both switched, so the limiting ratio
of derivatives is again unity.

\textbf{Step 2:} Proof of equation (\ref{Prop1W}).

Suppose first $\left.\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}\right|_{\alpha=0}<0$,
so that $r^{u}\left(y_{s}\right)>r^{u}\left(y_{c}^{\alpha}\right)$
for small $\alpha$. Because $L_{W}^{u}\left(r^u(y_{c}^{\alpha}),y_{s},H\right)$
(the numerator of the expression in equation (\ref{Prop1W})) and $d_{W}\left(r^{u}\left(y_{c}^{\alpha}\right),r^{u}\left(y_{s}\right)\right)$ (the denominator)
both converge to zero as $\alpha\rightarrow0$, we apply l'Hopital's
rule.

Consider the numerator, $L_{W}^{u}\left(r^u(y_{c}^{\alpha}),y_{s},H\right)$.
By definition,
\[
L_{W}^{u}\left(r^u(y_{c}^{\alpha}),y_{s},H\right)=\frac{1}{u_{m}(-r^{u}(y_{s}),y_{s})}\int_{r^{u}\left(y_{c}^{\alpha}\right)}^{r^{u}\left(y_{s}\right)}\left[u\left(-p,y_{s}\right)-u\left(0,y_{0}\right)\right]h(p)dp
\]
It follows that
\[
\frac{d}{d\alpha}L_{W}^{u}\left(r^u(y_{c}^{\alpha}),y_{s},H\right)=-\left[\frac{u\left(-r^{u}\left(y_{c}^{\alpha}\right),y_{s}\right)-u\left(0,y_{0}\right)}{u_{m}(-r^{u}(y_{s}),y_{s})}\right]\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}h\left(r^{u}\left(y_{c}^{\alpha}\right)\right)
\]
Given our assumptions concerning bounds on the density and derivatives of the utility functions, $\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}$, $\frac{1}{u_{m}(-r^{u}(y_{s}),y_{s})}$, and $h\left(r^{u}\left(y_{c}^{\alpha}\right)\right)$ are all bounded, so this expression converges to zero as $\alpha\rightarrow0$.
Anticipating the same property for the denominator, we take
the second derivative:
\[
\frac{d^{2}}{d\alpha^{2}}L_{W}^{u}\left(r^u(y_{c}^{\alpha}),y_{s},H\right)=-\left[\frac{u\left(-r^{u}\left(y_{c}^{\alpha}\right),y_{s}\right)-u\left(0,y_{0}\right)}{u_{m}(-r^{u}(y_{s}),y_{s})}\right]\left[\frac{d^{2}r^{u}\left(y_{c}^{\alpha}\right)}{d\alpha^{2}}h\left(r^{u}\left(y_{c}^{\alpha}\right)\right)+\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}\frac{dh\left(r^{u}\left(y_{c}^{\alpha}\right)\right)}{d\alpha}\right]
\]
\[
+\frac{u_{m}\left(-r^{u}\left(y_{c}^{\alpha}\right),y_{s}\right)}{u_{m}(-r^{u}(y_{s}),y_{s})}\left(\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}\right)^{2}h\left(r^{u}\left(y_{c}^{\alpha}\right)\right)
\]
As before, our assumptions imply that $\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}$, $\frac{1}{u_{m}(-r^{u}(y_{s}),y_{s})}$, and $h\left(r^{u}\left(y_{c}^{\alpha}\right)\right)$ are bounded. Boundedness of the derivative of the density then implies that $\frac{dh\left(r^{u}\left(y_{c}^{\alpha}\right)\right)}{d\alpha}$ is bounded. It is also straightforward to verify that $\frac{d^{2}r^{u}\left(y_{c}^{\alpha}\right)}{d\alpha^{2}}$ is bounded under our assumptions. It follows that, as $\alpha\rightarrow0$, the first term converges to zero. Because the
ratio in the second term converges to unity, we have:
\[
\lim_{\alpha\rightarrow0}\frac{d^{2}}{d\alpha^{2}}L_{W}^{u}\left(r^u(y_{c}^{\alpha}),y_{s},H\right)=h\left(r^{u}\left(y_{s}\right)\right)\lim_{\alpha\rightarrow0}\left(\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}\right)^{2}
\]

Now consider the denominator, $\frac{1}{2}\left(r^{u}\left(y_{s}\right)-r^{u}\left(y_{c}^{\alpha}\right)\right)^{2}$.
Taking the derivative yields
\[
\frac{d}{d\alpha}\left[\frac{1}{2}\left(r^{u}\left(y_{s}\right)-r^{u}\left(y_{c}^{\alpha}\right)\right)^{2}\right]=-\left(r^{u}\left(y_{s}\right)-r^{u}\left(y_{c}^{\alpha}\right)\right)\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}
\]
Because $\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}$ is bounded, this expression converges to 0 as $\alpha\rightarrow0$, as claimed above. Taking
the second derivative yields
\[
\frac{d^{2}}{d\alpha^{2}}\left[\frac{1}{2}\left(r^{u}\left(y_{s}\right)-r^{u}\left(y_{c}^{\alpha}\right)\right)^{2}\right]=-\left(r^{u}\left(y_{s}\right)-r^{u}\left(y_{c}^{\alpha}\right)\right)\frac{d^{2}r^{u}\left(y_{c}^{\alpha}\right)}{d\alpha^2}+\left(\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}\right)^{2}
\]
As $\alpha\rightarrow0$, the first term converges to zero (given the boundedness of the second derivative noted above), so we
have
\[
\lim_{\alpha\rightarrow0}\frac{d^{2}}{d\alpha^{2}}\left[\frac{1}{2}\left(r^{u}\left(y_{s}\right)-r^{u}\left(y_{c}^{\alpha}\right)\right)^{2}\right]=\lim_{\alpha\rightarrow0}\left(\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}\right)^{2}
\]
Thus, the ratio of the limit of second derivatives for the numerator
and the denominator is $h\left(r^{u}\left(y_{s}\right)\right)$. Equation (\ref{Prop1W}) follows (in this case) immediately
from L'Hopital's rule.

Now suppose $\left.\frac{dr^{u}\left(y_{c}^{\alpha}\right)}{d\alpha}\right|_{\alpha=0}>0$,
so that $r^{u}\left(y_{s}\right)>r^{u}\left(y_{c}^{\alpha}\right)$
for small $\alpha$. The same analysis of the denominator applies.
For the numerator, the limits of the integral change places, as do
the two terms in the integrand. As a result, the first and second
derivatives are unchanged. Consequently, the same arguments deliver
equation (\ref{Prop1W}) in this case as well.


\hfill $\blacksquare$


\subsection{Proof of Proposition \ref{IdealizedWelfare}}


The proof involves two steps.
\

\textbf{Step 1:} We claim that there exists a constant $K(y_{s})$ such that, for
all $y_{c}$ and $y_{s}$,
\begin{equation}
\label{RRlimit}
\lim_{\alpha\rightarrow0}\left(\frac{r^{U}\left(y_{c}^{\alpha}\right)-r^{U}\left(y_{s}\right)}{r^{V}\left(y_{c}^{\alpha}\right)-r^{V}\left(y_{s}\right)}\right)=K(y_{s}).
\end{equation}

Because the numerator and denominator of the preceding expression both converge to zero, we apply L'Hopital's rule:
\[
\lim_{\alpha\rightarrow0}\left(\frac{r^{U}\left(y_{c}^{\alpha}\right)-r^{U}\left(y_{s}\right)}{r^{V}\left(y_{c}^{\alpha}\right)-r^{V}\left(y_{s}\right)}\right)=\frac{\lim_{\alpha\rightarrow0}\frac{dr^{U}\left(y_{c}^{\alpha}\right)}{d\alpha}}{\lim_{\alpha\rightarrow0}\frac{d r^{V}\left(y_{c}^{\alpha}\right)}{d\alpha}}
\]
Recall that $r^{V}(y)$ is defined by the
following equation:
\[
V\left(-r^{V}\left(y\right),\varphi\left(y\right)\right)=V\left(0,\varphi\left(y_{0}\right)\right)
\]
Evaluating $r^V(y)$ at $y_c^\alpha$ and differentiating implicitly with respect to $\alpha$, we obtain:
\[
\frac{dr^{V}\left(y_{c}^{\alpha}\right)}{d\alpha}=\frac{V_{\varphi}\left(-r^{V}\left(y_{c}^{\alpha}\right),\varphi\left(y_{c}^{\alpha}\right)\right)}{V_{m}\left(-r^{V}\left(y_{c}^{\alpha}\right),\varphi\left(y_{c}^{\alpha}\right)\right)}\left(\nabla\varphi\left(y_{c}^{\alpha}\right)\cdot\left(y_{c}-y_{s}\right)\right)
\]
Using the fact that $\lim_{\alpha\rightarrow0}r^{V}\left(y_{c}^{\alpha}\right)=r^V(y_s)$,
it follows that
\[
\lim_{\alpha\rightarrow0}\frac{d r^{V}\left(y_{c}^{\alpha}\right)}{d\alpha}=\frac{V_{\varphi}\left(-r^{V}(y_{s}),\varphi\left(y_{0}\right)\right)}{V_{m}\left(-r^{V}(y_{s}),\varphi\left(y_{0}\right)\right)}\left(\nabla\varphi\left(y_{0}\right)\cdot\left(y_{c}-y_{s}\right)\right)
\]
Repeating these calculations for $U$, we obtain
\[
\lim_{\alpha\rightarrow0}\frac{d r^{U}\left(y_{c}^{\alpha}\right)}{d\alpha}=\frac{U_{\varphi}\left(-r^{U}(y_{s}),\varphi\left(y_{0}\right)\right)}{U_{m}\left(-r^{U}(y_{s}),\varphi\left(y_{0}\right)\right)}\left(\nabla\varphi\left(y_{0}\right)\cdot\left(y_{c}-y_{s}\right)\right)
\]
Accordingly, we have
\[
\lim_{\alpha\rightarrow0}\left(\frac{r^{U}\left(y_{c}^{\alpha}\right)-r^{U}\left(y_{s}\right)}{r^{V}\left(y_{c}^{\alpha}\right)-r^{V}\left(y_{s}\right)}\right)=\frac{U_{\varphi}\left(-r^{U}(y_{s}),\varphi\left(y_{s}\right)\right)V_{m}\left(-r^{V}(y_{s}),\varphi\left(y_{s}\right)\right)}{U_{m}\left(-r^{U}(y_{s}),\varphi\left(y_{s}\right)\right)V_{\varphi}\left(-r^{V}(y_{s}),\varphi\left(y_{s}\right)\right)}\equiv K(y_{s})>0
\]
As claimed, $K$ does not depend on $y_{c}$ (even though the limits of the derivatives
of the numerator and denominator with respect to $\alpha$ in equation (\ref{RRlimit}) do).

\textbf{Step 2:} Proof of equations (\ref{Prop2M}) and (\ref{Prop2W}).

Consider $e=M$. We have
\[
\lim_{\alpha\rightarrow0}\left(\frac{L_{M}^{U}\left(r^U(y_{c}^{\alpha}),y_{s}\right)}{d_{M}\left(r^{V}\left(y_{c}^{\alpha}\right),r^{V}\left(y_{s}\right)\right)}\right)=\lim_{\alpha\rightarrow0}\left(\frac{L_{M}^{U}\left(r^U(y_{c}^{\alpha}),y_{s}\right)}{d_{M}\left(r^{U}\left(y_{c}^{\alpha}\right),r^{U}\left(y_{s}\right)\right)}\right)\lim_{\alpha\rightarrow0}\left(\frac{d_{M}\left(r^{U}\left(y_{c}^{\alpha}\right),r^{U}\left(y_{s}\right)\right)}{d_{M}\left(r^{V}\left(y_{c}^{\alpha}\right),r^{V}\left(y_{s}\right)\right)}\right)
\]
From Proposition 1 (which holds for $U$ as well as $V$), we know that the first limit
after the equals sign converges to unity. For the second term, we
have
\[
\lim_{\alpha\rightarrow0}\frac{\left|r^{U}\left(y_{c}^{\alpha}\right)-r^{U}\left(y_{s}\right)\right|}{\left|r^{V}\left(y_{c}^{\alpha}\right)-r^{V}\left(y_{s}\right)\right|}=\lim_{\alpha\rightarrow0}\left|\frac{r^{U}\left(y_{c}^{\alpha}\right)-r^{U}\left(y_{s}\right)}{r^{V}\left(y_{c}^{\alpha}\right)-r^{V}\left(y_{s}\right)}\right|=\left|\lim_{\alpha\rightarrow0}\frac{r^{U}\left(y_{c}^{\alpha}\right)-r^{U}\left(y_{s}\right)}{r^{V}\left(y_{c}^{\alpha}\right)-r^{V}\left(y_{s}\right)}\right|=K(y_{s})
\]
Therefore, equation (\ref{Prop2M}) holds for $K_{M}(y_{s})=K(y_{s})$.

Now consider $e=W$. We have
\[
\lim_{\alpha\rightarrow0}\left(\frac{L_{W}^{U}\left(r^U(y_{c}^{\alpha}),y_{s},H\right)}{d_{W}\left(r^{V}\left(y_{c}^{\alpha}\right),r^{V}\left(y_{s}\right)\right)}\right)=\lim_{\alpha\rightarrow0}\left(\frac{L_{W}^{U}\left(r^U(y_{c}^{\alpha}),y_{s},H\right)}{d_{W}\left(r^{U}\left(y_{c}^{\alpha}\right),r^{U}\left(y_{s}\right)\right)}\right)\lim_{\alpha\rightarrow0}\left(\frac{d_{W}\left(r^{U}\left(y_{c}^{\alpha}\right),r^{U}\left(y_{s}\right)\right)}{d_{W}\left(r^{V}\left(y_{c}^{\alpha}\right),r^{V}\left(y_{s}\right)\right)}\right)
\]
From Proposition 1, we know that the limit after
the equals sign converges to $h\left(r^{U}(y_{s})\right)$. For the
second term, we have
\[
\lim_{\alpha\rightarrow0}\frac{\left(r^{U}\left(y_{c}^{\alpha}\right)-r^{U}\left(y_{s}\right)\right)^{2}}{\left(r^{V}\left(y_{c}^{\alpha}\right)-r^{V}\left(y_{s}\right)\right)^{2}}=\lim_{\alpha\rightarrow0}\left(\frac{r^{U}\left(y_{c}^{\alpha}\right)-r^{U}\left(y_{s}\right)}{r^{V}\left(y_{c}^{\alpha}\right)-r^{V}\left(y_{s}\right)}\right)^{2}=\left(\lim_{\alpha\rightarrow0}\frac{r^{U}\left(y_{c}^{\alpha}\right)-r^{U}\left(y_{s}\right)}{r^{V}\left(y_{c}^{\alpha}\right)-r^{V}\left(y_{s}\right)}\right)^{2}=\left[K\left(y_{s}\right)\right]^{2}
\]
Therefore, equation (\ref{Prop2W}) holds for $K_{W}\left(y_{s}\right)=h\left(r^{U}(y_{s})\right)\left[K\left(y_{s}\right)\right]^{2}$.


\hfill $\blacksquare$


\subsection{Proof of Proposition \ref{Correction}}


Proceeding as in the proof of Proposition \ref{IdealizedWelfare},
Step 1, we see that, for any given $\theta$,

\begin{equation}
\lim_{\alpha\rightarrow0}
\left(
\frac
{r^{U}\left(y_{c\theta}^{\alpha}\right)-r^{U}\left(y_{s}\right)}
{r^{V}\left(y_{c\theta}^{\alpha},\theta\right)-r^{V}\left(y_{s},\theta\right)}
\right)
=\frac{U_{\varphi}\left(-r^{U}(y_{s}),\varphi\left(y_{s}\right)\right)V_{m}\left(-r^{V}(y_{s},\theta),\varphi\left(y_{s}\right),\theta\right)}{U_{m}\left(-r^{U}(y_{s}),\varphi\left(y_{s}\right)\right)V_{\varphi}\left(-r^{V}(y_{s},\theta),\varphi\left(y_{s}\right),\theta\right)}\label{eq:welfare ratio}
\end{equation}
Moreover, it is straightforward to check that

\begin{equation}
\rho^{V}\left(y_{s},\theta\right)=\left.\frac{\partial r^{V}\left(y_{s}^{\beta},\theta\right)}{\partial\beta}\right|_{\beta=1}=\frac{V_{\varphi}\left(-r^{V}(y_{s},\theta),\varphi\left(y_{s}\right),\theta\right)}{V_{m}\left(-r^{V}(y_{s},\theta),\varphi\left(y_{s}\right),\theta\right)}\left(\nabla\varphi\left(y_{s}\right)\cdot\left(y_{s}-y_{0}\right)\right)\label{eq:r derivative}
\end{equation}
Defining $\kappa(y_{s})\equiv\frac{U_{\varphi}\left(-r^{U}(y_{s}),\varphi\left(y_{s}\right)\right)}{U_{m}\left(-r^{U}(y_{s}),\varphi\left(y_{s}\right)\right)}\nabla\varphi\left(y_{s}\right)\cdot\left(y_{s}-y_{0}\right)$,
equations (\ref{eq:welfare ratio}) and (\ref{eq:r derivative}) then
imply
\begin{equation}
\lim_{\alpha\rightarrow0}\left(\frac{r^{U}\left(y_{c\theta}^{\alpha}\right)-r^{U}\left(y_{s}\right)}{r^{V}\left(y_{c\theta}^{\alpha},\theta\right)-r^{V}\left(y_{s},\theta\right)}\right)=\frac{\kappa(y_{s})}{\rho^{V}\left(y_{s},\theta\right)}\label{eq:limiting value}
\end{equation}

Consider $e=M$. Analogously to Step 2 of Proposition \ref{IdealizedWelfare}, we have
\begin{equation}
\lim_{\alpha\rightarrow0}\left(\frac{L_{M}^{U}\left(r^U(y_{c\theta}^{\alpha}),y_{s}\right)}{d_{M}\left(r^{V}\left(y_{c\theta}^{\alpha},\theta\right),r^{V}\left(y_{s},\theta\right)\right)}\right)=\lim_{\alpha\rightarrow0}\left(\frac{L_{M}^{U}\left(r^U(y_{c\theta}^{\alpha}),y_{s}\right)}{d_{M}\left(r^{U}\left(y_{c\theta}^{\alpha}\right),r^{U}\left(y_{s}\right)\right)}\right)\lim_{\alpha\rightarrow0}\left(\frac{d_{M}\left(r^{U}\left(y_{c\theta}^{\alpha}\right),r^{U}\left(y_{s}\right)\right)}{d_{M}\left(r^{V}\left(y_{c\theta}^{\alpha},\theta\right),r^{V}\left(y_{s},\theta\right)\right)}\right)\label{eq:product1}
\end{equation}
Proposition \ref{propInterpretation}, as applied to $U$, is unchanged. Therefore,
the first term on the right equals 1. Using the same arguments as in Step 2 of Proposition \ref{IdealizedWelfare}, equation (\ref{Prop3M}) follows directly from equations (\ref{eq:limiting value})
and (\ref{eq:product1}), with $\kappa_M(y_s)=\kappa(y_s)$.

Now consider $e=W$. Analogously to Step 2 of Proposition \ref{IdealizedWelfare}, we have
\begin{equation}
\lim_{\alpha\rightarrow0}\left(\frac{L_{W}^{U}\left(r^U(y_{c\theta}^{\alpha}),y_{s},H\right)}{d_{W}\left(r^{V}\left(y_{c\theta}^{\alpha},\theta\right),r^{V}\left(y_{s},\theta\right)\right)}\right)=\lim_{\alpha\rightarrow0}\left(\frac{L_{W}^{U}\left(r^U(y_{c\theta}^{\alpha}),y_{s},H\right)}{d_{W}\left(r^{U}\left(y_{c\theta}^{\alpha}\right),r^{U}\left(y_{s}\right)\right)}\right)\lim_{\alpha\rightarrow0}\left(\frac{d_{W}\left(r^{U}\left(y_{c\theta}^{\alpha}\right),r^{U}\left(y_{s}\right)\right)}{d_{W}\left(r^{V}\left(y_{c\theta}^{\alpha},\theta\right),r^{V}\left(y_{s},\theta\right)\right)}\right)\label{eq:product2}
\end{equation}
Again, Proposition \ref{propInterpretation}, as applied to $U$, is unchanged. Therefore,
the first term on the right equals $h\left(r^{U}\left(y_{s}\right)\right)$. Using the same arguments as in Step 2 of Proposition \ref{IdealizedWelfare}, equation (\ref{Prop3W}) follows directly from equations (\ref{eq:limiting value})
and (\ref{eq:product2}), with $\kappa_W(y_s)=\left[\kappa(y_s)\right]^2h\left(r^U(y_s)\right)$.


\hfill $\blacksquare$


\section{Experiment details}

\label{experimentDetails}

In this section we detail the implementation of the experiment. Screenshots
of the instructions and the experimental interface are in Appendix
\ref{Instructions}.

\paragraph{Amazon Mechanical Turk}

Workers log on to AMT through an interface that displays a list of
\emph{Human Intelligence Tasks} (HITs), each with a title, an estimated
duration, and an estimated remuneration rate. Other HITs include taking
surveys, categorizing images, writing product descriptions, and identifying
performers on music recordings.

To ensure that subjects were \emph{technically} able to view the videos,
we told them at the outset of the study that access to youtube.com
was required. We also asked them to reproduce the last word spoken
in the welcome video, and the last word of the title slide of whichever
treatment video they viewed. Subjects who were not able to complete
these tasks correctly were not allowed to continue with the study.
The videos were embedded in the survey so that subjects could not
find the other treatment videos used in this study.

We ensured that each subject participated in our study only once using
the unique identifying numbers assigned by AMT.\footnote{Nonetheless, one subject managed to participate in our study twice.
Both times, this subject exhibited multiple switching points, and
hence is excluded from all analyses.} A subject can only receive payment for participation in the study
if she correctly provides this information, and hence has no incentive
for misrepresentation.

\paragraph{Initial Financial Literacy}

Before participating in the main stages of the experiment, subjects
completed the unincentivized financial literacy test in Table \ref{financialLiteracyQuestionnaire}.
This test of financial literacy originated with \citet{LusardiMitchellOrdinaryConsumers}
and \citet{vanRooijetal.}, and has been used in many other studies
\citep{LusardiMitchellReview}.

\paragraph{Attention to the Video}

Before subjects watched the treatment video, we informed them that,
with 25\% probability, their earnings would be entirely determined
by their performance on a test,\footnote{\citeappendix{Hastingsetal.Review} criticize most existing studies
that use such test scores as outcome measures on the grounds that
the tests are unincentivized. One of the few exceptions is \citeappendix{LevyTasoff}.} and that `to be able to answer the questions in the test, you need
to both understand and know the contents of the video.' We also explained
that the video could help them make better decisions both during the
experiment and in real life, inasmuch as it was made by `internationally
recognized academic experts on financial decision making.' Finally,
we disabled the \emph{continue} button for the duration of the video.

\paragraph{Iterated Multiple Price List}

Each line of each price list was a binary choice between the future
reward and a specified dollar amount to be received no more than two
days after completion of the experiment. For the first price list,
the immediate payment varied from \$0 to \$20 in increments of \$2.
For the second price list, it varied from \$$x$ to \$$(x+1.8)$ in
increments of \$0.20, where $x+2$ is the smallest amount chosen over
the future reward in the first list. (See appendix \ref{Instructions}
for screenshots of the computer interface.) If a subjects' payment
was determined according to a price list, the randomization over lines
proceeded as follows. A line was randomly selected from the first
price list. If that line did not correspond to $x$ (defined above),
it was implemented. Otherwise, a random line from the second price
list was selected, and the decision for that line was implemented.
With this procedure, truthful revelation of preferences is optimal.


\paragraph{Questionnaire}

Questions concerning decision strategies employed the following wording.
Use of the rule of 72 in complexly framed problems: ``Sometimes in
this experiment, you were given a choice such as `We will invest \$10
in an account with 1\% interest per week. Interest is compounded weekly.
We will pay you the proceeds in 72 days.' When deciding about this
choice, did you use the rule of 72?''\footnote{The survey question incorrectly described the interest rate as pertaining
to a week rather than a day. We believe the meaning of the question
was nevertheless clear despite this typo.} Use of the rule of 72 in simply framed problems: ``Sometimes in
this experiment, you were given a choice such as `We will pay you
\$20 in 36 days.' When deciding about such a choice, did you use the
rule of 72?'' In both cases, subjects answered either ``Yes'',
``No'', or ``I don't know the rule of 72.'' Number of problems
for which the future reward was calculated explicitly: ``In total,
you were given 10 rounds in which one of the options was something
like `we will invest \$... in an account with ...\% interest per day.
Interest is compounded daily. We will pay you the proceeds in... days.'
Out of these 10 rounds, how many times did you explicitly calculate
the money amount that this investment would yield within the specified
time?'' Subjects responded by selecting an integer between 0 and
10. Use of external help on the test: ``When you completed the test
about the video on financial investing, did you use external resources
(such as other websites, books, etc.) to find the right answers?''
Subjects answered either ``Yes'' or ``No.''

We also asked subjects how much attention they had paid to their choices,
how much attention they had paid to the video, whether they had any
suggestions about the study, and whether they had experienced any
technical difficulties. The overwhelming majority of subjects reported
the highest level of attention in answer to both questions---a finding
we interpret with caution.

\setcounter{table}{0}
\begin{table}
\caption{\label{financialLiteracyQuestionnaire}Financial Literacy questionnaire.}
\setstretch{1.2}


\begin{small}
\begin{tabular}{p{\textwidth}}
\toprule
\textsf{FL1. Suppose you had \$100 in a savings account and the interest
rate was 2 percent per year. After 5 years, how much do you think
you would have in the account if you left the money to grow?}\\
 \emph{More than \$102 (92.03\%), Exactly \$102 (4.61\%), Less than
\$102 (1.99\%), Do not know (1.37\%)} \medskip\\

\textsf{FL2. Suppose you had \$100 in a savings account and the interest
rate is 20 percent per year and you never withdraw money or interest
payments. After 5 years, how much would you have on this account in
total?} \\
 \emph{More than \$200 (75.84\%), Exactly \$200 (19.68\%), Less than
\$200 (2.74\%), Do not know (1.74\%)}\medskip\\

\textsf{FL3. Imagine that the interest rate on your savings account
was 1 percent per year and inflation was 2 percent per year. After
1 year, how much would you be able to buy with the money in this account?}
\\
 \emph{More than today (7.35\%), exactly the same (4.98\%), less than
today (84.43\%), do not know (3.24\%)} \medskip\\

\textsf{FL4. Assume a friend inherits \$10,000 today and his sibling
inherits \$10,000 3 years from now. Who is richer because of the inheritance?}
\\
 \emph{My friend (56.79\%), his sibling (6.72\%), they are equally
rich (29.89\%), do not know (6.60\%)}\medskip\\

\textsf{FL5. Suppose that in the year 2015, your income has doubled
and prices of all goods have doubled too. In 2015, how much will you
be able to buy with your income?}\\
 \emph{More than today (4.23\%), the same (90.16\%), less than today
(4.73\%), do not know (0.87\%)}\\
\bottomrule
\end{tabular}
 \end{small}
 \footnotesize
\textbf{\textit{Notes:}} Numbers in brackets indicate the percentage of subjects who chose a given answer. The fraction of correct answers to each question only differs slightly across experiments and treatments, see Table \ref{DemographicsTable}.
\end{table}


\paragraph{Test questions in Control interventions}
Panels A and B of Table \ref{testQuestionsControl} display the test questions about the Control intervention in Experiments A and B, respectively. The two experiments involve different sets of questions about their respective Control interventions because the Control interventions differ. We decided to use a different Control intervention for Experiment B because the Control intervention in Experiment A is largely descriptive, and hence is not well-suited to incorporating practice questions with individualized feedback.


\begin{table}
\caption{\label{testQuestionsControl} Test questions concerning the Control
interventions.}
\setstretch{1}

\begin{small}

\begin{centering}
\textbf{Experiment A} \\
 \hrulefill{}
\par\end{centering}
\textsf{Which of the following quotes is attributed to Benjamin Franklin?
}\\
 \emph{Compound interest is the most powerful force in the universe;
Youth is wasted on the young; Money makes money. And the money that
money makes, makes money; }\\

\textsf{Which quote is attributed to the author Upton Sinclair?} \\
 \emph{Only liars manage always to be out of the market during bad
times and in during good times; It is difficult to get a man to understand
something when his salary depends upon his not understanding it; There
are three classes of people who do not believe that markets work:
the Cubans, the North Koreans, and active managers; Nobody knows more
than the market. }\\

\textsf{What percentage of mutual funds tends to be outperformed by
the market (S\&P 500 Index) each year?} \\
 \emph{between 10 and 30\% between 30 and 50\% between 50 and 70\%
between 70 and 90\% }\\

\textsf{What is an ``indexing'' investment strategy?} \\
 \emph{Buying index funds, which hold assets that have been indexed
as particularly profitable by financial experts; Buying index funds,
which hold stocks of companies that provide information about the
stock market as a whole (stock market indices); Buying index funds,
which hold the market portfolio; Buying index funds, which hold optimally
diversified, custom tailored portfolios.}\\

\textsf{Professional investors as a whole are responsible for what
percentage of stock market trading? }\\
 \emph{30\%; 50\%; 70\%; 90\%.}  \\

\vspace{-0.5cm}
 \hrulefill{}

\begin{center}
\textbf{Experiment B} \\
\end{center}
\vspace{-0.3cm}
 \hrulefill{} \\
\textsf{In order to limit your risk, you might invest in which of
the following pairs of stocks? }\\
 \emph{Microsoft and Google; General Motors and Chrysler; Coca-Cola
and Pepsi; General Motors and Microsoft; Facebook and Twitter. }\\

\textsf{We would expect the degree of relation between the returns
of Coca-Cola stock and the returns of Pepsi stock to be closest to
\textunderscore\textunderscore\textunderscore\textunderscore\textunderscore
? {[}-1 means perfect negative relation and +1 means perfect positive
relation{]}:} \\
 \emph{-0.7; -0.3; 0; 0.3; 0.7. }\\

\textsf{Considering a long time period (for example 10 or 20 years),
which asset normally gives the highest return?} \\
 \emph{Savings accounts; Corporate bonds; Government bonds; T-Bills;
Stocks. }\\

\textsf{Normally, which asset displays the highest fluctuations over
time? }\\
 \emph{Savings accounts; Corporate bonds; Government bonds; T-Bills;
Stocks.}\\

\textsf{A degree of relation of \textunderscore\textunderscore\textunderscore\textunderscore\textunderscore
between two assets will NOT help reduce your risk.} \\
 \emph{1; 0.5; 0; -0.5; -1.} \\
 \vspace{-0.3cm}
 \hrulefill \\
\end{small}

\end{table}

\newpage
\section{Additional Data Analysis}

\label{Robustness}\label{Inertia}

\subsection{Demographics}

\label{Demographics}

Table \ref{DemographicsTable} presents detailed demographics of our
subject pool by treatment, as well as their initial financial literacy.\footnote{These statistics only include subjects who did not exhibit multiple
switching points in any of the price lists.} Column 5 lists data for the representative US citizen. Demographic
variables are taken from the 2010 US Census. Employment variables
are for April 2014, and come from the Bureau of Labor Statistics.
Financial literacy scores are from \citeappendix{LusardiRepresentative},
and from the 2012 FED bulletin for stock holdings.\footnote{J. Bricker, A. B. Kennickell K. B. Moore, and J. Sabelhaus, 2012, \textit{Changes in U.S. Family Finances from 2007 to 2010: Evidence from the Survey of Consumer Finances}, Federal Reserve Bulletin, 98(2).} (Representative
data on financial literacy only exist for questions FL1 and FL3.)
For empty cells, no representative data are available. Column 6 reports,
for each variable, the $p$-value of an $F$-test for differences
across treatments. The number of significant differences is well within
the range we would expect given the number of tests performed.

As reported in section \ref{preliminaryAnalysis}, our sample is poorer,
better educated, and more likely to live in larger households than
the average US citizen. While the incidence of full-time employment
in our sample mirrors that of the general population, the fraction
of respondents who classify themselves as employed part-time is double
that of the general population. Our subjects are also disproportionately
male and white, younger, slightly more urban, and more likely to have
never been married than the representative US citizen.

\setcounter{table}{0}

\begin{table}[htbp!]
\caption{\label{DemographicsTable} Demographics and financial literacy.}
\noindent
\noindent\begin{minipage}[c]{1\textwidth}
\begin{center}
\setlength{\tabcolsep}{3pt}

\begin{tabular}{lccccccccc}
\toprule
 & (1)  & (2)  & (3)  & (4)  & (5)  & (6)  & (7)  & (8)  & (9) \\
 & \multicolumn{5}{c}{Experiment A} & \multicolumn{3}{c}{Experiment B} & \\
\cmidrule(r){2-6} \cmidrule{7-9}Treatment  & Treat.  & Cont.  & Subst.  & Rhet.  & p-value  & Treat.  & Cont.  & p-value  & US \\
 &  &  & only  & only  &  &  &  &  & \\
\midrule
FL1  & 0.93  & 0.92  & 0.92  & 0.95  & 0.83  & 0.90  & 0.92  & 0.59  & 0.65 \\
FL2  & 0.81  & 0.73  & 0.73  & 0.71  & 0.32  & 0.75  & 0.80  & 0.29  & - \\
FL3  & 0.82  & 0.82  & 0.83  & 0.85  & 0.93  & 0.88  & 0.85  & 0.38  & 0.64 \\
FL4  & 0.58  & 0.64  & 0.50  & 0.59  & 0.17  & 0.53  & 0.59  & 0.31  & - \\
FL5  & 0.96  & 0.90  & 0.87  & 0.91  & 0.09{*}  & 0.91  & 0.88  & 0.30  & - \\
FL1 - FL3 all correct  & 0.71  & 0.63  & 0.62  & 0.62  & 0.44  & 0.68  & 0.70  & 0.72  & - \\
FL1 - FL5 all correct  & 0.47  & 0.45  & 0.34  & 0.40  & 0.20  & 0.40  & 0.42  & 0.60  & - \\
\midrule
Male  & 0.57  & 0.57  & 0.61  & 0.50  & 0.40  & 0.47  & 0.53  & 0.28  & 0.49 \\
Age (median)  & 28  & 32  & 29  & 29  & 0.09{*}  & 36  & 34  & 0.11  & 37.2\\
Household Income (median)\tablefootnote{In our survey, household income is interval coded. The values stated
are the midpoints of the median intervals.}  & 45,000  & 35,000  & 45,000  & 35,000  & 0.06{*}{*}  & 45,000  & 57,500  & 0.15  & 53,046 \\
\emph{Race}  &  &  &  &  &  &  &  &  & \\
\hspace{0.5cm}African-american  & 0.08  & 0.06  & 0.08  & 0.04  & 0.68  & 0.05  & 0.10  & 0.10{*}  & 0.13 \\
\hspace{0.5cm}Asian  & 0.08  & 0.11  & 0.12  & 0.05  & 0.22  & 0.06  & 0.06  & 0.90  & 0.05 \\
\hspace{0.5cm}Caucasian  & 0.81  & 0.72  & 0.72  & 0.77  & 0.34  & 0.82  & 0.79  & 0.41  & 0.63 \\
\hspace{0.5cm}Hispanic  & 0.03  & 0.07  & 0.03  & 0.10  & 0.06{*}{*}  & 0.04  & 0.04  & 0.88  & 0.17 \\
\hspace{0.5cm}Other  & 0.01  & 0.04  & 0.05  & 0.04  & 0.44  & 0.02  & 0.01  & 0.37  & 0.02 \\
\emph{Education}  &  &  &  &  &  &  &  &  & \\
\hspace{0.5cm} Less than high school  & 0.01  & 0.00  & 0.00  & 0.00  & 0.35  & 0.00  & 0.00  & 1.00  & 0.14 \\
\hspace{0.5cm} High school  & 0.13  & 0.12  & 0.15  & 0.14  & 0.92  & 0.14  & 0.13  & 0.71  & 0.31 \\
\hspace{0.5cm} Vocational / technical  & 0.08  & 0.08  & 0.08  & 0.03  & 0.29  & 0.07  & 0.06  & 0.56  & 0.09 \\
\hspace{0.5cm} Some college  & 0.35  & 0.37  & 0.33  & 0.44  & 0.34  & 0.31  & 0.34  & 0.67  & 0.19 \\
\hspace{0.5cm} College  & 0.39  & 0.37  & 0.38  & 0.34  & 0.90  & 0.37  & 0.37  & 0.97  & 0.18 \\
\hspace{0.5cm} Graduate degree  & 0.05  & 0.06  & 0.07  & 0.05  & 0.88  & 0.11  & 0.11  & 0.88  & 0.09 \\
\emph{Employment}  &  &  &  &  &  &  &  &  & \\
\hspace{0.5cm}Full time employed  & 0.50  & 0.50  & 0.48  & 0.43  & 0.70  & 0.61  & 0.68  & 0.16  & 0.48\tablefootnote{Percentage of civilian noninstitutional population that is full-time
employed.} \\
\hspace{0.5cm}Part time employed  & 0.21  & 0.23  & 0.26  & 0.27  & 0.72  & 0.18  & 0.16  & 0.60  & 0.11\tablefootnote{Percentage of civilian noninstitutional population that is part-time
employed.} \\
\emph{Marital Status}\tablefootnote{Our questionnaire included the option ``Prefer not to say''. The
three subjects who chose this response are not accounted for in this
table. }  &  &  &  &  &  &  &  &  & \\
\hspace{0.5cm} Married  & 0.28  & 0.30  & 0.32  & 0.29  & 0.94  & 0.46  & 0.47  & 0.80  & 0.27 \\
\hspace{0.5cm} Widowed  & 0.00  & 0.00  & 0.00  & 0.00  & 1.00  & 0.01  & 0.01  & 0.53  & 0.56 \\
\hspace{0.5cm} Divorced  & 0.07  & 0.05  & 0.04  & 0.04  & 0.80  & 0.04  & 0.06  & 0.37  & 0.06 \\
\hspace{0.5cm} Never married  & 0.64  & 0.65  & 0.64  & 0.64  & 1.00  & 0.49  & 0.46  & 0.69  & 0.10 \\
\emph{Urban / Rural}  &  &  &  &  &  &  &  &  & \\
\hspace{0.5cm}Urban and suburban  & 0.17  & 0.17  & 0.11  & 0.17  & 0.48  & 0.25  & 0.17  & 0.06{*}  & 0.81 \\
\hspace{0.5cm}Rural  & 0.83  & 0.83  & 0.89  & 0.83  & 0.48  & 0.75  & 0.83  & 0.06{*}  & 0.19 \\
\emph{Household size}  &  &  &  &  &  &  &  &  & \\
\hspace{0.5cm}1  & 0.18  & 0.13  & 0.11  & 0.19  & 0.26  & 0.10  & 0.13  & 0.33  & 0.22\\
\hspace{0.5cm}2  & 0.22  & 0.24  & 0.25  & 0.24  & 0.95  & 0.27  & 0.23  & 0.42  & 0.36\\
\hspace{0.5cm}3  & 0.14  & 0.19  & 0.17  & 0.22  & 0.46  & 0.18  & 0.22  & 0.35  & 0.17\\
\hspace{0.5cm}4 or more  & 0.46  & 0.44  & 0.47  & 0.35  & 0.23  & 0.45  & 0.41  & 0.50  & 0.26\\
Owns stocks  & 0.16  & 0.23  & 0.20  & 0.23  & 0.54  & 0.43  & 0.39  & 0.44  & 0.15 \\
\midrule
$N$  & 109  & 106  & 128  & 112  & -  & 169  & 179  & -  & -\\
\midrule

\end{tabular}
\par\end{center}
\footnotesize
\textbf{\textit{Notes:}}  The sample includes all subjects who completed the study and did not exhibit any multiple switching points. Column 5 presents the $p$-values of an $F$-test for joint equality of the coefficients listed in columns 1 - 4. Column 8 presents the $p$-value of a $t$-test for joint equality of the coefficients in columns 6 - 7. Column 9 lists comparison values for the representative US citizen whenever available. See text for data sources.

\end{minipage}
\end{table}


\subsection{Main results controlling for demographics}

\label{demoControlsAppendix}

Table \ref{demoControls} presents our main results in a regression
that includes data from both experiments and controls for demographics.
Demographic controls consist of all variables listed in Table \ref{DemographicsTable},
except for the summary statistics ``FL1-FL3 all correct'' and ``FL1-FL5
all correct''. For brevity, we pool across the timeframes.

In each case we see that coefficient estimates are barely changed in comparison to the estimates in the main text, which do not control for demographic characteristics. We conclude that the differences between experiments
A and B reflect differences in the interventions rather than differences in subject characteristics.


\begin{table}
\caption{Main results controlling for demographics in joint analysis of both experiments.}
\label{demoControls}
\begin{centering}
\setstretch{1}

\begin{tabular}{lccccccc}
\toprule
 & (1) & (2) & (3) & (4) & (5) & (6) \\
VARIABLES  & \multicolumn{2}{c}{Test scores on module} & \multicolumn{2}{c}{Valuation in frame}  & \multicolumn{2}{c}{{Deliberative Competence}} \\
\cmidrule(r){2-3}  & Treatment  & Control  & Complex  & Simple &  & \\
Correction for changes \\
\quad in valuations in simple frame & &&&& No & Yes \\
\midrule
 &  &  &  &  &  & \\
Difference to Control B  &  &  &  &  &  & \\
\hspace{0.4cm}\textit{Treatment B}  & 1.640*** & -0.375*** & 15.067*** & 7.242*** & 7.901*** & 12.883*** \\
 & (0.139) & (0.124) & (2.790) & (2.164) & (1.773) & (2.563) \\
\hspace{0.4cm}\textit{Treatment A}  & 1.528*** & -0.660*** & 11.928*** & 1.522 & 1.133 & -1.283 \\
 & (0.197) & (0.158) & (3.969) & (3.189) & (2.521) & (4.652) \\
\hspace{0.4cm}\textit{Control A}  & 0.117 & 0.405** & -1.504 & 1.121 & 0.832 & 3.151 \\
 & (0.201) & (0.174) & (3.740) & (3.059) & (2.430) & (3.309) \\
\hspace{0.4cm}\textit{Substance-Only}  & 1.400*** & -0.886*** & 3.406 & 2.158 & 2.525 & -0.418 \\
 & (0.190) & (0.164) & (3.789) & (3.086) & (2.258) & (5.042) \\
\hspace{0.4cm}\textit{Rhetoric-Only}  & 0.608*** & -0.651*** & 16.741*** & 6.611** & 5.247** & 7.755** \\
 & (0.210) & (0.165) & (4.082) & (3.148) & (2.259) & (3.386) \\
Demographic controls  & Yes  & Yes  & Yes  & Yes  & Yes  &  Yes \\
 &  &  &  &  &  & \\

$p$-value  &  &  &  &  &  & \\
\hspace{0.4cm}\textit{Control A $=$ Treatment A}  & 0.000  & 0.000  & 0.000  & 0.890  & 0.894  & 0.277 \\

Observations  & 803  & 803  & 8,030  & 8,030  & 8,030  &8,020 \\
Subjects  & 803  & 803  & 803  & 803  & 803  & 802 \\
\bottomrule
\end{tabular}
\par\end{centering}
\footnotesize
\textbf{\textit{Notes:}}
Each column corresponds to a separate regression. Deliberative Competence measured as $d_M^{i,R,t}=-\big|r_{c}^{i,R,t}-r_{s}^{i,R,t}\big|$. Column 6 omits the subject for whom $r_S^{i,R,t} = 0$ for all instruments. Standard errors in parentheses, clustered on the subject level. ***$p<0.01,$***$p<0.05,$*$p<0.1$.
\end{table}


\subsection{Effects on individual test questions}

\label{individualTestQuestions}

We analyze the effect of the treatments on answers to individual test
questions in table \ref{TestSplit}. The test questions differ by
how closely they follow the material in the education intervention,
and by how easily they are answered without knowledge of the rule
of 72.

Q1 is the only question for which the answer was explicitly given
in the education video (including in the Substance-Only treatment but not in the Rhetoric-Only treatment). The video also discussed an example that is similar, but not identical, to Q2.\footnote{The example is: ``To double your money in 10 years, what rate of return do you need? The answer: 10 times $X$ = 72, so $X$ = 7.2
percent.''}

The remaining questions require more flexible thinking. Q3 and Q4
can easily be answered with the rule of 72. Knowledge of this rule,
however, is not necessary to answer these questions correctly. Q3
can be answered by iteratively multiplying a starting value with 1.07,
and counting the number of iterations required for the amount to increase
to the desired value. Likewise, Q4 can be answered by calculating
the factor by which an investment grows within 8 years at 9 percent
interest (either iteratively, or using the compound interest formula),
and then dividing 500 by this number. Q5 is a standard compound interest
calculation, and parallels the calculations that need to be made in
the complexly framed decision problems.

Table \ref{TestSplit} displays the treatment effects on the success
rates for each of these questions. Baseline rates of correct answers are highly similar across the two experiments. Moreover, in both experiments, the significant effect of the Full and Substance-Only treatments on the total score derive
from questions Q1, Q2, and Q5. The fact that performance in Q5 increased
in these treatments is reassuring, as it demonstrates that the increase
in test scores is at least partly due to subjects' increased ability
to analyze previously unseen problems properly. Moreover, while treatment effects are similar across the experiments for questions Q1 to Q4,  the treatment effect on Q5 in Experiments B is more than double that in Experiment A, tentatively hinting at our finding that our intervention in Experiment B is more effective than that in Experiment A.


\begin{table}
\caption{\label{TestSplit}Fraction of correct responses on individual  questions in the test about the Treatment intervention. }
\setstretch{1} \centering{}


\begin{center}
\begin{tabular}{lccccc}
\toprule
Question  & Q1  & Q2  & Q3  & Q4  & Q5 \\
\midrule
&\multicolumn{5}{c}{\textbf{Experiment A} } \\
\cmidrule(r){2-6}
\emph{Treatment effects}  &  &  &  &  & \\
\hspace{0.5cm}Treatment  & 0.566*** & 0.619*** & 0.062 & 0.021 & 0.174*** \\
 & (0.054) & (0.053) & (0.068) & (0.068) & (0.067) \\
\hspace{0.5cm}Substance-Only  & 0.584*** & 0.592*** & -0.037 & 0.023 & 0.109* \\
 & (0.051) & (0.053) & (0.065) & (0.065) & (0.065) \\
\hspace{0.5cm}Rhetoric-Only  & 0.072 & 0.191*** & 0.067 & 0.114* & 0.050 \\
 & (0.065) & (0.061) & (0.067) & (0.067) & (0.067) \\
Level in Control  & 0.330*** & 0.220*** & 0.514*** & 0.422*** & 0.477*** \\
 & (0.045) & (0.040) & (0.048) & (0.047) & (0.048) \\
 &  &  &  &  & \\
Observations  & 455  & 455  & 455  & 455  & 455 \\
\cmidrule(r){2-6}
\\


&\multicolumn{5}{c}{\textbf{Experiment B} } \\
\cmidrule(r){2-6}
\emph{Treatment effect}   & 0.559*** & 0.696*** & 0.045 & -0.075 & 0.375*** \\
 & (0.042) & (0.038) & (0.054) & (0.052) & (0.048) \\
Level in Control  & 0.346*** & 0.168*** & 0.464*** & 0.436*** & 0.436*** \\
 & (0.036) & (0.028) & (0.037) & (0.037) & (0.037) \\ \\
Observations  &348 &348 &348 &348 &348  \\
\bottomrule
\end{tabular}
\end{center}
\footnotesize
\flushleft
\textbf{\textit{Notes:}}  Standard errors in parentheses. {*}{*}{*}$p<0.01$, {*}{*}$p<0.05$, {*}$p<0.1$.
\end{table}


\subsection{Self-reported behavior} \label{selfReports}

The study ends with a brief non-incentivized questionnaire. We ask
subjects whether they had used the rule of 72 in the complexly framed
problems, and whether they had used it in the simply framed problems.
We also elicit the number of complexly framed valuation tasks for
which subjects explicitly calculated the future value of the investment,
and ask whether they obtained help when taking the test on compound
interest. The questionnaire also addresses a small number of additional
issues.

Subjects in the Control condition report similar numbers of decisions
for which they engaged in explicit calculations in each of the experiments
(6.4 and 6.7 in Experiments A and B, respectively). The Treatment condition significantly increases that number, by 1.7
problems in Experiment A (column 1, $p<0.01$) and by 2.6 problems in Experiment
B (column 2, $p<0.01$). The treatment effect on the fraction of subjects reporting
to have used the rule of 72 in their decision making in complexly
framed decisions does not differ substantially across experiments (57.9\%
and 60.5\% in Experiments A and B, respectively). There is, however,
a difference in levels. Only 12.8\% of subjects in the Control condition
of Experiment A report using the rule, whereas 31.8\% of subjects
in the Control condition of Experiment B do so.


As expected, the fraction of subjects reporting to have used the rule of 72 for simply framed problems is substantially smaller; averaging 9.2\% and 22.3\% in the Control conditions of Experiments A and B, respectively. In  both experiments the Treatment condition increases the frequency of such reports, but does so almost twice as
much in Experiment A (by 17.2 percentage points) than in Experiment
B (by 9.6 percentage points). Finally, when asked about the use of
external help with the test questions at the end of the experiment,
we do not find treatment effects in either experiment, although the
fraction of subjects reporting the use of such help exceeds 20\% in
Experiment A, whereas it is lower than 8\% in Experiment B.

Unlike performance on test scores and directional behavioral changes,
these self-reported behaviors suggest that the effects of the Treatment
interventions differ across the experiments, though that interpretation is complicated by the fact that baseline levels differ across the experiments. Like the conventional
measures, however, data on self-reported behavior suggest that the Treatment interventions are effective in either experiment.


\begin{table}[htbp!]
\caption{Self-reported behavior.\label{selfReportsTable}}
\setlength{\tabcolsep}{3pt}


\begin{centering}
\begin{tabular}{lcccccccc}
\toprule
 & (1)  & (2)  & (3)  & (4)  & (5)  & (6)  & (7)  & (8) \\
Self-report & \multicolumn{2}{c}{Engages in} & \multicolumn{2}{c}{Uses of rule of 72} & \multicolumn{2}{c}{Uses of rule of 72} & \multicolumn{2}{c}{External help}\\
 & \multicolumn{2}{c}{explicit calculation} & \multicolumn{2}{c}{in complex frame} & \multicolumn{2}{c}{in simple frame} & \multicolumn{2}{c}{with test}\\
\cmidrule(r){2-3} \cmidrule(r){4-5} \cmidrule(r){6-7} \cmidrule(r){8-9}
Experiment  & A  & B  & A  & B  & A  & B  & A  & B \\
\midrule
 &  &  &  &  &  &  &  & \\
Levels  &  &  &  &  &  &  &  & \\
\hspace{0.4cm} \textit{Control}  & 6.404{*}{*}{*}  & 6.693{*}{*}{*}  & 0.128{*}{*}{*}  & 0.318{*}{*}{*}  & 0.092{*}{*}{*}  & 0.223{*}{*}{*}  & 0.220{*}{*}{*}  & 0.078{*}{*}{*} \\
 & (0.377)  & (0.277)  & (0.032)  & (0.035)  & (0.028)  & (0.031)  & (0.040)  & (0.020) \\
\hspace{0.4cm} \textit{Treatment}  & 8.142{*}{*}{*}  & 9.331{*}{*}{*}  & 0.708{*}{*}{*}  & 0.923{*}{*}{*}  & 0.264{*}{*}{*}  & 0.320{*}{*}{*}  & 0.208{*}{*}{*}  & 0.059{*}{*}{*} \\
 & (0.342)  & (0.209)  & (0.044)  & (0.021)  & (0.043)  & (0.036)  & (0.040)  & (0.018)\\
Difference  & 1.738{*}{*}{*}  & 2.639{*}{*}{*}  & 0.579{*}{*}{*}  & 0.605{*}{*}{*}  & 0.172{*}{*}{*}  & 0.096{*}{*}  & -0.013  & -0.019 \\
 & (0.509)  & (0.347)  & (0.055)  & (0.041)  & (0.051)  & (0.048)  & (0.056)  & (0.027) \\
 &  &  &  &  &  &  &  & \\
Observations  & 215  & 348  & 215  & 348  & 215  & 348  & 215  & 348 \\

\bottomrule
\end{tabular}
\par\end{centering}
\footnotesize
\textbf{\textit{Notes:}}
 Each column displays the coefficients of a separate OLS regression. Standard errors in parentheses. ***$p<0.01,$***$p<0.05,$*$p<0.1$.
\end{table}


\subsection{Analysis using the welfarist measure of Deliberative Competence}\label{resultsSquared}

Here, we present our main empirical results on Deliberative Competence using the measure $d_W^{i,R,t}=-\big(r_{c}^{i,R,t}-r_{s}^{i,R,t}\big)^2$ which approximates the average rather than the maximal loss from characterization failure.

Table \ref{finCompSq} replicates Table \ref{finComp} with this alternative measure. Panel A shows that the intervention in Experiment A leaves $d_W^{i,R,t}$ nearly unchanged on average whereas the intervention in Experiment B leads to a substantial and statistically highly significant increase both in the pooled sample and separately within each timeframe.

Panel B applies Proposition \ref{Correction} to correct for changes in valuations in the simple frame.\footnote{Proceeding as in Table \ref{finComp}, we account for noise in the elicitation of valuations in the simple frame. Specifically, we calculate the mean valuation for simply framed choices for each timeframe and use the square of the resulting average as the correction factor. Moreover, by our normalization, $y_{sI}-y_{0}=1$ for all instruments $I$.} Again, we find that the intervention in Experiment A, if anything, harms subjects, whereas the intervention in Experiment B significantly increases their welfare. The relative magnitude of these effects differs from those in Table \ref{finComp}. Here, we find that the harm caused by the intervention in Experiment A is of a similar magnitude as the benefits caused by the intervention in Experiment B, whereas in Panel B of Table \ref{finComp} the benefits of the intervention in Experiment B exceed the magnitude of the harm in Experiment A severalfold. One reason for this divergence is the stronger sensitivity of $d_W^{i,R,t}$ to large valuation differences, $r_C^{i,R,t} - r_S^{i,R,t}$.


\begin{table}
\caption{Deliberative Competence based on expected welfare loss.}
\label{finCompSq}
\setlength{\tabcolsep}{3pt}
\setstretch{1}


\begin{centering}
\begin{tabular}{lcccccc}
\multicolumn{7}{c}{\textbf{A. Deliberative Competence: Welfarist Measure}}\\
\midrule
 & (1)  & (2)  & (3)  & (4)  & (5)  & (6) \\
Delay in days  & 72 and 36  & 72 and 36  & 72  & 72  & 36  & 36 \\
\cmidrule(r){2-3} \cmidrule(r){4-5} \cmidrule(r){6-7} Experiment  & A  & B  & A  & B  & A  & B \\
\midrule
 &  &  &  &  &  & \\
Levels \\
 \quad \textit{Control} & -31.635*** & -34.235*** & -30.287*** & -33.938*** & -32.983*** & -34.531*** \\
 & (2.329) & (2.064) & (2.462) & (2.199) & (2.504) & (2.116) \\
\quad \textit{Treatment}  & -34.017*** & -28.382*** & -33.157*** & -26.907*** & -34.877*** & -29.858*** \\
 & (3.110) & (2.067) & (3.192) & (2.050) & (3.471) & (2.297) \\
\quad \textit{Substance-Only}  & -29.025*** &  & -27.255*** &  & -30.795*** &  \\
 & (2.174) &  & (2.190) &  & (2.398) &  \\
\quad \textit{Rhetoric-Only} & -31.431*** &  & -31.640*** &  & -31.223*** &  \\
 & (2.763) &  & (2.907) &  & (2.877) &  \\
 &  &  &  &  &  &  \\
$p$-value of difference \\
to Control \\
\quad \textit{Treatment}   & 0.540 & 0.046 & 0.477 & 0.020 & 0.658 & 0.136 \\
\quad \textit{Substance-Only} & 0.413 &  & 0.358 &  & 0.528 &  \\
 \quad \textit{Rhetoric-Only}  & 0.955 &  & 0.723 &  & 0.645 &  \\
Observations & 4,550 & 3,480 & 2,275 & 1,740 & 2,275 & 1,740 \\
Subjects & 455 & 348 & 455 & 348 & 455 & 348 \\
\midrule
 &  &  &  &  &  & \\
\multicolumn{7}{c}{\textbf{B. Deliberative Competence: Welfarist Measure}} \\
\multicolumn{7}{c}{\textbf{corrected for changes in simply framed valuations}}\\
\midrule
 & (1)  & (2)  & (3)  & (4)  & (5)  & (6) \\

Delay in days  & 72 and 36  & 72 and 36  & 72  & 72  & 36  & 36 \\
\cmidrule(r){2-3} \cmidrule(r){4-5} \cmidrule(r){6-7} Experiment  & A  & B  & A  & B  & A  & B \\
\midrule
 &  &  &  &  &  & \\
Levels \\
 \quad \textit{Control} & -63.503*** & -74.623*** & -68.950*** & -79.013*** & -58.055*** & -70.234*** \\
 & (5.603) & (6.236) & (8.005) & (7.043) & (4.839) & (7.107) \\
\quad \textit{Treatment}  & -97.289*** & -50.961*** & -97.543*** & -52.627*** & -97.035*** & -49.295*** \\
 & (16.557) & (4.683) & (16.502) & (5.118) & (19.657) & (4.651) \\
\quad \textit{Substance-Only}  & -114.092*** &  & -149.662** &  & -78.521*** &  \\
 & (40.816) &  & (67.113) &  & (15.593) &  \\
\quad \textit{Rhetoric-Only} & -66.779*** &  & -75.521*** &  & -58.036*** &  \\
 & (7.662) &  & (9.977) &  & (6.560) &  \\
 &  &  &  &  &  &  \\
$p$-value of difference \\
to Control \\
\quad \textit{Treatment}   & 0.054 & 0.003 & 0.120 & 0.003 & 0.055 & 0.014 \\
\quad \textit{Substance-Only} & 0.220 &  & 0.233 &  & 0.211 &  \\
 \quad \textit{Rhetoric-Only}  & 0.730 &  & 0.608 &  & 0.998 &  \\
Observations & 4,550 & 3,470 & 2,275 & 1,735 & 2,275 & 1,735 \\
Subjects & 455 & 347 & 455 & 347 & 455 & 347 \\

\bottomrule
\end{tabular}
\par\end{centering}
\footnotesize
\textbf{\textit{Notes:}}  Each column displays the coefficients of
a separate OLS regression of the welfarist measure of Deliberative Competence, $d_W^{i,R,t}=-\big(r_{c}^{i,R,t}-r_{s}^{i,R,t}\big)^2$, on treatment indicators. Standard errors in parentheses, clustered
by subject. The reason for the smaller number of observations in Panel
B in Experiment B is one subject who consistently made choices consistent
with a valuation of zero in the simple frame. As the correction consists
in dividing by simply framed valuations, this subject is excluded
from that analysis. {*}{*}{*}$p<0.01,${*}{*}{*}$p<0.05,${*}$p<0.1$.
\end{table}


Table \ref{finCompAggregationSq} replicates Table \ref{finCompAggregation} using using the welfarist measure of Deliberative Competence, $d_W^{i,R,t}=-\big(r_{c}^{i,R,t}-r_{s}^{i,R,t}\big)^2$. As in Table \ref{finCompAggregation}, we find that the intervention in Experiment A harms subjects in the lowest quartile of valuations in the simple frame ($p<0.1$), and has beneficial effects for subjects in the highest quartile ($p<0.01$). Also paralleling the result in Table \ref{finCompAggregation}, the intervention in Experiment B does not harm subjects in any quartile, but has substantially positive effects for subjects in the second-highest ($p<0.1$) and highest quartiles ($p<0.01$). We conclude that our inferences regarding practice and feedback are not driven by an unintended relationship between welfare weights and biases that impact simply framed valuations, also based on the welfarist measure.


\begin{table}[htbp!]
\caption{Welfarist measure of Deliberative Competence by quartiles of simply framed valuations.}
\label{finCompAggregationSq}
\begin{centering}

\begin{tabular}{lcccc}
\multicolumn{5}{c}{\textbf{Experiment A}}\\
\toprule
VARIABLE & \multicolumn{4}{c}{Deliberative Competence}\\
\midrule
 & \multicolumn{4}{c}{Quartile simply framed valuation}\\
\cmidrule(r){2-5}  & 1  & 2  & 3  & 4 \\
Levels  &  &  &  & \\
\hspace{0.4cm}\textit{Control}   & -17.117***  & -27.762***  & -34.273***  & -46.826*** \\
 & (2.880)  & (4.659)  & (3.696)  & (5.064) \\
\hspace{0.4cm}\textit{Treatment}     & -33.360 & -32.323 & -33.912 & -36.446 \\
 & (8.733) & (5.078) & (3.533) & (6.432) \\
Effect & -16.243*  & -4.561  & 0.361  & 10.380 \\
 & (9.196)  & (6.891)  & (5.114)  & (8.187) \\
\cmidrule(r){2-5} Observations  & \multicolumn{4}{c}{2,150}\\
Subjects  & \multicolumn{4}{c}{215}\\
\bottomrule
 &  &  &  & \\
\multicolumn{5}{c}{\textbf{Experiment B}}\\
\toprule
VARIABLE & \multicolumn{4}{c}{Deliberative Competence}\\
\midrule
 & \multicolumn{4}{c}{Quartile simply framed valuation}\\
\cmidrule(r){2-5}  & 1  & 2  & 3  & 4 \\
Levels  &  &  &  & \\
\hspace{0.4cm}\textit{Control}   & -16.939***  & -33.560***  & -35.883***  & -50.172*** \\
 & (2.719)  & (4.078)  & (3.257)  & (4.574) \\
\hspace{0.4cm}\textit{Treatment}   & -20.018  & -30.052  & -30.076  & -33.268 \\
 & (3.634)  & (3.306)  & (3.922)  & (5.111) \\
Effect   & -3.079  & 3.508 & 5.807 & 16.905** \\
 & (4.539)  & (5.250)  & (5.099) & (6.859) \\
\cmidrule(r){2-5} Observations  & \multicolumn{4}{c}{3,480}\\
Subjects  & \multicolumn{4}{c}{348}\\
\bottomrule
\end{tabular}
\par\end{centering}
\footnotesize
\textbf{\textit{Notes:}} Effect of treatments on the welfarist measure of Deliberative Competence, $d_W^{i,R,t}=-\big(r_{c}^{i,R,t}-r_{s}^{i,R,t}\big)^2$,
by quartiles of simply framed valuations. Each panel presents the
output of a single OLS regression. Standard errors in parentheses,
clustered by subject. {*}{*}{*}$p<0.01,${*}{*}{*}$p<0.05,${*}$p<0.1$.
\end{table}


\subsection{Deliberative Competence with correction for policy-induced confounds based on the approach of Appendix  \ref{MoreTheory}}\label{resultsCorrectionAlt}


In this section, we provide estimates of Deliberative Competence corrected for policy-induced framing effects using the alternative approximation approach developed in Section \ref{MoreTheory}.

We estimate the correction factor $\rho$ by regressing valuations in the simple frame on the future amount the subject can receive from purchasing the instrument. We run a separate OLS regression for each timeframe and for each treatment within each Experiment.  The slope coefficient is a consistent estimate for $\rho^{V}\left(y_{s},\theta\right)$; in each case, we divide the absolute valuation difference by this estimate.

Panel A of Table \ref{correctionAltApprox} applies this correction factor to replicate Table \ref{finComp}. Panel B performs parallel analysis using the welfarist measure $d_W$ and the corresponding correction factor $\rho^2$. In each case we find large and statistically highly significant treatment effects for Experiment B but not for Experiment A.


\begin{table}[htbp!]
\setlength{\tabcolsep}{3pt}


\begin{centering}
\caption{Deliberative Competence corrected for policy-induced confounds based on the approach of Appendix \ref{MoreTheory} \label{correctionAltApprox} }
\begin{tabular}{lcccccc}
\multicolumn{7}{c}{A. Deliberative Competence, $d_M$} \\
\toprule
 & (1)  & (2)  & (3)  & (4)  & (5)  & (6) \\
Delay in days  & 72 and 36  & 72 and 36  & 72  & 72  & 36  & 36 \\
\cmidrule(r){2-3} \cmidrule(r){4-5} \cmidrule(r){6-7} Experiment  & A  & B  & A  & B  & A  & B \\
\midrule
 &  &  &  &  &  & \\
Levels \\
 \quad \textit{Control} & -36.875*** & -38.517*** & -39.765*** & -40.982*** & -33.985*** & -36.052*** \\
 & (2.472) & (2.036) & (2.880) & (2.255) & (2.304) & (1.978) \\
\quad \textit{Treatment} & -31.591*** & -26.576*** & -33.291*** & -27.800*** & -29.891*** & -25.353*** \\
 & (2.402) & (1.684) & (2.609) & (1.821) & (2.476) & (1.699) \\
\quad \textit{Substance-Only} & -30.356*** &  & -30.824*** &  & -29.888*** &  \\
 & (1.972) &  & (2.062) &  & (2.127) &  \\
\quad \textit{Rhetoric-Only} & -27.375*** &  & -29.278*** &  & -25.471*** &  \\
 & (1.941) &  & (2.115) &  & (1.977) &  \\
 &  &  &  &  &  &  \\
$p$-value of difference \\
to Control \\
\quad \textit{Treatment} & 0.126 & 0.000 & 0.096 & 0.000 & 0.227 & 0.000 \\
\quad \textit{Substance-Only} & 0.040 &  & 0.012 &  & 0.192 &  \\
 \quad \textit{Rhetoric-Only}  & 0.003 &  & 0.004 &  & 0.005 &  \\
Observations & 4,550 & 3,480 & 2,275 & 1,740 & 2,275 & 1,740 \\
Subjects & 455 & 348 & 455 & 348 & 455 & 348 \\\bottomrule \\

\multicolumn{7}{c}{B. Deliberative Competence, $d_W$} \\
\toprule
 & (1)  & (2)  & (3)  & (4)  & (5)  & (6) \\
Delay in days  & 72 and 36  & 72 and 36  & 72  & 72  & 36  & 36 \\
\cmidrule(r){2-3} \cmidrule(r){4-5} \cmidrule(r){6-7} Experiment  & A  & B  & A  & B  & A  & B \\
\midrule
 &  &  &  &  &  & \\
Levels \\
 \quad \textit{Control} & -72.230*** & -77.119*** & -82.620*** & -84.680*** & -61.840*** & -69.557*** \\
 & (5.363) & (4.666) & (6.715) & (5.487) & (4.695) & (4.263) \\
\quad \textit{Treatment} & -64.912*** & -58.021*** & -69.717*** & -61.145*** & -60.106*** & -54.898*** \\
 & (5.925) & (4.223) & (6.712) & (4.658) & (5.982) & (4.224) \\
\quad \textit{Substance-Only}  & -55.189*** &  & -57.307*** &  & -53.071*** &  \\
 & (4.140) &  & (4.605) &  & (4.133) &  \\
\quad \textit{Rhetoric-Only} & -60.167*** &  & -66.526*** &  & -53.808*** &  \\
 & (5.291) &  & (6.112) &  & (4.959) &  \\
 &  &  &  &  &  &  \\
$p$-value of difference \\
to Control \\
\quad \textit{Treatment} & 0.360 & 0.003 & 0.175 & 0.001 & 0.820 & 0.015 \\
\quad \textit{Substance-Only} & 0.012 &  & 0.002 &  & 0.162 &  \\
 \quad \textit{Rhetoric-Only}  & 0.110 &  & 0.077 &  & 0.240 &  \\
Observations & 4,550 & 3,480 & 2,275 & 1,740 & 2,275 & 1,740 \\
Subjects & 455 & 348 & 455 & 348 & 455 & 348 \\\bottomrule

\end{tabular}
\par\end{centering}
\footnotesize
\textbf{\textit{Notes:}} Each column displays the coefficients of a separate OLS regression of Deliberative Competence, on treatment indicators. Deliberative Competence is corrected for the change in simply framed valuations using the approximation strategy developed in Section  \ref{MoreTheory}. Panel A uses the maximal loss measure of Deliberative Competence, $d_M$; Panel B uses the welfarist measure $d_W$.
\end{table}

\FloatBarrier


\subsection{Valuation difference compared to noise in the simple frame}

\label{simpleNoise}


\begin{figure}[htp]
\caption{Replication of CDFs from Figure \ref{CDF} with a measure of the distribution of noise in the simple frame superimposed. \label{CDFsAppendix}}
\begin{centering}
\begin{tabular}{cc}
\includegraphics[width=0.5\textwidth]{graphs/cdfOldWithSimple}  & \includegraphics[width=0.5\textwidth]{graphs/cdfNewWithSimple} \\
Experiment A  & Experiment B \\
\end{tabular}
\par\end{centering}
\end{figure}

Here we investigate the possibility that the measured overestimation of compound
interest exhibited by a fraction of our subjects is solely attributable
to elicitation noise. To test the hypothesis, we estimate the amount of noise in simply framed decisions. We then compare the frequency of excessive valuations we would observe based on that amount of noise alone to the frequency of excessive valuations we actually observe in the complex frame.  Specifically, we calculate, for each subject and each timeframe $t\in \{36, 72\}$, the values $\Delta_1^t = r^{j,20,t}_{S}-r^{j,18,t}_{S}$,
$\Delta_2^t = r^{j,18,t}_{S}-r^{j,20,t}_{S}$
$\Delta_3^t = r^{j,16,t}_{S}-r^{j,14,t}_{S}$,
and $\Delta_4^t = r^{j,14,t}_{S}-r^{j,16,t}_{S}$ (recall that valuations $r^{i,R,t}_S$ involve the normalization of the future value to \$1).  Figure \ref{CDFsAppendix} superimposes the CDF of $\Delta_k^t$ (pooled across $k\in\{1, .., 4\}$ and $t\in\{36, 72\}$) on the CDF of the valuation difference from Figure \ref{CDF}. If excessive valuations in the complex frame were solely attributable to elicitation noise, the CDF of $\Delta_k^t$ should coincide with the CDF of $r^{i,R,t}_C$ to the left of zero. By contrast, we find substantial differences between these curves in each experiment, and especially for the treatment in Experiment A. Accordingly, elicitation noise alone cannot explain the overestimation of compound interest in
either condition of either experiment.


\FloatBarrier


\newpage{}

\section{Instructions}

\label{Instructions} \vspace{1cm}
 \includegraphics[width=1\textwidth]{InstructionsScreenshots/AMT_HIT}

\includepdf[pages=1 - 1]{InstructionsScreenshots/1} \includepdf[pages=1 - 1]{InstructionsScreenshots/welcomeVideo}
\includepdf[pages=1 - 2]{InstructionsScreenshots/demographics} \includepdf[pages=1 - 1]{InstructionsScreenshots/FL}
\includepdf[pages=1 - 1]{InstructionsScreenshots/Instr2} \includepdf[pages=1 - 1]{InstructionsScreenshots/InstrTreatmentVideo}
\includepdf[pages=1 - 1]{InstructionsScreenshots/Instr10} \includepdf[pages=1 - 3]{InstructionsScreenshots/Instr11}
\includepdf[pages=1 - 2]{InstructionsScreenshots/Instr12} \includepdf[pages=1 - 1]{InstructionsScreenshots/MakeDecisions}
\includepdf[pages=1 - 1]{InstructionsScreenshots/Decisions1} \includepdf[pages=1 - 1]{InstructionsScreenshots/Decisions2}
\includepdf[pages=1 - 1]{InstructionsScreenshots/instrTest} \includepdf[pages=1 - 2]{InstructionsScreenshots/Test}
\includepdf[pages=1 - 1]{InstructionsScreenshots/exitSurvey}

\section{Practice problems with personalized feedback}

\label{practiceProblems}
 \textbf{\textit{Education
Intervention Part 1: }}\textit{\url{https://www.youtube.com/watch?v=EnFVLiM1dTs}}
\\
 \textbf{\textit{Part 1 Practice Question}} \\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E2}\\
 \textbf{\textit{If the answer is correct in the first trial:}} \\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E3b}\\
 \newpage
 \textbf{\textit{If the answer is incorrect in the first trial:}}
\\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E3a}\\
 \textbf{\textit{If the answer is correct in the second time:}} \\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E4b}\\
 \newpage
 \textbf{\textit{If the answer is incorrect in the second trial:}}
\\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E4a}\\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E5}\\
 \textbf{\textit{Education Intervention Part 2:
}}\textit{\url{https://youtu.be/3pjkVdOXMlk}} \\
\newpage
 \textbf{\textit{Part 2 Practice Questions}} \\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E6}\\
 \textbf{\textit{If the answer is correct in the first trial:}} \\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E7hE9a}\\
 \newpage\noindent\textbf{\textit{If the answer is incorrect in the first
trial:}} \\
 \textit{Subjects see one of the following explanations depending
on their previous answer and they re-attempt the question.} \\
 \textbf{\textit{If the subject selected answer \$100:}} \\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E7a}\\
 \newpage
 \textbf{\textit{If the subject selected answer \$200:}} \\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E7b}\\
\newpage
 \textbf{\textit{If the subject selected answer \$388:}} \\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E7c}\\
\newpage
 \textbf{\textit{If the subject selected answer \$400:}} \\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E7d}\\
 \newpage
 \textbf{\textit{If the subject selected answer \$600:}} \\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E7e}\\
 \newpage
 \textbf{\textit{If the subject selected answer \$800:}} \\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E7f}\\
 \newpage
 \textbf{\textit{If the subject selected answer \$1200:}} \\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E7g}\\
 \newpage
 \textbf{\textit{Re-attempt the question:}} \\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E8}\\
 \textbf{\textit{If the answer is correct in the second time:}} \\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E7hE9a}\\
 \newpage
 \textbf{\textit{If the answer is incorrect in the second trial:}}
\\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E9b}\\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E10}\\
 \textit{\textbf{Education Intervention, part 3: }}\url{https://youtu.be/kjPYqcZNzPI}\\
\newpage
 \textbf{\textit{Practice Questions at the end of the Intervention}}
\\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E11}\\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E12}\\
 \newpage
 \textbf{\textit{If the answer to part (a) is incorrect in the first
trial:}} \\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E13a}\\
 \newpage
 \textbf{\textit{If the answer to part (a) is incorrect in the second
trial:}} \\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E14}\\
 \newpage
 \textbf{\textit{If the answer to part (a) is correct in the first
trial or later trials:}} \\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E13bE15}\\
 \newpage
 \textbf{\textit{If the answer to part (b) is incorrect in the first
trial:}} \\ \includegraphics[width=1\textwidth]{screenshotsBLB/E16a}\\
\newpage
 \textbf{\textit{If the answer to part (b) is correct in the first
trial or later trials:}} \\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E16b}\\
 \newpage
 \textbf{\textit{If the answer to part (c) is incorrect in the first
trial:}} \\ \includegraphics[width=1\textwidth]{screenshotsBLB/E18}\\
\newpage
 \textbf{\textit{If the answer to part (c) is correct in the first
trial or later trials:}} \\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E19}\\
 \newpage
 \textbf{\textit{If the answer to Question 2 is incorrect in the
first trial:}} \\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E20}\\
 \newpage
 \textbf{\textit{If the answer to Question 2 is correct in the first
trial or later trials:}} \\
 \includegraphics[width=1\textwidth]{screenshotsBLB/E21}\\


\end{appendix}
\end{document}
