We all make bad decisions sometimes. The role of behavioral scientists is to understand why we make these poor choices and develop policies to help us make better ones.
With a Nobel Prize for Richard Thaler’s work on individual decision-making and runaway bestseller status for books like Daniel Kahneman’s Thinking, Fast and Slow, behavioral science has garnered wide and growing attention. More than 200 government teams around the world now work on applying behavioral science to develop, test, and scale new interventions. The work of these “Nudge Units” demonstrates that even small, inexpensive changes in how information is conveyed can induce large changes in behavior.
The current evidence suggests that our brains are susceptible to overreacting to present temptations. Making it easier or more attractive to choose actions associated with longer-term benefits can help realize those benefits. Another behavioral barrier is that our brains tend to rely too much on routine or on what’s top of mind. Having access to wise advice or salient alternative options can also help. Stress, uncertainty, complexity, and social influences exacerbate the biases that lead to bad choices.
Education was one of the last areas of public policy to receive attention from behavioral scientists. That is surprising given youth’s predisposition to instant gratification, in tension with the important long-term impacts of their education-related decisions.
The first behavioral experiment in education may have been conducted by Justine Hastings and Jeff Weinstein. They examined the impact of mailing families in North Carolina a list of possible schools children could attend along with corresponding test-score performance information. They compared that to allowing the families to access the information independently by internet. The simplified and more salient information led to a significant increase in the number of families who applied to schools outside their catchment area and an increase in test scores among children who moved.
Around the same time, Susan Dynarski and Judith Scott-Clayton pointed out the complexity of the U.S. Free Application for Federal Student Aid and suggested that the many hurdles required to complete the application may significantly slow the application process, or even prevent some from applying at all. Eric Bettinger, Bridget Long, Lisa Sanbonmatsu, and I put this theory to the test by teaming up with H&R Block, a large tax preparation company. After helping low-income clients complete their annual returns, tax professionals invited those who were potentially interested in college to remain for a few minutes to participate in a study (and receive $20). One group received a general informational brochure about college, a second group received a personalized report of eligible federal grant and loan aid against tuition costs of nearby colleges, and a third group received the brochure and the report, plus assistance in completing the FAFSA for themselves or for their children about to graduate from high school. Much of the data needed to complete the form was already collected from completing the tax return, so the process to complete the FAFSA took only about an additional 10 minutes. While the information treatment had no impact, the personalized assistance increased FAFSA filing and college enrollment the following year. For the high school sample, enrollment increased by 8 percentage points, to 42 percent from 34 percent.
Another notable early nudging success was achieved by Benjamin Castleman and Lindsay Page, who showed that simply sending a series of text message reminders of key tasks to complete over the summer to recent high school graduates and their parents can help keep college-accepted youth on track to begin their program in the fall. College enrollment was 4 to 7 percentage points higher for students who received the text messages, relative to a randomized control group who did not.
The field of behavioral science applied to education has since exploded, with efforts to improve not only college application rates, enrollment, and completion, but many other outcomes such as class attendance, field of study, school breakfast, school choice (choosing two-year versus four-year colleges, for example, or more selective schools), scholarship receipt, on-time graduation, grade point average, study effort, study time, student attendance at faculty office-hours or take-up of other services, loan repayment, and earlier childhood outcomes such as literacy, numeracy, and executive functioning. However, the results have not always been encouraging. Many recent studies of nudges with large samples that stick to a pre-analysis plan or attempt to scale earlier interventions find tiny or no effects. Some areas of focus, such as nudging parents, show more promise, whereas others, like trying to improve test scores or adopt better learning habits, show less.
This article takes stock of where the field of behavioral science applied to education policy seems to be at, which avenues seem promising and which ones seem like dead ends. I present below a curated set of studies rather than an exhaustive literature review, categorizing interventions by whether they “nudge” (keep options intact) or “shove” (restrict choice), and whether they apply a “high touch” or a “low touch” (whether they use face-to-face interaction or not). I argue that we should continue to make administrative processes in education easier, information more salient, and communication more friendly. The cost for many low-touch nudges, such as changing the content of a letter or sending an email reminder, are small enough to merit doing even if the impact might be zero. In cases where financial and nuisance costs matter more, replication studies and iterating over what works best (a process sometimes referred to as “A/B testing”) can further help decide what interventions are worth scaling. But we should not expect this kind of tinkering to serve as a panacea for education policy’s key challenges. The current evidence suggests that we could make better progress by adding more choice-limiting scaffolding to a youth’s routine, like restricted screen time and mandated tutoring, and by focusing on children at younger ages, when preferences and behavioral traits are more malleable.
Nudging and Shoving, High-Touch and Low-Touch
It helps to think about behavioral interventions based on whether they nudge or shove and whether they use a high touch or a low touch.
A nudge is a subtle adjustment to an individual’s environment to steer the person towards a more desirable outcome while not meaningfully altering options or costs. The underlying principle for nudging is to “make it easy.” Defaults—automatically selecting individuals into one choice option if no action is taken—are among the most influential ways to nudge. For example, opting individuals into organ donation programs and employer retirement savings programs, with the option to opt out, has been shown to dramatically increase take-up. Changing the default, however, is not always possible or practical for the outcome of interest. For example, defaulting high school seniors into being enrolled in college would be administratively complex, require guessing which school and program would be best, and offer no guarantee that the students would show up to campus. More common nudges use marketing techniques, such as simplifying take-up procedures, sending reminders, or providing information through text, email, signs, or phone calls.
Unlike a nudge, a shove restricts an individual’s set of options to steer the person towards more desirable outcomes. Requiring workers to participate in a government retirement benefits program by taxing them is a type of shove because no opt-out option exists. Banning large containers of soda is a shove. Requiring students to attend school is also a shove. Restricting choice can occur more indirectly from deciding how to structure an individual’s schedule—especially a child’s schedule. For example, I consider the act of parents planning their child’s weekend a shove. Teachers who decide what and how to teach also restrict how students spend their time.
How can parents, teachers, and policymakers know which behaviors are more desirable? They cannot. It is impossible to know for sure whether an individual’s own inclination stems from a behavioral mistake or from carefully weighing long-term costs and benefits. At least with nudges, individuals are still free to choose, an argument originally made by Richard Thaler and Cass Sunstein. Nevertheless, nudges and shoves both aim to alter behavior. The choice architect must explicitly or implicitly decide in which direction to steer to try to make individuals better off. The consequences of steering in the wrong direction, and how many people might fall into this category, should be taken into account.
Take the case of going to college: we can’t be sure everyone benefits. Indeed, many who drop out probably don’t. On the other hand, we think some high school students, especially those from more disadvantaged backgrounds who receive less support from parents and schools, miss out on college and its benefits because of application barriers. Evidence suggests that making the application process easier can cause more people to attend. Should we support a scale-up of this effort, or deliberately maintain the status quo? It’s a normative question, because neither situation makes everyone better off. A starting point might be to estimate completion effects or predict even longer-term effects. Shoving by making college compulsory does not seem prudent, since it would likely result in many students being unable to complete even their first year. The H&R Block FAFSA nudge found an 8-percentage-point increase in both first- and second-year college enrollment for a sample of high school seniors. A follow-up study, however, estimated that the treatment increased degree completion by only about 4 percentage points, suggesting about half of those nudged into college finished, but half didn’t. The dropout rate was the same compared to the control group, reminding us it may be unrealistic to expect that everyone who is nudged will go on to graduate. Still, a tradeoff exists, and we should be aware of our implicit assumptions about who we’re helping and who we’re not when nudging or shoving.
Behavioral interventions also differ importantly by whether they are high-touch or low-touch. Researchers sometimes distinguish these two cases based on cost. A more useful, but related, distinction is whether the intervention involves in-person interaction or not. Texting students to remind them to complete the FAFSA is a low-touch nudge. Meeting with them to provide more direct assistance is a high-touch nudge. Encouragement is often more effective if delivered in person than through signage, text, or email. Someone trying to steer another person towards a particular action can express empathy, respond to questions, and use body language or facial expressions. A person can even be the intervention by providing guidance or advice (as a coach, caseworker, or parent, for example). High-touch nudges make it easier to receive in-person interaction. High-touch shoves make them mandatory. Requiring students to meet with a guidance counselor is an example of a high-touch shove. These kinds of interventions are expensive, and their success likely depends on the quality and frequency of the in-person interaction. Not all researchers would consider these more intensive programs nudges, though I think they should, because, as with low-touch nudges, the programs also aim to lower behavioral barriers and influence individuals towards more desirable behavior.
Low-Touch Nudges
For years, corporations have been using low-touch nudges to influence consumers. Governments and nonprofits have since embraced many of these approaches to nudge “for good.” Hundreds of nudge experiments have now been conducted, allowing researchers to take a step back and consider their overall success. A meta-analysis by Stefano DellaVigna and Elizabeth Linos of all 126 low-touch nudge experiments (except defaults), covering two of the largest government “Nudge Units” in the United States, found an average impact on program take-up of 1.4 percentage points and a median impact of 0.5 percentage points. This average impact represents an increase of only 8.1 percent over the share of the control group that exhibited the nudged behavior, which was 17.2 percentage points on average. Most of the interventions, therefore, generated only small or no effects. The study also documents severe publication bias among university-based experiments, with only 10 percent of experiments with insignificant effects getting reported in academic journals. The true number of studies that estimate no impact from nudging efforts may therefore be considerably larger than what gets reported, perhaps because editors are less likely to accept such studies or researchers are less excited about documenting a failed effort to nudge.
In terms of low-touch nudges in education policy specifically, overall effectiveness appears to be on par with what DellaVigna and Linos find for behavioral interventions across all policy areas. Some education nudges lead to small but cost-effective achievement gains (at least in the short term), while others generate precisely estimated null effects. Understanding when and under which circumstances low-touch nudges work may be the next frontier of this research.
A popular low-touch nudge in education is trying to increase college enrollment and persistence. A recent study finds that using an artificially intelligent text-message chatbot to support incoming undergraduates proactively increased on-time enrollment by 3.3 percentage points. Leveraging technology may therefore be a promising avenue for offering more personalization while keeping costs low.
Encouraging college students to maximize financial aid, including taking out loans, may also be a promising initiative. One study found that randomly including loan offers when sending grant-aid award letters increased borrowing, subsequent GPA, credits completed, and transfers to four-year public colleges. Conversely, a text-message outreach campaign to college student-loan applicants about both costs and benefits of loans reduced borrowing, led to worse academic achievement, and lowered persistence. Longer-term research is needed to explore impacts of these kinds of nudges on persistence without graduation, repayment, and eventual labor-market outcomes.
Another set of recent interventions tries to nudge graduating high school students from low-income backgrounds into more selective colleges with higher graduation rates. Susan Dynarski and her colleagues identified a sample of these students with high enough SAT scores to qualify for likely admission into the University of Michigan. The authors arranged for the university to mail a random subset of these students personally addressed packages promising free tuition if accepted (which most would have qualified for anyway) and a cover letter from the president encouraging them to apply. Letters and emails were also sent to the students’ parents and principals. Compared to the control group, who received only postcards with application deadlines, the application, enrollment, and persistence rates of students in the experimental group more than doubled.
In a related study, tens of thousands of high-achieving, low-income high school students across the United States were mailed packages with information and encouragement to consider selective in- and out-of-state colleges, along with waivers to apply to selective institutions without paying application fees. Enrollment in selective institutions increased to 34 percent from 29 percent. However, in an attempt to scale up this promising study, researchers at the College Board randomized 785,000 graduating high school students from low- to middle-income backgrounds, sending some carefully constructed personalized packages with easy-to-read information on a set of personalized “safety,” “match,” and “reach” colleges, along with simplified cost information and encouragement to apply. A subset also received text messages and was offered phone-based college advising. More than one third of treated students viewed specific materials provided for them on the College Board’s website, but none of the various treatments generated significant average effects on enrollment or measures of college quality.
Many other recent attempts to test large-scale nudges found precisely estimated null effects. One of them randomized 800,000 students that registered for an online account with either the Common Application or a large state-sponsored portal for applying to college. The study examined several efforts to encourage early or any FAFSA completion, including by email, text message, and mail, varying the frequency, timing, and presentation of the messages. None of the interventions increased financial aid receipt, college enrollment, or persistence. In another attempt, researchers emailed information about tax credits for college tuition payments to more than one million students who had accessed Texas’ main website for applying to a public university or community college. They varied whether the outreach discussed costs of college, benefits of college, or neither; the number of tax benefits described; and the amount of detail about the benefits and how to claim them. None of the emails affected college enrollment or reenrollment.
Social-Psychology Nudges
In addition to targeting specific external actions, such as completing a college application, behavioral scientists also try to nudge internal feelings and beliefs to improve longer-term education outcomes. In one study, reading a story to preschoolers about a character who struggled with waiting but eventually found it energizing increased children’s own ability to wait longer for a larger candy reward. According to another study, assigning middle schoolers to think and write about their core values during a series of 15-minute exercises increased academic achievement for minority students and their later college enrollment by more than 10 percentage points(!). A third example found that a detailed two-hour goal-setting exercise increased struggling college students’ GPAs by 0.8 points, more than half of a standard deviation. The sizes of these effects are off the charts, even compared to programs that cost thousands of dollars per student, but they are estimated using small samples.
If administering a couple of exercises lasting less than a few hours can consistently generate even small long-term impacts, we shouldn’t hold back from providing them to every student in the country. The time and monetary costs are small. The likelihood of causing long-term harm seems small, as well. We could even iterate over time on what works best, testing each intervention against a newer edition to boost benefits and experience over time.
That was generally the idea when I created the Student Achievement Lab with Uros Petronijevic in 2014. For six years, partnering with several other colleagues, we teamed up with first-year college instructors across six campuses to make a mandatory short online warm-up exercise that almost every student completed for a participation grade. We started by creating a goal-setting exercise similar to the one mentioned above and added a second treatment, sending additional motivational text messages throughout the year. We found no effects. We worked with leading social psychologists to develop interventions to help students develop more positive perspectives on facing challenging course material (a growth mindset) and more patient views on assimilating into campus socially (a belonging mindset). We found no effects. We created online exercises to encourage better study habits, more study time, more use of student services and office hours, and a healthier, more patient attitude towards school. We also added one- and two-way text-message coaching. With a combined sample of more than 25,000 students, none of the interventions generated significant improvement in student grades or persistence, whether for the full sample or for sub-samples of students more at risk of poor performance.
We started the Student Achievement Lab hoping to provide evidence towards supporting at least some nudges worth scaling. Not being able to recommend continuing any of the 15 warm-up exercises we tested was disappointing. Based on the inspiring open-ended responses we received from students, and the positive coach-student interactions we observed, we felt at the time that the interventions were working well. They did improve students’ sense of support from the university and proxy measures of mental health a little, but they had no detectable impact on academics. Nudging to affect habits, feelings, thoughts, and beliefs may therefore be less effective than nudging specific one-time actions. Lindsay Page, Jeonghyun Lee, and Hunter Gelbach, who tested a text-message support program for undergraduates, came to the same conclusion. Their chatbot was effective when targeting specific, time-sensitive actions, like adding or dropping a course by the deadlines, but it had no significant impact on use of student services, credit hours, GPA, or graduation.
Efforts to replicate or scale social psychology nudges aimed at changing attitudes, perspectives, or motivation often lead to inconsistent results, evidence of publication bias, or smaller effects than earlier studies with smaller samples. Two recent studies deserve particular attention for testing interventions that could be implemented at a national level. René Kizilcec and colleagues attempted to scale several short, online behavioral interventions similar to those above for more than 250,000 students taking Massive Open Online Courses through Harvard, MIT, and Stanford (for example, asking students to make concrete plans about when and how they will complete coursework, write about their core values and how taking the course reflects and reinforces these values, and reflect on the benefits and barriers to achieving their goals). Although earlier studies with smaller samples showed some of these interventions to have initial promise, none of them were found to have significant persistence effects for this larger sample. In the National Study of Learning Mindsets, thousands of 9th-grade students across 63 high schools were randomly assigned a short growth-mindset intervention. The average student’s GPA increased 0.05 points, and the course failure rate fell by 2.4 percentage points, with effects concentrated among students from the lower half of preprogram academic scores. While these impacts are small, they are not zero. Given the program’s trivial marginal cost, the exercise may be worth offering to all 9th-grade students if benefits persist.
Parent Nudges
Low-touch nudges to parents appear more consistently effective than ones to children. Parents seem to welcome the help. Texting parents suggestions for ways to interact more with children improves early literacy. Providing low-income parents of preschool students tablets with stories to take home, setting weekly reading goals, and sending reminders to the parents doubled the amount of reading time spent on the tablet. Sending middle- or high-school parents automated text messages about their children’s missed assignments, grades, and class absences reduced course failure, increased class attendance, and increased retention. Interestingly, inviting parents to opt in to receiving notices did not lead to large enough take-up to generate the same degree of impact. School administrators need to make the messages opt-out. When they are opt-out, parent text-message information campaigns generate significant gains in education attainment. Social-psychology nudges to parents also look encouraging. In one study, parents were told about the malleability of their child’s reading abilities and how to support their child by praising effort rather than performance. Second grade language skills improved two and seven months after parents completed the activity.
High-Touch Nudges
Nudging with in-person support also shows more promise than nudging without it. In the H&R Block FAFSA experiment, providing information and encouragement to apply for college financial aid had no impact on applying or enrollment, while having a tax professional walk someone through the process did. As a follow-up, Reuben Ford and I incorporated the college application process into the high school curriculum, such that graduating seniors were provided in-person assistance over three workshops to help choose a program they would likely get into, apply to that program, and apply for financial aid. College enrollment increased by 5 percentage points overall and by 9 percentage points among those not enrolled in university-track courses.
Easy access to an in-person coach who reaches out to offer support also increases engagement and effectiveness, compared to relying on text-message or email nudges. For example, contacting college students regularly to help set goals, manage time, and work through challenges increased graduation rates by 4 percentage points among nontraditional students at colleges with low levels of completion. Similarly, the sole treatment arm found to be successful at increasing college achievement among the 15 that Uros Petronijevic and I tried at the Student Achievement Lab included proactive upper-year coaching. Reaching out to first-year students and trying to meet weekly significantly increased average GPA scores and persistence, as well as subjective measures of wellbeing. But in terms of scalability, both programs were significantly more expensive than low-touch nudges. The first cost $500 per semester. In the second, one coach could handle only a maximum of 5 students, compared to our (ineffective) text-message coaches who could handle communicating with more than 100 students.
In addition to proactively coaching students, proactively coaching parents can be considered a type of high-touch nudge. In one study, texting parents about their high-school child’s performance was not as effective at improving test scores as visiting parents directly to discuss how to interpret the information and offering suggestions for getting students college-ready. Home visits to parents of younger-aged children have generally been found to improve early child development. Offering personal support to parents of preschoolers to help them learn how to be more interactive and engaging also appears to help. All of these programs, however, are substantially more expensive than nudges without a personalized touch.
Where Do We Go From Here?
Making decisions with immediate costs to obtain uncertain long-term incremental benefits is difficult. What’s the big deal about missing one practice, eating one more serving, or waiting one more day? Children and youth especially struggle, in part because their brains are not fully developed, and in part because they often have little experience making such consequential decisions. Behavioral science explores ways to help them by nudging thoughts and actions more likely to generate long-term gains without meaningfully altering up-front costs or options. Applying this research to education has become extremely popular, as nudges are often cheap in terms of both money and time. Removing behavioral barriers could unlock skill development and large lifetime rewards, and even small improvements to education outcomes could be cost effective.
We may, though, want to temper our expectations around the potential for nudging to help address education policy’s major issues. Low-touch nudges more easily influence one-time actions, such as completing an application or accessing student services, than they influence more ingrained habits or routines. We should continue to test when text messages, reminders, carefully constructed letters, and online exercises generate predictable gains that don’t depend on unknown operational details or unknown population differences. So far, major attempts to scale low-touch nudges have found very small or no short-term impacts.
High-touch nudges look more promising. It is perhaps not surprising that person-to-person interactions are more persuasive compared to text messages, email, or mail. Interventions using real people offering up-close help, such as application assistance, coaching, or tutoring, show more consistent positive impacts than low-touch nudges. They are also more expensive, making them a tougher sell to policymakers. Again, more research is needed to demonstrate when the tradeoff is worthwhile. New technologies, allowing for virtual social interactions or artificial intelligence, may help lower costs.
We should also consider shoving over nudging. Shoving restricts options in order to steer individuals towards more desirable behavior. There are some good examples of shoves in education policy: prohibiting the use of smartphones or computers in classrooms; making class attendance mandatory; and imposing homework assignment deadlines spread evenly throughout the term, as opposed to any time before the end of the course. In all three of these cases (evaluated with random assignment), academic grades improved. We don’t know for sure whether these restrictions make students better off, but many students would acknowledge the negative behavioral tendencies of checking their phones too often, sleeping in and missing class, or procrastinating on assignments. Shoving, to me, includes teachers choosing class content, schools choosing mandatory courses, and parents arranging children’s routines. The power to organize others’ daily activities involves high stakes.
Some shoves lead to better outcomes than others. In one study, struggling high school students were given daily in-school delivery of 2-on-1 tutoring from a supportive older peer. The researchers found math grades improved by almost half a standard deviation. In another study, elementary school teachers added into their daily activities a year-long curriculum of videos, case studies, and exercises to emphasize the role of effort in enhancing skills and achieving goals. Standardized math and verbal test scores and measures of executive functioning improved significantly, even measured more than two years after the experiment. A third example is New York’s Guttman Community College, offering limited-choice programs in which students must enroll full-time, take a fixed set of first-year courses, attend a three-week summer bridge program, and are assigned a “student success advocate” whose job is to help with the college transition. Structure and scaffolding around daily routines therefore seem to hold promise, but further replication of specific shoves, as well as evaluation of overall costs and benefits, is needed.
Nudging parents and teachers may be more effective than nudging children. Examples include encouraging more regular engagement, advising topics to talk about or teach, and sending information about progress and attendance. Parents and teachers want to help children and are more aware of their own behavioral barriers. They may be more likely to welcome trusted personal assistance, text reminders, or suggestions compared to the children they’re interested in helping. The relatively few studies involving nudging parents consistently document positive effects on learning outcomes. None of them, however, have been examined at scale.
Underlying behavioral barriers are personality traits, such as self-control, motivation, and self-esteem. Rather than trying to steer individuals predisposed to making poor decisions towards actions that are in their long-run best interests, we might make better progress targeting the reasons why such predispositions develop in the first place. To close the education achievement gap and meaningfully improve academic outcomes, we should look more towards shaping personality traits at younger ages. Understanding how may be one of the most important questions for social science.
Philip Oreopoulos is professor of economics at the University of Toronto. A list of references for this article is here.
The post Nudging and Shoving Students Toward Success appeared first on Education Next.
By: Philip OreopoulosTitle: Nudging and Shoving Students Toward Success
Sourced From: www.educationnext.org/nudging-shoving-students-toward-success-what-research-shows-promise-limitations-behavioral-science-education/
Published Date: Tue, 15 Sep 2020 09:00:19 +0000
No comments:
Post a Comment