Yes, it is Possible to Close the First-Generation Achievement Gap

BY MAI MIKSIC | There is some good news about college enrollment: more racial and ethnic minority students are going to college than ever before (Snyder & Dillow, 2013). From 1976 to 2011, the enrollment of Hispanic students increased by 10 percentage points, Asian/Pacific Islander students increased by 4 percentage points, Black students increased by 5 percentage points, and American Indian/Alaska Native increased by .2 percentage points (Snyder & Dillow, 2013). Many of these students are first generation, meaning neither of their parents attended college. First generation students now account for about a third of all first-time incoming freshmen (Ho & Chang Wei, 2011).

The bad news is that many of these students struggle to succeed in college. Those who enroll often need remedial courses and perform at the low end in their classes; many eventually drop out (Chen, 2005; Ho & Chang Wei, 2011). These low-achievement and dropout patterns have created what Stevens, Hamedani, and Destin (2014) call the social-class achievement gap: the difference in the academic performance of first generation students compared to continuing generation students (students who have at least one parent went to college). Although the authors call this the social-class achievement gap, it refers solely to the generation gap and not socioeconomic status.

First generation students have trouble succeeding in college for many reasons, including lack of academic preparedness, social and cultural capital, and financial resources. Many universities acknowledge this as a problem and offer “bridge” programs, which provide first-generation students with general academic tips and strategies, such as how to study for a test or choose a major. However, such programs vary in quality and therefore in results.

A new research study contends that when first generation students are aware of their own social class and understand how it might affect their college experience, they do better in school. Stevens et al. (2014) devised an intervention that capitalized on the experiences and knowledge of senior students and transmitted this to incoming freshman students. They called this a difference-education intervention and examined its effects on first-generation and continuing generation students.

Sampling Strategy

At a mid-sized private university, the authors sent out an email to all first generation freshmen and accepted anyone who wanted to be in the study. This is called a “convenience-sampling” method, which has the benefit of yielding larger sample sizes, especially when working with hard-to-reach populations. Random sampling, which consists of randomly selecting students, would have been a more rigorous method because it allows for generalization of results to a greater population. However, it was clear that the authors were focused on maximizing their sample size in order to have greater statistical power to detect an effect. Sixty-six students, out of 158 total first generation students, were included in the final sample.  

Since the authors did not randomly select their sample, they needed to compare the general demographic characteristics of their sample to the other 92 first-generation students not in the study.They determined that there were no significant differences between the sample and the rest of the first-generation freshmen which suggests that the selected 66 first generation students were indeed representative of those enrolling in the university.

Next, the authors recruited continuing generation students using a stratified sampling approach. This method entails creating racial/ethnic group quotas based off the general continuing generation student population at the university, and randomly selecting individuals that met criteria until the quota was met. For example, since White students consisted of 62% of the continuing generation population, the authors randomly selected White students into their sample until Whites constituted 62% of the sample. This was probably the most appropriate sampling strategy to maintain the representativeness of the population, and in the end, 81 continuing generation students were included in the final sample.

The total sample recruited was 168 students, and of these, 147 students completed the study. This is a good retention rate (87.5%) for an experiment. What is missing is clarity on which cohort dropped out more frequently: first or continuing generation students. The authors try to address this issue by doing attrition analyses. They determined that the students who left the study were not statistically different based on demographic factors than the students who remained in the study. Still, it would have been helpful to know if, for example, all the students who left were first generation students.

Next, the 147 students (both first generation and continuing generation) were randomly assigned to the difference-education intervention or to the comparison group which received a standard intervention, thus ensuring that the two groups were comparable. In the end, 75 students were put into the experimental condition and 72 students were put into what the authors called their control condition although, strictly speaking, it should have been called a “comparison group,” since these students received some type of intervention as opposed to no intervention. Finally, the authors also included a campus-wide group which received no intervention of any kind. This group, rightly called a “control group,” consisted of 81 first generation students and 1,697 continuing generation students. It was unclear whether this control group consisted of only freshmen.

This study was more complex than it appears at first blush. There were actually six groups being compared: first and continuing generation students who received the difference-education intervention; first and continuing generation students who received the standard intervention; and first and continuing generation students who received no intervention. What is unknown is how many first generation students were included in the difference-education intervention and how many were there in the standard intervention. Conversely, we do not know how many continuing generation students were in the difference-education intervention and how many were in the standard intervention. This information is not found in the article or the supplemental materials. While the authors did nothing incorrect in the implementation of the study, the omission of information is frowned upon, and readers should be given as much information as possible in order to determine if the randomization properly worked.

Difference-Education Intervention (n=75) Standard Intervention (n=72) Control Group (n=1778)
First Generation Students (n=?) Continuing Generation Students (n=?) First Generation Students (n=?) Continuing Generation Students (n=?) First Generation Students (n=81) Continuing Generation Students(n=1697)

 

Interventions

What did the two interventions consist of? Both featured a panel of eight seniors (three first-generation students and five continuing generation students), who spent an hour discussing the factors that influenced their college experiences. The key distinction: students in the difference-education intervention emphasized how their social-class backgrounds contributed to their college experiences. For example, one first-generation student might talk about how she did not know how to apply for scholarships because her parents did not understand the process and could not assist her. In addition to identifying the struggles they had as a result of their social-class backgrounds, the students also highlighted strengths that they had and strategies that they used to succeed.

Students in the standard intervention, by contrast, also talked about their general experiences, but did not highlight social-class as an issue. For example, one of them talked about how attending a very small high school affected their transition to a bigger university. Others gave general advice about how to succeed in college, such as to sit in the front row in classes and to join social groups on campus.

After the students finished with the intervention, they were asked to fill out a survey and give a video testimonial about their thoughts on what they had heard. This served as a manipulation check, to determine whether the students had truly internalized the messages they had received. The videos were coded and analyzed, and it was determined that the students indeed internalized the messages. What is striking is that the difference-education interventionstudents mentioned that they learned that their social-class could negatively affect their college experiences and that they could succeed despite this disadvantage.

Analyses

The authors used academic achievement after one year, as measured by students’ official transcript GPAs, as the outcome of interest. They also measured behavioral patterns, such as whether the students sought out campus resources, and psychosocial functioning, such as stress and anxiety levels. Additionally, the authors measured whether the students retained the information they had learned at the panel session at the beginning of that year. With the exception of academic achievement, all other outcomes were collected through self-report surveys.

In order to isolate the effects of the intervention, the authors controlled for race, gender, income, SAT scores, and highest high school GPA. They conducted an analysis of covariance (ANCOVA) to determine whether the intervention had any effects on student outcomes. ANCOVAs are very similar to regressions, but are the preferred method of analysis for experiments when there are two or more groups being examined. Essentially, ANCOVAs measure whether the mean, or average, outcomes of the groups differ while controlling for a set of other variables.

Results

The results of this study were strong. In the difference-education intervention, there were no statistically significant differences in GPA between the first generation students and the continuing generation students. The social-class gap was closed: first generation students performed just as well academically as continuing generation students. Meanwhile, in the standard intervention, there was a .30 grade point difference between first generation students and continuing generation students. That means that the achievement gap was 63% smaller in the difference-education intervention compared to the standard intervention. Additionally, first generation students in the difference-education intervention did better than first generation students in the standard intervention and in the campus-wide control group. Continuing generation students in the difference-education intervention did not differ in academic achievement from the continuing generation students in the standard intervention and the campus-wide control group, which means that the intervention was able to specifically target its intentional subgroup: first generation students. Importantly, the course-taking patterns of the students did not influence these GPA differences.

The difference-education intervention also affected first generation students’ behaviors; the degree to which they sought campus resources was statistically eliminated. Conversely, first generation students in the standard intervention did not seek out resources as often as continuing generation students.

Students in the difference-education intervention also experienced less stress and anxiety, better adjustment to college life, and more academic and social engagement than those in the standard intervention. Somewhat surprisingly, the difference-education intervention also improved such outcomes for the continuing generation students in this cohort.

Finally, and most impressively, the authors also conducted mediation moderation analyses. Mediation moderation analyses examine the mechanism through which an intervention works – the so-called “black box.” That means that the authors not only sought to understand whether the difference-education worked, but they also sought to understand how it worked. In this case, the authors found that the difference-education worked by improving first-generation students’ behaviors, specifically prompting them to take advantage of college resources.

Conclusions

This study was carefully thought out and executed. Comparing six groups was important to understanding the effects of the intervention. The difference-education intervention itself was simple, and thus made results easy to interpret. The authors showed that their samples were representative, which means that these results are likely to recur again if the study is performed again with a different sample from this university or similar universities. Finally, the authors’ use of random assignment allowed them to isolate the effect of the intervention from other potential factors.

The article did exclude information that would have helped verify the study’s soundness. Even after reading the supplemental materials of the article, I still had questions about sample sizes. It was not clear if a majority of the first generation students ended up in the difference-education intervention or the standard intervention. The random assignment should have taken care of this problem, but without knowing the sample sizes I could not be sure.

This study addresses one of the core concerns of modern education policy: the college completion gap. Stevens et al. (2014) showed that an hour long intervention that highlights the role of social class in the college experience can have profound effects on first generation students’ outcomes at the end of freshman year. The current study is exciting, because the presenting intervention is low-cost and easily implemented elsewhere. Here we have some evidence that closing the social-class achievement gap for first-generation college students is possible. The next obvious steps are to follow up on the first generation students through degree completion, and to replicate the study at other universities.

References

Chen, X. (2005). First Generation Students in Postsecondary Education: A Look at Their College Transcripts.Washington DC:U.S. Department of Education, National Center for Education Statistics. Retrieved from http://nces.ed.gov/pubs2005/2005171.pdf

Ho, P., & Chang Wei, C. (2011).  Trends in Attainment Among Student Populations at Increased Risk of Noncompletion: Selected Years, 1989–90 to 2008–09. Washington DC: U.S. Department of Education, National Center for Education Statistics. Retrieved from http://nces.ed.gov/pubs2012/2012254.pdf

Snyder, T.D., & Dillow, S. A. (2013). Digest of Education Statistics, 2012. Washington DC: U.S. Department of Education, National Center for Education Statistics. Retrieved from http://nces.ed.gov/pubs2014/2014015.pdf

Stevens, N. M., Hamedani, M. G., & Destin, M. (2014). Closing the social-class achievement gap: A differences-education intervention improves first-generation students’ academic performance and all students’ college transition. Psychological Science, 1-11.

Leave a reply

Recent Posts

IdeaLab

Moynihan and the Modern American Family at 50

By Gerard Robinson | Senior Fellow, CUNY Institute for Education Policy “The gap between the Negro and most other groups in American society is widening. The fundamental problem, in which this is clearly the case, is that of family structure.” — Daniel Patrick Moynihan (1964) “If the misery of the poor be caused not by… Read More

Director's Perspective

An Open Letter On School Accountability

Below you will find the outline of some thoughts on educational accountability. The topic has become one of the central issues in contemporary reform efforts in American education, with particularly heated debate over efforts to measure the effectiveness of teachers through test scores as well as classroom observations. But the issue is wider than this,… Read More

Reports from the Field

Institutional Commitment to Civic Education: Public Montessori Secondary Schools

By Katie Dulaney | Lakewood Montessori School Even as we hold schools increasingly accountable for student achievement, we rarely seem to judge schools for their performance in citizenship preparation, an inauspicious “accountability gap” for a democracy. -Michael Johanek and John Puckett, “The State of Civic Education” (2005) Education leaders across the political spectrum – from… Read More