By Chelsea Farley | Ms. Farley is the Communications Director for the Research Alliance for New York City Schools.
In recent years, states and districts have made academic outcomes, such as standardized test scores and graduation rates, the primary measure of a school’s performance. This focus on outcomes has been valuable for tracking student achievement and identifying both schools and students in need of support. One of the weaknesses of the approach, however, has been its failure to provide information about how schools should improve. Faced with lackluster performance data, struggling schools are often left wondering, “Now what do we do?”
The good news is that there is a growing body of evidence about specific areas of school practice and capacity that are essential for improving student achievement. The University of Chicago Consortium on Chicago School Research used data from hundreds of schools in that city to identify five “essential supports for school improvement,” including elements like effective leadership and strong family-community ties.[i] The researchers found that schools doing well in most of the five areas were at least 10 times more likely than schools with just one or two strengths to produce substantial gains in reading and math. Prolonged weakness in even one area “undermined virtually all attempts at improving student learning.”[ii]
This study, and others that have reinforced its findings, help us understand much more concretely the skills and capacities that enable schools to improve. The next question is how to provide schools with information about these key areas and help them use that information to bolster their work.
An initiative taking place in New York City may begin to shed light on this question. In the fall of the 2014-15 school year, the New York City Department of Education introduced a “Framework for Great Schools” based largely on the Chicago research. A critical first step toward using the “Framework” has been to figure out how to measure its elements in each of the City’s schools. With help from the Research Alliance for New York City Schools, the NYC DOE overhauled the 2015 Parent, Student, and Teacher School Surveys, to collect better information about where each school stands in relation to the necessary capacities identified by research. The DOE will use the survey results, along with other information, to assess schools’ current strengths and weaknesses, and to target areas in need of improvement. Below, we describe changes made to the Survey and outline some crucial issues we hope this work will address as it develops.
Retooling the Survey
The NYC School Survey is administered each spring to students in grades 6 through 12 and to all district parents and teachers, making it one of the largest efforts of its kind in the nation. In 2014, nearly a million parents, students, and teachers completed the Survey, yielding information about some 1,800 schools.[iii]
Since 2010, the Research Alliance and the DOE have collaborated to strengthen the survey. Our 2012 report Strengthening Assessments of School Climate, provides an interim view of this process; we had found that while the Survey had robust response rates and effectively tapped into parents’, students’, and teachers’ unique perspectives, the information gathered was of limited value for assessing key aspects of school practice and capacity. Thus, the report recommended incorporating new measures that would better distinguish between schools and that offered clear links to other performance indicators, such as grades, attendance, and test scores.
The introduction of the “Framework for Great Schools” provided an opportunity to do just that. Throughout the summer and fall of 2014, teams from the Research Alliance and the NYC DOE developed strategies for measuring the elements in the “Framework”. In some cases, we were able to use measures that research had already validated. When these were not available, we created new measures. As a result, the 2015 Survey looks quite different than past iterations. For example:
- To assess Effective Leadership, the Survey will ask teachers if their principal “communicates a clear vision for this school,” “sets high standards for student learning” and “knows what’s going on in my classroom.”
- To assess Family-Community Ties, the Survey will ask if “teachers really try to understand parents’ problems and concerns,” if “parents are greeted warmly when they call or visit the school,” and if “school staff regularly communicates with parents about how they can help their children learn.”
- To assess Rigorous Instruction, the Survey will ask students if they “build on each other’s ideas during class discussions,” “use data and text references to support their ideas” and “provide constructive feedback to their peers/teachers.”
- To assess Supportive Environment, the Survey will ask students how safe they feel in and around school and will ask parents if students at the school “feel it is important to come to school every day,” “think doing homework is important,” and “try hard to get good grades.”
- To assess Collaborative Teachers, the Survey will ask teachers about professional development opportunities and whether they “look forward to each working day at this school” and “feel loyal to this school community.”
- To assess Trust, the Survey will ask students if teachers “always keep their promises” and “treat me with respect.”
Over the next several years, the Research Alliance will continue to analyze the Survey’s results and provide feedback about the quality of the measures being used.
What We Hope to Learn
New York City’s use of research to develop the “Framework,” and to recalibrate the Survey accordingly, is a strong beginning to an ongoing process. In the coming years, the Research Alliance anticipates advancing our understanding of school capacities and outcomes in at least two areas:
- Building a better survey. By analyzing survey results, alongside information collected through other instruments (like the Quality Review or the district’s instructional audits of struggling schools), we hope to help NYC build a Survey that effectively gauges schools’ capacity—in areas that matter most for enhancing students’ achievement and development. We will examine, for instance, whether the organizational capacities identified as essential in Chicago and other cities also predict improved student outcomes here in NYC. And we will make suggestions for improving specific capacity measures.
- Developing a meaningful cycle of improvement. We also hope to work with a small group of schools, as part of an intensive pilot project that will explore strategies for sharing and making use of capacity data. Among the questions we will examine are: How is information about school capacity best communicated to reflect the unique assets and challenges of individual schools? What kind of support do schools need to interpret and make use of that information? And what actions do schools take in response to the data they receive?
Ultimately, we hope that when schools have a clearer picture of their strengths and weaknesses, and receive support in making important changes, they will be able to serve students and communities more effectively. As New York City’s initiative unfolds, it is sure to produce valuable lessons that inform school improvement efforts, not only in New York City but across all of our country’s large urban school districts.