☀️Summer 2024 MTSS Mini Summit: MTSS-Behavior Essentials at Your Fingertips 💙  👉Sign up now

    Despite growing investments in education, student dropout rates continue to be a global issue. Recent research shows that in the United States, 1 in 5 students do not complete high school.1 The risk of not graduating remains elevated for students from marginalized demographic groups, where high 9th-grade retention rates increase the chance of dropping out before graduation.2 In some schools, labeled “dropout factories,” the student population decreases more than 40% from 9th to 12th grade.3 

    Early intervention is one of the most effective ways to prevent academic failure and school dropout.4 However, educators require reliable and valid data systems to accurately identify students who are at risk. Early Warning Indicators (EWIs) are critical for determining which students need intervention. The following guide provides:

    • An overview of EWIs and the research supporting their use in schools

    • Practical tips for selecting indicators 

    • How to integrate EWIs into a Multi-Tiered System of Supports (MTSS) 

    • Help for overcoming limitations and common pitfalls

    •  
    •  

    ewi-guide-icon (1)

    Use this guide to help your school improve educational outcomes for all students
    through the effective use of Early Warning Indicators. 

     



    Don't Have Time To Read? 


    Overview of Early Warning Indicators (EWIs) 

    Early Warning Indicators (EWIs) are typically known as predictors of academic failure based on the performance factors of students. These indicators are warning signs that can identify students who are at risk of falling behind academically, dropping out of school, or experiencing other challenges. There can be many factors that schools and districts choose to use as indicators, but it is important that they are linked to student outcomes.5 EWIs are used to identify students early on and provide targeted interventions and support to help them stay on track toward their educational goals. 

    Important note: It is more effective to use EWIs than to rely on tracking academic risks based on student demographics.6 Although these indicators are warning signs that can identify students who are at risk of falling behind academically, dropping out of school, or experiencing other challenges. Certain demographic characteristics, such as race, ethnicity, and socioeconomic status, can be related to educational outcomes; they are not strong predictors on their own. Relying on these factors can also lead to biases in decision-making.

    Which Indicators Predict Student Outcomes? 

    Suspensions, attendance, and course failures are the most critical and powerful EWIs to identify students who are at risk of academic failure. All three have been shown to predict educational outcomes. Although much of the research on EWIs has focused on students in 9th grade7, indicators can also predict student outcomes from as early as 6th grade. 

    Collectively, the combination of suspensions, attendance issues, and course failures are negatively associated with academic performance in middle school and predict the likelihood of high school graduation. Although it is difficult to determine the indicators that are the most predictive of student outcomes, there are particular ways that each indicator should be measured and reported to ensure that educators can effectively identify students at risk.8

     

     

     Suspensions, attendance, and course failures are the most critical and powerful EWIs to identify students who are at risk of academic failure. 

     

     

    out-of-school-supensions

    Out-of-School Suspensions

    Out-of-School Suspensions (OSS), whether measured by the number of incidents that result in an OSS or the total number of days of OSS, consistently predict graduation rates. Suspensions are known as one of the core behavioral indicators and are related to both discipline referrals and detention. Ways that suspensions can be tracked include:

    • Number of suspension events 

    • Number of suspension days 

    Research has shown that students who were suspended out of school for more than three days had less chance of graduating as they reached the end of high school, compared to students who had any one incident resulting in an OSS throughout their education.9

    attendence-risk

    Attendance Risk

    Attendance Risk is measured by the number of days a student is absent out of the total number of school days to date. Ways that attendance data can be defined include:

    • Daily

    • By semester

    • Yearly

    • By academic course

    • By school or district-designated intervals

    Using attendance as an EWI has proven to be a consistent way of predicting graduation rates as well as academic and social-emotional student outcomes. Although the reasons students are absent can vary, high rates of absenteeism mean that students are not able to participate in academic learning and other school activities. Students who missed 15 or more days of school were found to have an increased chance of not graduating, compared to students who only missed a total of 10 days.10 Regardless of the reason for the absence, both excused and unexcused absences should be monitored and tracked due to the connection with poor performance.

    course-failure

    Course Failures

    Course failure data has been shown to significantly predict student graduation.11 Students who fail at least one course in their first semester of 9th grade are less likely to graduate compared to those who are passing.12 Research also shows that 85% of students with zero course failures graduated on time compared to only 55% of students with multiple course failures.13 Ways that course failure data can be defined include:

    • Quarterly, midterm, or end-of-year course grades

    • Course credits earned by students

    Although the term “course failure” implies an end-of-year outcome for a student, using data from marking periods throughout the year provides educators with data that can be used to proactively intervene.14 Waiting until final grades are released at the end of the school misses the chance to intervene and possibly correct the failing grade.

     

    Developing an Early Warning Indicator System (EWIS)

    More than half of public high schools in the United States implement some variation of a system that uses EWIs.15 An Early Warning Indicator System (EWIS) can vary in terms of the specific indicators included and how the data is used to determine who needs targeted support. Best practice recommendations can be used by schools to help ensure that these EWIS are being implemented properly so that educators have visibility into the student population that is truly at risk. 

    The following are factors and practices to consider when developing an EWIS:

    available-data

    Available Data

    • EWI data has to be easily accessible and available. It is important that the early warning indicator data is methodical: meaning ingested frequently and consistently. This can be data already collected by the school and/or district.

    defining-indicators

    Defining Indicators

    • Clearly defined EWIs include not only which indicators will be used but the specific cut-points or benchmarks that identify students at risk. How are cut points and benchmarks decided? Studies have shown that the following are effective examples of descriptors for EWIs that predict graduation rates and outcomes16:

      • Suspensions: More than two per year 

      • Attendance: More than 36 absences a year (can be broken down by quarter or semester) or depending on the grade level missing more than 10 - 20% of instructional time

      • Course Failure: One or more failures in any academic course or receiving a 2.0 or lower grade point average. Subject areas can be taken into consideration as well. For example, English and Math course grades are suggested to start monitoring as early as 6th grade.

    • Make sure that the established cut points for each indicator are consistent. Cutpoints with the most consistency are suspensions and course failures.17

    Graphic Examples of Defining Indicators

    summary of high yeild early warning indicators-min

    This graphic (Heppen & Therriault, 2008) is a good example of showing a benchmark for both attendance and course failures.

    EWI and cut points for high school drop out-min

    A graphic that shows similar cut points from (Li et al., 2016)


    research-based early warning indicator system-min

    Another graphic (AIR, 2015) that helps identify benchmarks or cut points for EWIs


    make-it-proactive

    Make it Proactive

    • Many EWISs primarily focus on just tracking the data and indicators without “next steps” or a framework for getting students back on track to graduate.18 A successful EWIS must be as proactive and action-focused as possible. This includes tracking indicators early and providing educators with direct access to strategies and interventions that EWI team members can implement right away.

    clarify-roles

    Clarify Roles

    • All related school personnel in the building should have a role in the EWIS. This ensures collaboration and a consistent, aligned approach to the needs of the student.19 Getting everyone involved, from the school and district levels, can help prioritize efficient collection, sharing, and review of the indicators linked to student outcomes. This also means that educators require access to up-to-date EWI data.


    Limitations of an Early Warning Indicator System

    Students’ academic and behavioral skills across K-12 can impact learning outcomes; however, when it comes to EWIs, the research has been focused on middle and high school. There is little evidence supporting the use of these specific indicators in elementary school to predict student graduation or dropout. Instead, at the elementary level, it is recommended to focus on students who need support in foundational academic, behavioral, and social-emotional skill areas. 

    EWIs might also not answer the questions as to why a student is struggling. Once a student is identified as being at risk for academic failure, additional data may be needed to determine the underlying issue as well as an aligned intervention approach. This can be done using academic, behavioral, and social-emotional assessment data. For example, it is important to know if a student’s poor academic performance is due to a foundational skill deficit, an issue related to school belonging and climate, or something related to mental health and wellness. All of this information will help guide the types of supports provided.

    Using Locally-Developed Indicators

    There are various state initiatives focusing on identifying the direct and local needs of districts and schools. Some schools may wish to include additional indicators, beyond suspensions, attendance rates, and course failures, in their EWIS. When developing localized indicators, it is important to consider a few things:

    more-data-not-better-1

    1.

    More data is not always better. 

    There is a common belief that having more data points will lead to more precise estimations and, therefore, better outcomes for students. But this is often not the case! It is much better to have fewer, high-quality, data sources. Although counterintuitive, when additional indicators get thrown into a system, it can result in students who are at-risk not getting the support they need. For example, a student might have a high rate of absenteeism, but according to a test their teacher gave them last week, they are performing adequately. Staff who are reviewing these data could decide that because the student appears to be doing well in this class, they do not require additional intervention. Without the needed intervention, the student could continue to be absent frequently and then stop showing up at school altogether, despite the passing test grade.

    relevant-indicators-1

    2. 

    Use indicators that are relevant to your student population. 

    When developing local indicators, schools should consider the unique needs of the students they serve. There could be specific barriers or challenges that a student population faces when it comes to educational success. Measuring these barriers can help educators understand who may be at risk. For example, in districts serving a large proportion of students from low socioeconomic backgrounds with less housing stability, it may be beneficial to track students who are highly mobile (i.e., frequently move from one school to another) to make sure that those students are receiving the support they need to successfully transition from one school to another. Using an MTSS Platform like Branching Minds means that EWI and intervention data are “attached” to the student and are instantaneously available to the new campus when a student changes schools. This helps prevent a lag in student support at what is often a vulnerable time. 

    indicators-validated-locally-1

    3. 

    Additional indicators should be validated locally.

    If schools want to include their own additional indicators, they should have some kind of data linking those indicators to student outcomes. For example, climate surveys are commonly being used by schools to assess belonging, relationships with peers and staff, and engagement in learning. If educators want to include these variables in their EWIS, they should examine whether or not student scores on these surveys are related to important indicators of success, such as performance on a benchmark or standardized assessment or graduation rates.


    The Role of Early Warning Indicators in an MTSS

    In a Multi-Tiered System of Supports (MTSS), academic, behavioral, and social-emotional data are leveraged to guide decision-making and help educators meet the needs of all students. Thus, EWIs can be used within an MTSS to provide educators with the information they need to identify students who may require additional support or intervention. Nevertheless, many educators struggle to integrate their EWIs within their MTSS practices, and as a result, these systems remain separate. The following strategies can be used to keep these practices aligned with the same goal of student success in mind:

    1.

    Provide school- and grade-level data teams with access to EWIs and the ability to review these data on a regular basis. Leadership teams should decide how these data will be collected and how often data will be updated and shared. It is common for data to be collected in various platforms and data systems, so including staff who have the technical ability to gather and organize these data is essential.

    2.

    Establish criteria and thresholds for determining whether or not a student requires intervention based on their EWI data, and make sure these are consistent across a district. In districts with a lot of variation in risk factors by school, it may be tempting to set different thresholds; however, this practice will often lead to inequity of support. Consistent cut points allow districts to allocate resources in equitable ways. In schools with a high proportion of students displaying these risk factors, additional support should be provided at the Tier 1/Universal level. 

    3.

    Develop a set of vetted, evidence-based interventions and strategies that educators implement immediately. Educators should have access to evidence-based interventions and strategies that are developmentally, culturally, and linguistically appropriate for the students that they serve. There are many resources available to help guide leadership toward effective programs and practices, such as PBIS, Attendance Works, and What Works Clearinghouse.

    4.

    Include systems and practices that allow for continuous improvement. All MTSS practices should be regularly reviewed and evaluated to ensure that components are being implemented with fidelity. Part of this review and evaluation should include the EWIS. Educators can use rubrics to answer questions such as:
    • Which indicators were used to identify students at risk? 

    • Were these indicators used consistently across schools? 

    • How many students identified received intervention? 

    • Were interventions implemented as planned? 

    • What were the outcomes for these students? 

    This high-level reflection helps educators gain a deeper understanding of their practices as well as what worked and where improvements can be made.


    Branching Minds_MTSS Platform_Support Library

    Library of Hundreds of Curated MTSS/RTI Evidence-Based Interventions

    The Branching Minds Library of Supports includes hundreds of evidence-based intervention programs, so if your school or district has purchased those resources, they can be added to a student’s intervention plan on Branching Minds, as well as nearly a thousand free evidence-based strategies, activities, and resources that can be added to a student’s intervention plan.

    Conclusion

    Developing a high-quality Early Warning Indicator System can prevent at-risk students from slipping between the cracks. Key risk factors can go unnoticed, especially at the middle- and high-school levels. Negative academic and developmental outcomes do not occur overnight; there is typically a cascading effect where patterns develop, which leads to more problematic behaviors that can be increasingly difficult to offset. Early identification and intervention help to offset potential negative trajectories. Identifying students at risk for academic failure and providing them with aligned supports and interventions is much more attainable when educators have a systematic practice in place to leverage high-quality data to inform their decision-making and intervention practices. 

    Bringing together the necessary pieces to create a high-quality EWIS takes time, but doing so will provide clarity for educators and opportunities for students to get the support that they need to be successful in school and beyond.

    ewi-guide-icon (1)

    EWIs on Branching Minds

    Branching Minds’ MTSS Solution helps educators proactively identify and support at-risk students by pulling together crucial data points for student success and making them visible across a student’s support network. Our Early Risk Report functionality gives teachers insights into student risk levels across attendance, behavior, and academic performance—data points that research20 consistently shows to be predictive indicators of student dropout. This approach supports the success of the whole student, allowing individual teachers and school systems to see where their student body is at risk, understand the barriers, and proactively deploy resources and interventions for the students in greatest need of support. 

    • For secondary students: To keep secondary students on track for course completion and graduation, Branching Minds can pull in attendance rates and grades from a district or school’s SIS and present it alongside behavior incidents, assessment results, and other MTSS data in a comprehensive Student Overview. 

    • For school and district leadership: The Graduation Risk Report provides an aggregated view of risk to inform system-level changes that can better support teachers and students. 

    Learn more about how your school can use Early Warning Indicators to predict student dropout and empower your educators to provide targeted support to students at risk.


    About the Authors

    essie-sutton-headshot-1

    Essie Sutton, PhD

    Dr. Sutton is an Applied Developmental Psychologist and the Director of Learning Science at Branching Minds. Her work brings together the fields of child development and education psychology to improve learning and development for all students. Dr. Sutton is responsible for studying the impacts of the Branching Minds platform on students’ academic, behavioral, and social-emotional outcomes. She also manages and curates the platform’s library of evidence-based supports and studies how different strategies, programs, and tools impact students’ educational outcomes.

    Dr. Sutton received her Ph.D. in Applied Developmental Psychology from Fordham University, where her research focused on evaluating academic and social-emotional learning programs in elementary classrooms. Her work has also examined how classroom contexts and the quality of teacher-student interactions influence students’ learning and social development. She has also developed and studied assessments used to measure indicators of children’s well-being and teachers’ classroom practices. Dr. Sutton’s research has been published in Child Indicators Research and the Journal of Applied Developmental Science. She has also presented her work at national conferences, including the American Education Research Association and the Society for Research on Child Development. Dr. Sutton also has an M.A. in Human Development, Learning, and Culture from the University of British Columbia and a B.A. from McGill University.

     

    Sarah SJ Vach_round cropped

    SJ Vach, PhD

    Dr. Vach was the Education Research Scientist at Branching Minds. She conducted research evaluations of the current literature focused on Multi-Tiered Systems of Support (MTSS) and the impact on student achievement and performance, providing insight on best practices. Dr. Vach also supported the Learning Science team by running data analyses to look at outcomes of students receiving Tier 2 and Tier 3 support.

    Dr. Vach received her Ph.D. in Special Education from the University of Nevada, Las Vegas with research focusing on identifying levels of math anxiety of students with disabilities (i.e., learning disabilities, autism spectrum disorder, speech and language impairments, emotional and behavioral disorders). Her work in education also focuses on looking for trends in achievement data and performance, and implementing data-driven decision making within marginalized populations including students at-risk for academic failure. Dr. Vach is a research methodologist and data analyst, using a multitude of methodologies and analyses to run both small and large data sets in various academic disciplines. She has had the opportunity to share her research at both national and international conferences, including the Council for Exceptional Children and the Council for Learning Disabilities. Dr. Vach has an M.S. in Autism Spectrum Disorders and Emotional and Behavioral Disorders, and a B.S. in Special Education, Generalist K-12, both from the University of Nevada, Las Vegas.


    With the Branching Minds partnership, we are gaining both a thought partner who will help us enhance all of our MTSS practices, structures and approach, and a platform that will help make the work easier and more efficient for all of our educators at CMS - from classroom teachers and support staff to school and district administrators.
    — Dr. Frank Barnes Chief Accountability Officer at Charlotte-Mecklenburg Schools, NC

    Want us to email you a copy? Fill out the form, and we will send you a PDF version of this guide

    References

    1. McMahon, Brian M., and Sabrina F. Sembiante. "Re‐envisioning the purpose of early warning systems: Shifting the mindset from student identification to meaningful prediction and intervention." Review of Education 8, no. 1 (2020).
    2. Balfanz, Robert, and Joanna Fox. "Sent home and put off-track: The antecedents, disproportionalities, and consequences of being suspended in the ninth grade." Journal of Applied Research on Children: Informing Policy for Children at Risk 5, no. 2 (2014); Taylor, Jacob-Paul, Malgorzata Zuber, and David Shoup. "Determinants of School Discipline: Examination of Institutional Level Factors Impacting Exclusionary Sanctions." Youth & Society 55, no. 3 (2023).
    3. Heppen, Jessica B., and Susan Bowles Therriault. "Developing Early Warning Systems to Identify Potential High School Dropouts. Issue Brief." National High School Center (2008).
    4. Allensworth, Elaine. "The use of ninth-grade early warning indicators to improve Chicago schools." Journal of Education for Students Placed at Risk (JESPAR) 18, no. 1 (2013). 
    5. Marshall, David T. "Course grades as actionable early warning indicators: predicting high school graduation in an urban school district." Preventing School Failure: Alternative Education for Children and Youth 66, no. 3 (2022).
    6. Mac Iver, Martha Abele, Marc L. Stein, Marcia H. Davis, Robert W. Balfanz, and Joanna Hornig Fox. "An efficacy study of a ninth-grade early warning indicator intervention." Journal of Research on Educational Effectiveness 12, no. 3 (2019).
    7. Knowles, Jared E. "Of Needles and Haystacks: Building an Accurate Statewide Dropout Early Warning System in Wisconsin." Journal of Educational Data Mining 7, no. 3 (2015).
    8. Li, Y., Scala, J., Gerdeman, D., & Blumenthal, D. (2016). District Guide for Creating Indicators for Early Warning Systems. San Francisco: REL West @ WestEd.
    9. Marshal, “Course grades as actionable early warning indicators”
    10. Marshal, “Course grades as actionable early warning indicators”
    11. Li et al., “District Guide for Creating Indicators”
    12. Heppen and Therriault, “Developing early warning systems”
    13. Heppen and Therriault, “Developing early warning systems”
    14. O'Cummings, Mindee, and Susan Bowles Therriault. "From Accountability to Prevention: Early Warning Systems Put Data to Work for Struggling Students." American Institutes for Research (2015); Heppen and Therriault, “Developing early warning systems”; Li et al., “District Guide for Creating Indicators”
    15. Davis, Marcia H., Martha Abele Mac Iver, Robert W. Balfanz, Marc L. Stein, and Joanna Hornig Fox. "Implementation of an early warning indicator and intervention system." Preventing School Failure: Alternative Education for Children and Youth 63, no. 1 (2019)
    16. O'Cummings and Therriault. "From Accountability to Prevention”; Heppen and Therriault, “Developing early warning systems”; Li et al., “District Guide for Creating Indicators”
    17. Stuit, David, Jim Lindsay, and Emily Loney. "Identifying Early Warning Indicators in Three Ohio School Districts. Stated Briefly. REL 2016-207." Regional Educational Laboratory Midwest (2016).
    18. Davis et al., “Implementation”
    19. O'Cummings and Therriault. "From Accountability to Prevention”
    20. REL Midwest at American Institutes for Research. (n.d.). Early Warning Indicators: An Introduction.
      https://ies.ed.gov/ncee/edlabs/infographics/pdf/REL_MW_Early_Warning_Indicators_An_Introduction.pdf