Interrupted time series is a research design in which an outcome is measured many times before and many times after a specific event, such as a new reading program or a schedule change. Schools generate this kind of data constantly through attendance logs, test results and discipline records.
The approach looks at trends rather than single snapshots. Instead of comparing one year to the next, it asks whether the line of results bent at the moment the change began. This article explains the logic, the steps involved and the limits every reader should keep in mind.
Why Single Comparisons Can Mislead School Leaders
Many schools judge a new initiative by comparing one year’s test scores with the year before. That comparison is tempting because it is simple, but it ignores everything else that changed between the two years. Students may have had a different teacher mix, a harder curriculum or an unusually strong cohort.
Scores also move up and down naturally from year to year, a pattern researchers call normal variation. A single jump might be real progress or just a lucky group of students. Without more data points, it is hard to tell the difference.
Interrupted time series addresses this problem by examining the full pattern of results over time. It treats the period before a change as a baseline and tests whether the trend shifts afterward in a way that is larger than the usual ups and downs.
The Basic Logic Behind the Design
The design rests on a simple idea: if a policy works, the outcome should change at the point of implementation, not randomly at some other time. Researchers collect many observations on each side of the event, often monthly or weekly, so that the pattern before the change is clear.
Two features are usually examined. The first is the level, meaning the average value just before and just after the event. The second is the slope, meaning the direction and steepness of the trend. A change can show up as a sudden drop or rise in level, as a shift in slope, or as both.
Statistical models are then fitted to estimate how much of the change can be attributed to the event. Analysts also check whether the pattern could have arisen by chance, using standard tests that they explain in their reports.
What Researchers Need Before Starting
A well-built interrupted time series depends on good preparation. Before collecting or analyzing any numbers, the research team should confirm that the data meet several basic requirements. The following checks help ensure that the results will be meaningful.
- A clearly defined intervention date that everyone agrees on
- At least a reasonable number of observations before and after the event
- A consistent measurement method that does not change midway through
- Records that are collected the same way across the entire time period
- A plan to note other major events, such as staff turnover or building renovations
These steps may seem routine, but they matter a great deal. If the intervention date is fuzzy, the analysis may attribute a change to the wrong cause. If the measurement method shifts, the trend may appear to change even when the students have not. Taking time to document each of these points makes the eventual conclusions far easier to trust.
Seasonality, Trends and Other Patterns to Watch
Educational data often contain patterns that repeat on a schedule. Attendance tends to dip around holidays, test scores often rise through the school year, and discipline referrals may spike near the end of a term. If these seasonal patterns are ignored, they can be mistaken for the effect of an intervention.
Analysts usually account for seasonality by including terms in the model that capture regular cycles. They also look at the long-run trend, which may already be rising or falling before the event. Separating a pre-existing trend from a true change is one of the most important skills in this design.
Good practice is to plot the raw data before any modeling. A simple graph often reveals whether the pattern looks like a genuine shift or just noise, and it helps readers understand the results without needing advanced statistics.
Threats That Can Weaken Conclusions
No design is free of risk, and interrupted time series has its own vulnerabilities. The most common is history, meaning that some other event happened at the same time as the intervention. If a new principal arrived the same month a reading program began, it may be impossible to separate their effects.
Another concern is instrumentation, which occurs when the way outcomes are measured changes during the study. A new grading rubric or a different test version can produce an apparent shift that has nothing to do with student learning. Researchers should check for such changes carefully.
The design cannot fully rule out regression to the mean, which is the tendency of extreme values to move back toward average. If a school adopts a policy after an unusually bad year, results may improve even without the policy. Adding a comparison group, when possible, strengthens confidence.
How Comparison Groups Strengthen the Design
One of the most useful upgrades to a basic interrupted time series is a comparison series. This is a parallel set of data from a group that did not receive the intervention, such as a neighboring school or a grade level that stayed on the old system.
If both series move together before the event but diverge afterward, the evidence for an effect becomes stronger.
The approach is sometimes called a controlled interrupted time series. It helps answer the question of whether any broad trend, such as district-wide changes or statewide testing shifts, could explain the result. The comparison group should be as similar as possible in student population and prior performance.
Even with a comparison, the groups are rarely identical, so caution is still needed. Researchers should describe the differences openly and avoid claiming more certainty than the data allow.
Reading and Reporting the Results
Once the data are analyzed, the findings need to be explained clearly to people who may not have statistical training. Good reports focus on what changed, how large the change was and how confident the authors are. The following sections describe the main elements to look for in any report.
Interpreting the Level Change
The level change describes the immediate jump or drop in the outcome right at the intervention point. A reader should look for the size of the change expressed in the original units, such as percentage points of attendance or scale score points on a test. Large numbers are not always meaningful if the baseline is already high.
It is also worth asking whether the level change persisted. Some interventions produce a short burst of improvement that fades within a few weeks. A sustained shift is usually more convincing than a temporary spike.
Interpreting the Slope Change
A slope change describes whether the trend itself became steeper or flatter after the intervention. For example, a school might see scores that had been rising slowly begin to climb faster after a tutoring program starts. Slope changes often reflect gradual learning or habit formation rather than sudden effects.
When reading a slope change, check whether the post-intervention period is long enough to show a stable trend. A few months of data may suggest a direction without proving it, so longer follow-up strengthens the claim.
Understanding Confidence and Uncertainty
Every estimate in an interrupted time series comes with uncertainty, usually shown as a confidence interval. This range indicates the values the true effect plausibly falls within. A wide interval means the estimate is imprecise, even if the central number looks impressive.
Readers should be cautious when a report presents a single number without any range. Honest reporting acknowledges both the best estimate and the limits of what the data can support. Phrases such as the evidence suggests are more accurate than certain claims in most educational settings.
Where Interrupted Time Series Fits Among Other Designs
Interrupted time series is one of several ways to study whether an intervention works. A randomized controlled trial, in which participants are assigned to groups by chance, is generally stronger for establishing cause. However, random assignment is often impractical for school-wide policies, where whole classrooms or buildings adopt a change together.
Interrupted time series is especially useful when an intervention is rolled out to everyone at once and no randomization is possible. It can also be applied retrospectively to existing records, which makes it efficient. Its main weakness is that it depends heavily on the assumption that nothing else changed at the same moment.
Researchers often combine several designs to build a fuller picture. Pairing a time series with qualitative interviews or teacher observations can explain why a change occurred, not just whether it did.
Ethical and Practical Considerations for Schools
Collecting and analyzing student data raises ethical questions about privacy and consent. Schools should follow applicable rules for handling records, remove names and identifiers where possible, and limit access to people who need it for the analysis. Families generally expect that information about their children will be used responsibly.
Practical considerations matter as well. Teachers should not be asked to collect extra data that disrupts instruction without good reason. Leaders should also avoid pressure to report favorable results, since that pressure can quietly shape how measurements are taken and interpreted.
Transparency is the best safeguard. Sharing the methods, the data sources and the limitations openly allows colleagues and families to judge the findings fairly and to decide whether the policy deserves to continue.
Common Mistakes Readers and Researchers Make
One frequent mistake is treating any visible change as proof of an effect. A line that moves after an event may be coincidence, especially when the data are noisy or the time window is short. Looking for a clear, sustained pattern helps avoid this error.
Another mistake is choosing the intervention date after seeing the data. If the date is moved to wherever the results look best, the conclusion is unreliable. The date should be fixed in advance based on when the change actually began.
A third error is ignoring the baseline. A school that was already improving may appear to gain from a new program that had little real effect. Careful analysis always asks what would most likely have happened without the intervention.
Questions to Ask Before Trusting a Time Series Report
Parents, teachers and administrators do not need to be statisticians to evaluate a report sensibly. A few well-chosen questions reveal a great deal about how strong the evidence is. The following points are a useful starting checklist when a school presents results from this kind of study.
- How many data points were collected before and after the intervention
- Was the intervention date set in advance or chosen afterward
- Did anything else change at the same time that could explain the result
- Was a comparison group used, and how similar was it to the treated group
- Are the results shown with uncertainty ranges rather than single numbers
Asking these questions does not require hostility toward the school or its staff. It reflects the same care people bring to any important decision. Reports that answer these questions openly tend to be more useful, because they show readers exactly what was learned and what remains unknown.
Putting Interrupted Time Series to Work in Your Community
Educators who want to use this design can start small. Choose one outcome that is already recorded regularly, such as weekly attendance or monthly quiz averages, and gather at least a year of data before and after a planned change. Many schools already have the raw material; the main task is organizing it carefully.
Involving teachers and families in planning the study improves its quality and its acceptance. When people understand what is being measured and why, they are more likely to trust the results, whether they are positive or disappointing. Honest findings, including those that show no clear effect, are valuable because they guide better decisions.
Over time, a habit of tracking outcomes this way helps a school learn from its own experience. Interrupted time series will not answer every question, but it offers a disciplined way to see whether change is real, and that clarity is worth building into everyday school practice.