The Pressing Need to Test for Autocorrelation: Comparison of Repeated Measures ANOVA and Interrupted Time Series Autoregressive Models

Abstract

Neglecting to measure autocorrelation in longitudinal research methods such as Repeated Measures (RM) ANOVA produces invalid results. Using simulated time series data varying on autocorrelation, this paper compares the performance of repeated measures analysis of variance (RM ANOVA) to interrupted time series autoregressive integrated moving average (ITS ARIMA) models, which explicitly model autocorrelation. Results show that the number of RM ANOVA signaling an intervention effect increase as autocorrelation increases whereas this relationship is opposite using ITS ARIMA. This calls the use of RM ANOVA for longitudinal educational research into question as well as past scientific results that used this method, exhorting educational researchers to investigate the use of ITS ARIMA.