Response to Atlantas article on cheating: This past Sunday, the Atlanta Journal Constitution published a series of articles claiming

that hundreds of school districts nationwide, including Dallas ISD, exhibit suspicious test scores that point to the possibility of cheating. Dallas ISDs Evaluation and Accountability Department, which oversees testing and test security in the district, has reviewed the methodology used by the Atlanta paper and finds the following. Atlanta Journal Constitutions analysis The analysis uses school level data, not student level data, with the assumption that the composition of the school is the same across the two analyzed points. Unfortunately, these compositions can fluctuate greatly across years for a variety of reasons (e.g., student mobility patterns, shifting demographics, redistricting school catchment areas, incorporating students from a defunct district, knocking down a set of apartment buildings, etc.). It is highly probable that this type of analysis will overestimate the number of schools that are engaged in system-wide cheating. A more refined analysis can be conducted using student level data that can pinpoint schools with anomalous results that then require further investigation. In fact, the Atlanta Journal Constitution points this out in its own analysis of its methodology by saying, Ideally we would look at how individual student test scores change from year to year, but federal privacy regulations precluded access to that data. The approximate cohorts we used were the only available substitute. It is unlikely that two groups of students in a cohort are perfectly identical. Urban districts in particular have high student mobility. Dallas ISD, which does have access to individual student test scores, has been conducting such studies since 2005. According to the Texas Education Agency, Dallas ISD is one of only a few districts statewide to screen for anomalies in this thorough a manner. Dallas ISD analysis A statistical analysis of test scores is annually conducted by the district to identify sufficiently anomalous results that warrant further investigation. Mathematics and reading scores are examined at the classroom level. A regression of the previous two years scores is conducted on the current years score for all students in the district at each grade level. The resulting residual scores are aggregated by class and converted into standardized scores with an average of zero and standard deviation of one. Classrooms that are two standard deviations or more above the mean are selected for further investigation. A second analysis is then conducted on the students scores for each of the identified classrooms. Anomalous results from the statistical analyses are viewed as an entry point for a more thorough investigation and not as the sole determining factor that cheating has occurred. Analyses of number of erasures, wrong-to-right erasures, other forms of test data, student grade and test patterns and interviews of teachers, staff and potentially students all could serve as evidence to help make these determinations. What is important to note is that investigations by Dallas ISD do not rely solely on anomalous scores, but evidence gathered across a variety of sources. The anomalous results simply act as an early warning system. When Dallas ISDs Evaluation and Accountability department conducts similar analyses, it does not find the large numbers of anomalous results that the Atlanta Journal Constitution purports to find, but again, Dallas ISD uses a much more refined instrument than the one used by the paper. Summary Dallas ISD aggressively stepped up its approach to test security when irregularities were first reported eight years ago. Teachers are no longer allowed to administer state tests or other secure exams for their own classes and testing is monitored by central staff to make certain that testing procedures are followed. In addition, Dallas ISD evaluates and analyzes results to identify test anomalies. According to the Texas Education Agency, Dallas ISD is one of the few districts in the state to conduct this kind of analysis. The district has taken personnel action, including termination, based upon violations of test security and/or investigations into alleged cheating.