NEW ROCHELLE, NY (March 19, 2023) — On Tuesday, March 14, we published an article about a report published by the New York State Comptroller Thomas DiNapoli. The following day we were contacted by New York State Education Department Deputy Director of Communications J.P. O’Hare.
NYSED is disputing the findings in the OSC report. In the interest of presenting both sides, we are publishing NYSED’s take on the OSC report in our article: DISASTER: New York’s Losses in 4th Grade Math & Reading Scores Are Double National Average, Says OSC
Hi Robert- I hope you are doing well. I wanted to reach out regarding your article published today, Wednesday, March 15 (the publishing date is March 14 not March 15) entitled “DISASTER: New York’s Losses in 4th Grade Math & Reading Scores Are Double National Average, Says OSC.” Upon review of the data presented in the article and the Comptroller’s report, NYSED found several errors and issues with the report. The Department works closely with our other state agencies. As such, we are in contact with the Comptroller’s Office to ensure these issues are addressed and corrected.
NYSED critique of OSC report:
The report switches between reporting on scale scores and percent of students reaching the proficiency category. There are two different metrics. This hides the fact that large scale score decreases do not always amount to major shifts in student proficiency (i.e., it feels like a larger impact to say that the average scale score went down 10 points than it does to say that 2% fewer students scored into a performance level).
Although there are decreases in both average scale score and percent of students in the proficient performance level, it is not accurate to make comparisons to other states beyond what NAEP itself provides in their technical data. When those data are reviewed, NY is similar or higher than:
- 31 states for Grade 8 math
- 36 states for Grade 8 Reading
- 27 states for Grade 4 math
- 41 states for Grade 4 reading
The report refers to Asian and Pacific Islanders. NAEP has categories for either Asian or Native Hawaiian/Other Pacific Islander. NY does not have a large enough population of Native Hawaiian/Other Pacific Islander students to be able to generate a reportable percentage, so only scores for Asian students are reported. The report from the Comptroller labels the data as coming from both groups, which is not accurate.
Finally, there are a couple of important considerations that are related to the properties NAEP itself:
NAEP does not measure NY’s learning standards, so it is entirely possible that the skills teachers emphasized are part of the NYS curricula but not NAEP’s. There (sic) scores would never be expected to correspond directly. The NYS assessments will always be a better measure of how students performed based on the expectations of our learning standards.
The sample size of NAEP is nowhere near large enough to support definitive conclusions about subgroups. NY has approximately 220,000 students in Grade 4. NAEP would have tested a sample of <2,000 (possibly much less, dependent on school-level participation). As a result, the number of students that is used to represent each subgroup must be incredibly small, possibly single digits in some cases.
Like NYSED, we know the Comptroller’s Office is dedicated to ensuring that New Yorkers, especially our educators, have access to accurate data in which to help make educational decisions on how to best support students. After our internal conversations with the Comptroller’s Office, we will share the outcomes of that discussion with you. We appreciate your coverage of academic outcomes for students here in New York.