At last week’s school board meeting, Dr. Jeffrey Korostoff made another of his “data presentations” which, as usual, failed to provide the data within any sort of meaningful context, offered no in-depth analysis and glossed over any information that did not reflect favorably on the district. Members of the school board, for the most part, passively received the presentation with only one member, David Lacher, raising any sort of serious challenge to any aspect of the presentation but then failing to sustain his point.
Here is the introduction:
CONTEXT: missing from the report is any mention that in 2007 the Federal government did tests which showed that gains in New York State exam results did not manifest themselves on the federal test. A recent report on New York City schools showed that despite an increase in grades or performance in math on the New York state tests, New York City students were not performing better on the federal test. In other words, there is some evidence that the New York State tests were made easier. In fact, there are jumps in performance in the New York State and New Rochelle test scores between 2007 and 2008 which lend support to this criticism. However, at a previous BoE, Dr. Korostoff meeting disparaged such concerns expressing his view that “we can’t win”, Korostoff complained that critics of the public schools will claim that if the tests show poor results then the schools are not doing their job and if the test results go up then the tests are being made easier. Preaching to the same old BoE choir, Korostoff’s observation was received with knowing nods of heads all around.
The fact is that is not possible to draw any definite conclusions about what an increase in performance means. It could be, as the district claims, that the school district is doing a better job of teaching the students and that’s being reflected in the increased performance — more students performing at level three and four — it could also be that the increase is a function of giving the students easier tests. While there’s no way to know for sure the most likely answer is a little of both: the test have been made easier after the teachers union pressured New York State to make the tests easier AND the teachers have gotten more comfortable teaching the material AND as students are now taking tests every year they are more comfortable taking tests AND the students have learned various test taking strategies.
Whichever is the case, and many experts believe it is a combination of these and other factors, there’s no way to say with any certainty what exactly explains the increase in test scores. That the school district would attribute any good news to their effort alone and any bad news to some other force outside of their control, is a good illustration of why the district has such low credibility when it makes claims about student performance.
One thing not shown in this report is a breakout of how many students are testing at each level. Why not provide charts that display bar graphs showing 100% of all scores, with each bar broken into 4 distinct pieces so the percentage of students achieving a three or four can be clearly seen. Rest assured that if the District is only displaying data that lumps Level 3 and Level 4 together then the vast majority of students are at Level 3. The advantage of such a breakout is that you might be able to see trends such as how many students are going from Level Two (below standard) to Level Three (above standard). Why would the District not want to feature that data?
On page 5, Korostoff presented a three-year trend of the ELA performance for grades 3-8. Here is how that data looks on a longitudinal basis:
The chart shows an increase from 2008 to 2009 due to rather large, across the board increases in performance. What is remarkable about this chart is both the significant decline in performance that has been occurring between elementary school and middle school and how that decline has suddenly been eliminated in one year. This is reminiscent of how racial gaps in math scores at Isaac Young suddenly disappeared last year. The single most remarkable statistic of all is the massive increase in ELA test scores between 2008 and 2009. Grade 6 scores jumped from 69% to 87%, a 26% increase in one year. Grade 7 scores jumped from 69% to 82%, a 19% increase in one year.
On Page 11 (above), Korostoff presented the three-year trend of math performance for grades 3-8 and once again the data directly contradicts the statements on Page 2 that performance continues to improve. This data shows mixed results. It was at this point that David Lacher asked about the longitudinal data in the chart (looking at a cohort over time such as we did above (Grade 3/2007, Grade 4/2008, Grade 5/2009).
The results here are mixed with some improvement and some declines. As was pointed out by David Lacher, a good way to look at this chart is to look at the data across cohorts as we have done above. This is what the District should always be doing in these reports — showing longitudinal studies of student performance where you were roughly following a cohort of students over a three-year period. If you do that here, you can see how the results are mixed for different cohorts.
Page 15 (above) seeks to sum up the charts by claiming steady incremental progress is evident in Math and ELA test scores. The exact opposite is the case. There has been some progress in some areas but in other areas there has been a decline in performance. More importantly, the increases are anything but steady. In fact, the vast majority of the performance increases come from a single source — Isaac E. Young Middle School — which shows a gigantic, unsustainable increase.
The most worrisome chart in Dr. Korostoff’s presentation is Page 17 (above) which indicates an increase in ELA test scores that is, at best, implausible. After two years (and likely more) of performance in the 50-59% range, the ELA scores at Isaac Young jumped to 76% — a 28% increase. This is by far and away the single largest jump in test scores between 2008 and 2009 and matched only by the equally unlikely increases in math scores reported last fall where the administration at Isaac Young claimed that math scores for black students went up 54% in two years and went up for hispanic students by 44% over the same two year period.
To put this in some context, there are NO SCHOOLS in New Rochelle that have posted any similar increases in performance for ANY subset of the student population or for an entire school population.
We have received numerous anecdotal reports that the increase in test scores at Isaac E. Young Middle School has very little to do with actual performance increases and everything to do with wholesale fraud and cheating. The surest indicator — a major red flag — is the incredible increases in test score performance in the past couple of years. From a school that was labeled underperforming by New York State and put on their equivalent of “double-secret probation” just a few short years ago, we are now supposed to believe that this school is producing performance increases unrivaled anywhere else in New Rochelle and, for that matter, in the entire State of New York.
The problem when you inflate test scores by cheating is two-fold: (1) to maintain the illusion you have to keep on cheating; (2) eventually those students move on to other schools which do not cheat and the actual performance reveals the fraud. If they have nothing to hide, the District should be willing to compare the performance of students at New Rochelle High School based on elementary and middle school of origin. As Talk of the Sound knows, and the District will not admin, they have done such a study. The results were not pretty.