School Report Cards are Misleading

From the Wisconsin Conservative Digest

School Report Cards are Misleading:

The school reports that supposedly give parents information actually do the exact opposite.

For the first time in decades, education is finally getting the attention it deserves. School board battles reverberate around the nation. But the question that rarely is thoughtfully asked: How are the students doing? There have been efforts to rank schools and grade schools and introduce “education accountability.” It sounds like a great idea - except that the devil is in the details. In the spring hundreds of thousands of Wisconsin students will be taking a standardized test - as they have for many, many years. However, the name of the test has changed a few times in recent years.

In 2014 Wisconsin switched from the Wisconsin Knowledge and Concepts Examination to the long awaited Smarter Balanced Assessment that had been under development for years: the Badger Exam - because naming a standardized test developed by a multi-state consortium after the local mascot will make all the controversy go away, right? Well, it didn’t - and we ended up with the third test in three years - the Forward Exam! (Good thing we only went through three tests - I think we’re running out of “pet in-state” names to give to these tests).

Along the way we decided that letting parents know how their student was doing wasn’t enough. In 2015, Assembly Speaker Robin Vos led the effort to impose “accountability” on every school in the state (AB1 -2015) - because who is against accountability? However, as the discussion went along, we realized we didn’t know what accountability meant. Was it ensuring that every student was above average - like our neighboring Lake Wobegon School District? What if parents wanted to make sure that their student was first of all - safe? In most of Wisconsin that seems to be a strange hypothesis, but sadly as Milwaukee has jumped to the #3 homicide rate in the nation, more and more parents want to make sure that their children are safe. So, despite the priority label of AB1 placed on the “accountability” bill - AB1 went nowhere in 2015. Thankfully, parents were trusted to pick the best school for their students based on their own on-the-ground observations. The legislature settled on a “report card” to give a standardized score to compare schools - public, charter, and voucher - with the Department of Public Instruction assigning a 0-100-score-accurate-to-a-decimal-point “Overall Accountability score.”

Now that there’s a single point of data - albeit buried in spreadsheets on a bureaucracy’s website - the local chamber of commerce has decided to make the data easier to access and use. The Metropolitan Milwaukee Association of Commerce has put together a tool it calls the “Common Report Card.”

However rather than adding any other data to the mix, they simply make the Department of Public Instruction’s Data easier to visualize. The Milwaukee School Quality Map allegedly shows the schools quality of performance, with a color-coded map.

MMAC doesn’t address the underlying problem: Garbage In, Garbage Out. DPI’s Report Card “Overall Accountability scores” that use a decimal point to demonstrate how accurate they are, are crippled with a fatal flaw: for many schools there is little correlation between the “Report Card score” and their actual performance.

The easiest way to demonstrate this is by examining the “Common Report Card.” Thankfully it allows more options than the default “Overall Accountability Score” - however that is the default ranking, and in fact they have an actual ranking of schools based on this score.

Not only do we see that several schools are without any proficient readers (such as the Milwaukee County Correctional Facility high school), but we see that many of the schools are exempt from the scoring system. Obviously, the Corrections system serves a different type of student than say University School of Milwaukee. However, a cursory scan of schools with 0% proficiency indicates that “Overall Accountability scores” range from 10.7 for North Division High School to 78.5 for Hampton Elementary.

Pulling up Hampton Elementary’s Report Card from DPI, we see that things have improved over the last year, and a few kids are proficient in reading. But for the purpose of examining the reliability of the Report Card, let’s look at the prior year’s numbers for Hampton Elementary.

The 2020-21 Report Card showed 3 years of decreasing performance - and this is skipping the COVID year of 2019-2020. By the Spring 2021, not one student was proficient in reading and hopefully a couple were proficient in Math. Now the point of this article is not to shame a specific school – we’ll address the real reason for performance results like this in future articles. But by all accounts, academic growth is not trending positively by the 2020-21 report card. But that’s ok accounting to the DPI Report Card because the academic growth numbers for this school outperformed 92.4% of all K-5 schools in Wisconsin!

How did that happen? I’m sure you can skim the 38-page Technical Guide and find an answer, but if the answer is that complicated - then something is wrong. Because of how DPI handles Priority Area Weights, this category makes up 67.5% of the “Overall Accountability score,” thus allowing a school that did not have a single student proficient in reading to claim it was “Exceeding Expectations.”

The state accountability system for education is messed up, and neither parent nor organizations should rely on the “Overall Accountability Score” to objectively compare schools. We’ll look at better ways of measuring schools in future editions.

Complied by the WCD staff….