Data and Respectability in Confusing Times
Justin Bathon on Tuesday, May 13, 2008 at 1:02PM Lately I have been sitting at my computer day after endless day crunching numbers. Meanwhile, as I am wont to do, I keep up with politics. It is no surprise this has been an exciting year for politics and the mass confusion surrounding the democratic primary has kept things interesting.
But, throughout this whole process have you noticed who is getting the most respect? Its the numbers guys and it is not even close. Chuck Todd, the numbers guy for MSNBC is probably respected as the best analyst this season and was probably the first mainstream media guy to understand that Sen. Obama won the primary in February as he ran up the numbers on Sen. Clinton. Also, Poblano, who runs the electoral statistics site fivethirtyeight.com (the number of electoral college votes) has gained wide acclaim for his amazing statistical pr
ojections, even calling the latest round of primaries in North Carolina and Indiana closer than ANY of the polls using statistical calculations based on demographics and past voting patterns. These two guys, and a few others (CNN's county by county maps have been very helpful), have actually managed to GAIN respect during this primary season as so many other talking heads seem increasingly like opinionated, partisan dimwits that know less about politics than most bloggers.
All of this has lead to my conclusion that there is something about quantitative data that is comforting in confusing times. All of these talking heads are going around speaking with people, listening to speeches, and gathering other qualitative-type evidence which they translate into bombastic opinions hour after hour on the news networks. But, Chuck Todd comes on MSNBC for a 2 min. segment and says more than all the hours of opinion combined. The numbers tell the story here folks, the rest is just fluff.
What is really, really odd about all this is just how simple some of these calculations are and yet we look at these numbers folks as geniuses. Most of what Chuck Todd does is counting, throw in a little addition and subtraction and boom you have a REAL analysis of the state of the race. Check out these ABC Charts and Graphs (ppt) that were put out today that are getting wide acclaim across the Net. Are you kidding me? My high school students could have done that. CNN's awesome county maps are no more than disaggregating the number of votes by county and then calculating a simple percentage. Poblano's statistics are little more involved because he calculates error for weighting purposes, but past that he is using pretty much just percentages disaggregated by state.
My conclusion is then that even simple, easily calculated data can be extremely powerful in confusing times. When everyone knows what's going on and there is general agreement, the numbers are quite meaningless. They just reaffirm the general knowledge, which is nice, but somewhat unnecessary. But, when no one knows what's going on, everyone falls back on the numbers and we put the numbers folks up on a pedestal. As computers become more and more powerful and numbers can be increasingly calculated and posted to the Web automatically in preformed graphical packages (ala Poblano's and CNN's sites). We need to integrate this more into education. Think of a Poblano site for education. Here is the recipe: It takes a certain amount of basic statistical knowledge (which any Doc. Student should have), a basic Web 2.0 knowledge, knowledge of Excel, and a general curiosity. That's it. But with that simple recipe, you can control the conversation in confusing times.
Okay, back to crunching numbers.
Reader Comments (4)
I will give an example. I presented some leadership preparation data in easy to read charts and graphs that required little explanation and/or mathematical knowledge and we got a great reception from state officials. They even paid for a conference room at an Indy Hotel and invited folks from all over the state so that we could give the presentation again.
Contrast that with a recent report I co-authored on teacher retirement systems and the reception of our data was substantially more muted. One of my co-authors was an economist and much of the paper was written as an economist would write it. I tried to add color to the paper (the other one was very colorful), but my co-author removed the color in my charts because the charts he produced were your standard black/white spreadsheet table with 50 numbers on it and little asterisks. You know the kind. Now, that was an excellent report (probably better than the other one to be honest) but the presentation of the data was not as good. The exact same state officials that I have worked with earlier thanked us for the work ... and that is about it. That is not meant to be a knock on the report, it was a good report and will eventually wind up in an economics journal, but it was not written with presentation to state officials in mind, imo.
Now, I present less frequently to groups of teachers (except my classes) and I will grant you that there is a level of DIScomfort/disinterest among those groups. Some people are just flat not interested in education data and some are already set in their notions. But, I think there are policy circles where simple presentations of simple data could be extremely powerful. One example I can give is bullypolice.org. Even with their simple "grading" of state policies they have had a big effect on the anti-bullying movement.
Even to the teacher groups, they are interested if you can make them relate to the data. For instance I present school finance data (U.S. Census/NCES) when teaching school law sometimes. But I don't just link to the NCES report or put one of their complex tables in a powerpoint, I isolate a couple of districts where I have students in my class from and show how the school finance policies affect those particular districts and why one of my students is probably getting paid more than another. So, I think they can be receptive if we can make it understandable and personal to them.
Also, I don't think we have really tapped the power of this yet. I think the key to this from my standpoint is the integration of simple data analysis, good presentation, and Web 2.0 (live) technologies. The Poblano model is really the goal here in my mind. In the near term, we could use Google Spreadsheets and embed gadgets in our sidebars that are automatically updated as the data changes. Once a month we update the data and the gadgets change to reflect that. It is not a perfect comparison because for politics only 1 criteria really matters, electoral votes, and you can orient your whole site around that single criteria. That is much less true in education and many criteria matter. Also, education has far fewer data collections to use. NCES repeats surveys every few years, not every few days like Rassmussen is currently polling. So there are challenges, but I think they could be overcome.
You certainly have a good point, Jon, but some folks ARE interested. I think the challenge is using the Web to make the audience large enough so that the few people that care here and there add up to a lot. Anyway, I know it is optimistic and idealistic, but I think there is something promising to be learned here.
It would have to be the right sorts of data to start andI think the data would have to be pretty macro level so as not tothreaten anyone in particular. But, I think you could do state andnational level data pretty easily without people being too upset. Youcould even do city wide data pretty easily. Check out this site:http://www.analyzedallas.org/AnalyzeDallas/Default.aspx With your knowledge you could easily put something like that together for Richmond (and maybe get a grant too).
As the technology becomes easier, which it will, the datasets can become more and more complex and the integration of geospatial representations of that data will become easier. (Probably, eventually it will all be capable of being plotted on a google map!) I think the technology is going to move pretty quickly on this front, though. As Google Docs and stuff like that become more popular, which they are, new applications are going to be developed all the time. Already I am amazed at what is possible with gadgets coming out of a google spreadsheet. You can even embed pivot tables on a website so that users can create their own data right there on your homepage! It is like an excel document embedded in your sidebar. This online user created content is gonna blow traditional stuff like Excel away. Already, google has motion graphics as you saw in my earlier post. Think how cool that would be to embed on your sidebar. Watching as school budgets increase or decrease relative to inflation over time? I could get into that. Think if we could do that on a school district by school district basis? Each school district website having motion graphics showing how things have been playing out over time.
It is one thing to require schools and states to keep all this data, but it is quite another to present it so that people are actually interested. I think we have been doing a pretty good job keeping data (what is available and going unused at the state is appalling), but now we need to focus on presenting that data better.
I willprobably post on this more in the next few days. I want to think aboutit more, but I am convinced there are possibilities here for data andWeb savvy folks interested in education. But, it faces the problem you brought up a while ago ... what's gonna count for tenure? None of this stuff translates well into dusty journals on bookshelves! AHHH!