In Which I Cite My Sources in an Attempt to Deflate the Hot Air from the Teacher Quality Debate

There is so much hot air in education reform, and it's extremely frustrating when one's arguments and supporting research are misconstrued. 

RiShawn Biddle writes the following:

A penchant among far too many education writers who embrace the Poverty Myth of Education is to oversimplify the debate over the role of education in stemming the long-term effects of poverty. First, they argue that school reformers proclaim that education is the sole solution for economic development in poor communities — even though no one ever says this. Then they argue that education can’t possibly be either the long-term or short-term solution for poverty — and find some flimsy data or examples to back it up.

Dana Goldstein of the Nation weakly pulled this funny trick earlier this month in her review of Steven Brill’s Class Warfare, trotting out the famed Coleman study to argue that high-quality teachers cannot help students overcome the consequences of economic poverty. Of course, she ignores the Coleman study’s conclusion that teaching accounts for nearly half of in-school effects on student achievement, and that it concludes that if teaching is of high-quality, schooling will be a bigger factor than socioeconomic background — and conveniently bypasses Brill’s ultimate conclusion that good-to-great teachers alone can’t improve education or stem poverty. 

This is a complete and utter misrepresentation of my essay on Class Warfare. First of all, I never mention the 1966 Coleman Report, whose research I consider too old to be reliable or definitive. Because my piece appeared in print, there are no hyperlinks embedded to my sources, so let me elucidate exactly how I came to the conclusion that differentials in teacher quality account for about 15 percent of student achievement outcomes, with much of the rest of the achievement gap attributable to students' socioeconomic status:

  • This 1998 paper by Eric Hanushek, John Kain, and Steven Rivkin–economists who support free-market education reforms–concludes that teacher quality accounts for "at least 7.5 percent" of student achievement outcomes.
  • This 2004 review of a number of studies on teacher effectiveness estimates that between 7 and 21 percent of student achievement differences can be attributed to teachers. See pg. 240. The authors are education researchers at Northwestern, the University of Chicago, and Tennessee State.
  • On pages 3 to 4 of this 2002 University of Pennsylvania study, the researchers conclude that depending on the valued-added method used, between 4 and 28 percent of student achievement gaps can be attributed to differences between classrooms within a school, of which the teacher would be the most significant. 

As you can see, by estimating teacher effects at 15 percent, I've interpreted the research consensus quite generously. Matthew DiCarlo, a sociologist with the Shanker Institute, has looked at this same body of research and concluded that another 20 percent of the causes of student achievement gaps are "unobservable" (ex; differences in innate intelligence, statistical error, other mystery causes); and that the rest, about 60 percent, can likely be explained by all the myriad factors associated with socioeconomic status. These conclusions are based on research far more sophisticated and current than the Coleman Report, much of it conducted by the education reform movement's own favorite economists and teacher quality experts.

Nor do I claim, as Biddle writes, "that high-quality teachers cannot help students overcome the consequences of economic poverty." To the contrary, in the essay I state outright that high-quality teachers can and do have an enormous impact on student achievement. I write:

Brill and the accountability crowd are correct to note that high-performing teachers are consistently able to raise the test scores of even the poorest children. Research shows that an improvement of one standard deviation in teacher quality leads to approximately two to four points of gain for a student on a 100-point test in reading or math. Five years of great teachers in a row, therefore, could raise a student’s test scores by ten to twenty points. Whether this potential growth is incidental or transformative depends on where a student starts out: if he began at the twentieth percentile in reading, he’d still be failing; a jump from the seventieth percentile to the ninetieth could make him a candidate for selective colleges.

My source for this paragraph was the research of Columbia University economist and teacher quality expert Jonah Rockoff.

Lastly, Biddle accuses me of bypassing "Brill’s ultimate conclusion that good-to-great teachers alone can’t improve education or stem poverty." Actually, I made every effort in the piece to sketch out the evolution in Brill's thinking over the course of his book. While Brill ultimately became more critical of Teach for America and the 24/7, manic work culture of some charter schools, he maintains throughout the book that good teaching can overcome poverty. Writing about the research of economists Kane and Staiger, whom I cite above, Brill concludes: “A snowballing network of education reformers across the country…were producing data about how teaching counted more than anything else. … It wasn’t that poverty or other factors didn’t affect student performance. Rather, it was that teacher effectiveness could overcome those disadvantages." (emphasis added)

Actually, Kane and Staiger's research does not show teacher quality counting "more than anything else;" it shows teacher quality counting, in their own words, for "at least 7.5 percent" of a student's achievement outcomes. 

Apologies for how detailed and wonky this is, but I did feel the need to respond to Biddle. It's ridiculous to claim that I think school reform doesn't matter or doesn't have a role to play in fighting poverty. If I believed that, I wouldn't have chosen to devote my professional life to this topic. If I thought schools didn't matter, I wouldn't spend time visiting schools to report on the awesome things they are doing to give poor kids a leg up. 

Back soon to your regularly-scheduled, non-confrontational programming…

7 thoughts on “In Which I Cite My Sources in an Attempt to Deflate the Hot Air from the Teacher Quality Debate

  1. CohenD

    Don’t worry, Dana. I don’t know if anyone takes Mr. Biddle very seriously – I hope they don’t. He’s a serial “inventor” – he’ll invent your position for you in order to lodge his complaints about an issue. Off the top of my head, I can tell you he’s done it to me, to Larry Ferlazzo, and to Robert Pondiscio. (I’m not going to bother citing the specific blog posts – they should be rather easy to find if you search Dropout Nation with such specific search terms).

    Reply
  2. Typically Non Confrontational

    He not only invents your position but from my observation he’ll also invent what motivates you to state your position! Why? I don’t know because I don’t get into speculations. That’s just what I’ve observed in the past 18 months.

    I enjoyed this article and learned from it. I like fact vs fiction.

    Reply
  3. Stuart Buck

    The parsing out of what causes student achievement seems very dubious. What if part of the way that socioeconomic status leads to higher achievement is that parents use it to buy houses in school zones with . . . better teachers? Seems very likely, but there’s no way to tell with the usual models.

    One way we could figure out how to divvy up responsibility would be to get 500 rich kids and randomly assign half to attend a school with teachers identified as horrible (but otherwise keeping everything else about the school the same, such as peers or school spending), and then compare them to the other rich kids who got to attend their regular school. Then you’d really be able to see how much rich kids were benefiting from being able to buy access to good teachers.

    But you’d never be able to do such a study — no one would sign up.

    Reply
  4. Dana Goldstein

    Stuart, yes I agree it’s very difficult to determine what factors affect student achievement and how much. Nevertheless, these estimates and studies are the best work that’s been done on the subject. It’s also important to point out that all these studies measure student achievement according to test scores, but of course there are other possible measures: high school graduation rates, college attendance, college completion, employment status, etc.

    Reply
  5. Alice Mercer

    Dana,
    I’ll second Mr. Cohen’s comments on Biddle, who is a bloviating buffoon, and is not worth efforts such as this. But, I want to thank you because even though doing this for your detractor is truly casting pearls before a swine, this is a gem of information on teacher effects, and I for one find it very informative and useful. I was recently in a discussion about teacher effects with colleagues, and I’m going to copy it to share with them since many were quoting higher effect levels provided by…consultants urging us onto greater academic achievement (that would be…test scores).
    As a teacher who has been around for over a decade (and most of that in “failing” schools) I would say the point that studies like these can’t express is that they are talking about averages, and aggregated numbers. When you are teaching, no matter how poor the school, you will have some outstanding students (you’ll also have some kids struggling at schools in the best neighborhoods too). You can’t write off a whole school or a whole class. Also, some kids will have a good year, and some will have a bad year. Sometimes things click and you will overcome whatever is happening in the rest of their life. I admit, I can’t predict who will bloom, and who will wilt, so what I try to do is have the humility not to write a kid off, and to ask myself what I can and could do better. But I also have to have the humility to realize that sometimes I’m not going to overcome what’s happening outside my classroom for every kid. There is an overwhelming arrogance in saying you can work miracles. I can be pretty arrogant myself, but even I don’t have those kind of huevos.

    Reply
  6. Robert D. Skeels

    Did you really need to respond to a highly discredited right wing libertarian so self-colonized that he was once fired for racism?

    link to indianapolisrecorder.com

    The only folks that take Biddle’s blather seriously are the hard core privatizers at places like the astroturf Parent Revolution. They actually quote Biddle’s “work” as if it’s something from the NEPC, when it’s nothing more than rehashed Rand/Freidman fantasies.

    Glad you posted such an impressive array of sources. May I suggest Helen F. Ladd recent paper from Duke Sanford School of Public Policy entitled “Education and Poverty: Confronting the Evidence” which pretty much puts Biddle’s specious arguments to rest.

    link to sanford.duke.edu

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>