(__Attention-conservation notice__: 1,600 words on why it may be a good idea not to read cranks. Also some words on academic orthodoxy. Scattered thoughts on building institutions for seeking the truth.)
My friend, who blogs pseudonymously and will hereafter be known as “PB” (for Pseudonymous Blogger), takes me to task for suggesting the existence of cranks. (Note that I didn’t dismiss any specific people out of hand.) PB has long invited me to read the heterodox folks that he follows, including some “with a large collection of old John Birch Society literature.” As my readers know, I tend to read more from the academic wing; PB attacks academia like so:
> Academics in the social science to do not get fired or demoted if they get things wrong. They do not get additional grad students if they are right. The grad school and peer review process reward one thing – conforming to the current intellectual fashions.
I believe I’ve heard about other places where leaders reward servants less for their objective correctness and more for hewing to what the leaders believe. I believe that’s called *every single human institution ever*.
I tease. If the problem is as PB describes, it’s an institutional incentives problem, and the question is how to build better institutions. Think of the various institutions we have in this world whose purpose is (ostensibly, anyway) to seek out truth. We have juries, to find truth in the legal realm; academia, to find it within more abstract domains; the media, to find what our leaders are hiding and bring policies to the broader public; and many others. Every one of them is guilty in some way of confirmation bias. Cass Sunstein, who’s now the head of OIRA, is famous for documenting how groups of people who believe the same things and only speak with one another are likely to arrive at a more-extreme conclusion than if they had some dissenters in their midst. He’s also famous for taking this line of research and using it to suggest that the Internet needs “general-interest intermediaries”, like the [newspaper: New York Times] and the [newspaper: Wall Street Journal], to help soothe society’s emergent extremism. Of course it’s an open question whether the general-interest intermediaries serve the purpose that he thinks they do.
PB focuses on a couple successes from his heterodox sources, but recall the proverb about stopped clocks. The question is how well an institution works overall. We statisticians talk about “type I” and “type II” errors, or “false positives” and “false negatives,” respectively. A false positive, in the context that PB and I are talking about, is when you identify something as true when it’s false; a false negative is when you identify something as false when it’s true. Suppose, for instance, that I adopt as a decision rule that I will never read anything written by someone who’s been a member of the KKK. I may well reject some smart writers because my rule is too crude; these would be false negatives. The basis for my rule is that I expect most of what the KKK member utters to be false; by rejecting KKK writers out of hand, I’m trying to minimize my rate of false positives (again, accepting something as true when it’s in fact false). There are costs associated with false positives, and costs associated with false negatives. To compute the total expected cost of a decision rule, multiply the cost of a false positive by the probability of a false positive, and add to it the cost of a false negative times the probability of a false negative. Going along these lines eventually gets you to the Neyman-Pearson Lemma, which is fundamental to statistics.
Rejecting too many people as cranks may give you a high rate of type-II errors: you may reject some good people out of hand. If PB’s right, mis-labeling cranks is *also* likely to give you too high a rate of type-I errors: you’re just confirming the conventional wisdom, which has a terrible track record. If I’m reading PB right, then, his claim is that academia’s error rate is terrible in both directions, hence “dominated” in the game-theoretic sense. My response would be twofold: first, find me an institution that balances type-I and type-II errors better than academia. This isn’t a rhetorical question; if there is such an institution, I’d like to find it. But the point is not to focus on isolated instances where someone predicted something better than someone else; the point is to look at overall error rates in both directions. Second, I’d ask PB to suggest institutional improvement that would make academia — or juries, or the media, or pick-your-favorite-institution — do its job better.
Based on what PB wrote, I suspect we’d both look for changes in the incentive structure. If it’s empirically true that academia hires on the basis of confirming what the incumbents already believe, how do we change that? To pick one example out of the air: is there any way to make academics put their money where their mouths are? The examples PB cites from macroeconomics, for instance … is there any way to make Ben Bernanke suffer financially if the economy goes south and benefit if GDP rises? You can look to what corporations do — stock options, for instance — to put some skin in the game, but we also know all the sorts of gaming that go along with those incentives. Unless you structure them properly, you have the epidemic of “I’ll Be Gone, You’ll Be Gone.” Structuring incentives is an incredibly nontrivial problem. To pick but one book on the subject out of the air, take a look at [book: Managerial Dilemmas: The Political Economy of Hierarchy]. Or, from another angle, read Herbert Simon’s paper on “The Proverbs of Administration”, wherein Simon notes that most every managerial proverb has an equal and opposite proverb that gets thrown around just as confidently.
So in short, I’m not at all confident in my ability to construct incentives that reward the right behavior within institutions, and I certainly don’t feel as though, if I were made Dictator of Academia, I could build better incentives than those that are already in place.
Also, I’m fairly convinced that PB is just empirically wrong about the ideological homogeneity of academia. I think he may be confusing what happens *within one institution* with what happens *in the academy as a whole*. Does PB really contend that the University of Chicago and Princeton University are hiring the same economists? No, of course not: they argue bitterly. Just look at Princeton’s Nobel laureate Paul Krugman denouncing the U of C’s Nobel laureate Ed Prescott. Or look at a good century of arguing in statistics over whether we ought to be Bayesians or frequentists. And that’s in statistics, where empirical and mathematical confirmation are, at least in principle, much more readily available than in the social sciences or the humanities. I’m curious what PB’s standard for homogeneity is. Are academic disciplines homogeneous whenever they avoid pistols at dawn?
If academia is argumentative, it may well be so because the incentives encourage it. Judge Richard Posner, in [book: Public Intellectuals: A Study of Decline], argues that academics have every incentive to be contentious — at least within the public sphere — because it gets you attention when you reject the status quo. I see very few books entitled [book: Most Everything You Know About The World Is Basically Correct].
All of that said, PB and I would surely agree that the conventional wisdom often gets things disastrously wrong. To take but one example, you can look at conventional views of market regulation. From the New Deal through World War II and up until the 1960s, the conventional wisdom — which took canonical form, perhaps, in the great Paul Samuelson’s 1948 textbook — was that the goal of economics was to control markets toward desired ends. Eventually the conventional wisdom switched to the idea that markets were best left on their own. You can argue both sides of this — and, importantly in this context, academia *has* argued both sides of it, continuously, for half a century. What made the switch happen? Well, it’s complicated, but surely a part of it is that it’s convenient for businessmen to argue that they’re best left unregulated. They were going to argue this anyway; academic economics just offered them some tools. But a whole set of entirely orthodox economic results says something quite different: what individual actors do rationally on their own can lead to a disastrous, unwanted result in the aggregate. You can look anywhere within orthodox economics for confirmation of this idea (see Bowles, [book: Microeconomics: Behavior, Institutions, and Evolution]; and Schelling, [book: Micromotives and Macrobehavior]). The problem probably isn’t academic rejection of heterodoxy; it’s that economics can be used as a tool of ideology in a more direct way than can mathematics, so it *is* used as such a tool.
Of course PB is right that there were big glaring warning signs that we were in an unsustainable bubble. Dean Baker flagged a lot of these in [book: Plunder and Blunder]. Lots of very intelligent keepers of the conventional wisdom, like Bernanke and Greenspan (Ph.D.s both), who should have known better, got it wrong. All this tells me is that, when the economy’s booming and lots of people are making money, it’s very hard to be the guy who (as the conventional saying goes) “takes the punch bowl away.” Now that everything’s collapsed, we’ll have more people honoring the conventional wisdom. The wisdom was always there; the will to follow it was not.
As for PB’s generous invitation to read along with him on one or more topics: it’s a generous offer, but take a look at how much other stuff is either in my queue or sitting on my floor, tempting me. Add to that a chapter-by-chapter read of Adam Smith, a heretofore-unnannounced chapter-by-chapter read of Gerard Debreu’s [book: Theory of Value], and a couple bits of big news that I’m waiting to fully ferment before I mention them here; the result is that I don’t have the time to read in what sound like fascinating areas. But I appreciate the offer.
Ooo, big news, you say? Can’t wait. Hope it’s a gig writing reviews full-time.
LikeLike
“I believe Ive heard about other places where leaders reward servants less for their objective correctness and more for hewing to what the leaders believe. I believe thats called every single human institution ever.”
Every institution is has the problem to some extent. To the extent an institution has the problem it is dysfunctional. My claim is that academia has the problem to a highly dysfunctional degree.
And to the extent that professors get rewarded for results rather than conformity, the result that matters is winning grants. And on what basis are grants awarded? Well, mostly ideological conformity to the grant maker.
“PB focuses on a couple successes from his heterodox source…”
A couple? I gave you 14 different topics, on issues of great importance, on which I think the heterodox views are much more accurate than the conventional wisdom. The two examples I gave in the beggining of the essay were just to whet the appetite.
“My response would be twofold: first, find me an institution that balances type-I and type-II errors better than academia.”
There isn’t one.
But imagine asking in 1500, what institution has better views about god/regligion/morality than the Church? The answer is that no institution did. Does that mean one would have been wise to believe the Church’s views on God?
My point is, if you care about the truth, don’t blindly trust institutions.
“But the point is not to focus on isolated instances where someone predicted something better than someone else; the point is to look at overall error rates in both directions.”
Right. If academia was wrong on a few isolated incidents I wouldn’t have written my post. I gave you a dozen topics that I through my own research I have found that academia totally misses the mark, and I am sure there are dozens of others.
“Second, Id ask PB to suggest institutional improvement that would make academia or juries, or the media, or pick-your-favorite-institution do its job better.”
Brainstorming solutions is jumping the gun a bit. If we don’t agree on the nature and extent of the problem, we can’t even begin to talk solutions.
That said, I am certainly not lacking for ideas about institutional improvements. If you’re curious about my positive prepscriptions for government check out my principles of formalism (short version and long version) or thoughts on election design. If you’re intrigued, I can make that a full topic, worthy of a challenge: “Topic 16: 16) What kind of institution/political system reform would actually improve the overall levels of governance in America?”
For incrementental reforms to academia, it’d be swell if government funded social scientists would actually treated their work like a well run open source project. All calculations should exist in computer code checked into source control with functioning build scripts and unittets, plus, all data should be published with the project. Bonus points if the code is made in a way to make sensitivity testing easy (ie, make it easy to turn control variables on and off to see how sensitive the results are to various plausible assumptions). Of course, this would never happen because it would put 99% of social scientists out of jobs. The sentivity analysis would reveal that none of their results were meaningful.
For systematic reforms, I’d think about tearing up the grant and journal system entirely, making each university truly independent. If a university gets government funds it should be held accountable to a board selected perhaps by either philanthropists or a lottery-election.
But I don’t think the academy is at all open to any sort of major reform. What it really needs is competition. The best proposal I’ve read for creating a competing truth institution is this one.
“The examples PB cites from macroeconomics, for instance is there any way to make Ben Bernanke suffer financially if the economy goes south and benefit if GDP rises?”
I am certainly not a fan of mechanical incentivization plans. Such plans barely work for sales people, and work even less well in fields where results are more subjective. It’s also not usually necessary. I think Bernanke wants to do the right thing. He just didn’t know what the right thing was.
“Also, Im fairly convinced that PB is just empirically wrong about the ideological homogeneity of academia. I think he may be confusing what happens within one institution with what happens in the academy as a whole”
Homogenity is not my beef with academia, my argument is that in its various disputes, neither side has the truth, and thus you have to look outside of academia for the truth.
When I refer to mainstream academia sources, I’m basically mean any source that would be normally found on a course syllabus for an undergraduate or graduate course at a U.S. News and World Report top 50 university. I definitely have UChicago in mind. UChicago is certainly smoking a different strain of crack than Mr. Krugman, but it is still crack. I don’t think either reading either Krugman or the UChicago can give you an accurate understanding of economics. The UChicago folk are much more transparently stupid, though. That leads smart people like you to examine the debate and conclude that since UChicago is wrong, and Krugman has the best arguments of the various academics, that therefore Krugman is correct.
Once again I’ll go back to the church analogy. The catholic church had a huge number of internal doctrinal disputes about the nature of God, Jesus, the trinity, etc. Did this means that its views about God in general were homogenous? No – the truth about God (or the truth as you and I believe – that a personal God does not exist) – was completely outside the terms of debate of the institutions of that time.
If you lived in 1500 the only way you could have known the truth about God would have been to read the Bible yourself, look through that telescope and do your own calculations, etc. There was no institution you could trust.
Similiarly, if you want to know the truth about economics you’ll have to do your own research and non-mainstream academic sources. You won’t find the truth from Keynes or Baker or UChicago or Krugman.
“To take but one example, you can look at conventional views of market regulation…The problem probably isnt academic rejection of heterodoxy; its that economics can be used as a tool of ideology in a more direct way than can mathematics, so it is used as such a tool.”
This entire paragraph is a distorted/one-sided version of events. Samuelson and Bowles have huge systematic errors of their own. The problem is all sides of the debate. I can explain in more detail, but you’ve read thousands of pages in defense of orthodox/keynesian-esque economics, and zero pages on my brand of Sith economics, so I’m not going to convince you in a paragraph comment. I’d be happy to right up a longer essay plus reading list if you’re interested, though.
“As for PBs generous invitation to read along with him on one or more topics: its a generous offer, but take a look at how much other stuff is either in my queue or sitting on my floor, tempting me”
Hmm, “generous”, I suppose you could consider Vader’s offer to Luke a form of generosity …
Let me ask a meta-question. Clearly if you’re reading Smith and Debreu you’re interested in creating a three dimensional, full color, accurate understanding of economics, and you’re willing to put in the time reading quite laborious texts. What would convince you that gaining a full understanding of economics would be better accomplished by undertaking one of my econ related challenges (7, 8, 10 or 14)? Or are we stuck in a catch 22 – you’re not willing to read my sources since you think they’re unreliable and less worthy of your time, and you think they’re unreliable because you’ve never given them a fair chance.
LikeLike
While I do think it’s a fine thing to be able to argue everything you believe down to first principles, as you’re suggesting, I don’t know that it’s necessary. Argue for me why evolution by natural selection explains the species we see today. Argue for me why the earth revolves around the Sun.
As for why I’m in no particular rush to read “Sith” stuff: from what I’ve seen, it’s thinly masked racism.
LikeLike
Bud, you gotta put down the book and step away slowly. Does your brain hurt? Sometimes I worry you think too much.
That said, read a fun book. Just something to keep your brain from working so hard. Or read a book completely out of character for you. Read “Outlander” by Diana Gabaldon.
And yes, I fully expect an e-mail re: Fully Fermented Big News, regardless whether you get vinegar or wine.
LikeLike
If we were both interested in the topic of heliocentrism, and I knew of interesting anti-heliocentric writings, then I might forward them to you and tell you to check them out. But as it is, neither of us care about the subject, and I’ve never read anything to lead me to doubt heliocentrism. Similiarly with evolution, I’ve never read any convincing arguments that would lead me to doubt the basic thrust of natural selection.
If I’m writing material attacking a certain point of view, and then someone who seems smart and reasonable tells me, “Hey you’re getting this all wrong, you should check out sources x, y, and z”, and I’ve never read the before sources or anything like them, then I usually either a) check out the sources and explain why he is wrong b) check out the sources and change my views or c) apologize for not having the time to read it at the moment, but hold off on any further attacks until I’ve had a chance to reduce my ignorance of the issue.
Also, with regards to heliocentrism, I trust the modern view of astronomy is more accurate than the medieval view because we have GPS devices that work and they didn’t. With regards to the question of say, monarchy versus democracy, the same cannot be said. Afghanistan under King Zahir Shah seems quite a bit more civilized than Afghanistan under President Karzai. Now there a gazillion cofounding factors, but the question of monarchy versus democracy versus some other system in Afghanistan should at least be a matter of discussion. Unfortunately, the folks who fill the halls of Harvard School of Government and the State Department have no interest in the topic, and no interest in learning from the experiences of King Shah.
As for comment about racism, of my 15 topics only 3 have anything to do with race. I’m not sure what I sent you that you found so objectional, and not everything I link to I endorse 100%. Harvard has Mein Kampf on its syllabus, it doesn’t mean Harvard is endorsing Nazism. I sent you Roissy links but I’m not a manwhore who beats his girlfriend. I sent you the Roissy links because despite what’s objectional in his writing, at least 20% of what he says is a) true b) important c) almost impossible to find elsewhere. So despite his misogyny I’ve become a better boyfriend for reading reading him. I sometimes link to imperialist literature for the same reason. I may disagree with much of it, but if you want to make a prediction about how Obama’s policies in Afghanistan will work out, then reading first hand accounts about the British experience can be very illuminating.
The word racism itself has been overloaded into meaninglessness. If by “racist” you’re saying that I’m personally bigotted or meanspirited towards other people on the basis of race, then I strongly object to the charge. If by “racist” you mean that I think different racial groups have different distributions of characteristics, and that this matters for understanding the world, then I’m guilty as charged. I fail to see how this views is objectional though. Certainly, most progressives are “racist” in this sense, the difference is where the lines are drawn. You don’t stop reading Krugman because he works for an institution that systematically discriminates against Asians? You don’t shun Harvard because it gave a full scholarship to a Black Panther murderer. Progressives, like most people, believe that some forms of racism are ok, and others taboo. I draw the lines differently than progressives, and have well thought reasons for doing so. If you think my views on some specific issue violate decency or the truth, I’d be happy to pick a topic and discuss it in depth.
LikeLike
The problem with saying: “I won’t read cranks” is that you limit yourself much more than you initially suspect.
Let’s use economics, as our example, since you brought up in the following context: “Does PB really contend that the University of Chicago and Princeton University are hiring the same economists? No, of course not: they argue bitterly. Just look at Princetons Nobel laureate Paul Krugman denouncing the U of Cs Nobel laureate Ed Prescott.”
I would argue that from a historical perspective, U of C and Princeton are indeed hiring identical economists. The differences you cite – across the entire spectrum of legit beliefs about economics – are indistinguishable. If one of these economists would triumph over the other, the resulting differences would minimal – by historical standards, they’d be indistinguishable.
What I find most interesting, is that in 1900, for example, Prescott and Krugman would have been considered cranks – it’s really impossible to understate how cranky they would be considered by 1900s standards. People like you would have been saying “don’t read Prescott and Krugman, they’re crazy” (unfortunately, in my opinion, people in 1900 must have been more open-minded, but I digress).
Have we endured enough economic hardship since 1900 to call into question the prevailing change in the direction of beliefs about the economy? I think the answer is obviously yes. However, mainstream economic beliefs continue to change in the same direction.
This gets us to the crux of the problem – when you rule out cranks, you rule out history. For example, if you’re unwilling to read things by racists (taking the modern definition of the word), you really can’t read any books written prior to the 1960s. If you don’t understand why this position is untenable, I’m afraid you’re a lost cause.
LikeLike
0 Pingbacks