Reminding myself how beautiful statistics is

As I think I’ve mentioned here before, my partner is taking a biostatistics course and thereby reminding me of how much I loved this stuff. And I’m reminded of the Galton quote about the Central Limit Theorem:

I know of scarcely anything so apt to impress the imagination as the wonderful form of cosmic order expressed by the “Law of Frequency of Error.” The law would have been personified by the Greeks and deified, if they had known of it. It reigns with serenity and in complete self-effacement amidst the wildest confusion. The huger the mob, and the greater the apparent anarchy, the more perfect is its sway. It is the supreme law of Unreason. Whenever a large sample of chaotic elements are taken in hand and marshalled in order of their magnitude, an unsuspected and most beautiful form of regularity proves to have been latent all along.

It’s not only beautiful, but it’s obviously extremely useful. Yet, given how often I’ve failed to explain how a random sample of a couple thousand people can adequately capture the political views of a nation of 318 million, clearly there’s something mysterious and objectionable about it. For that matter, given how many people took umbrage at Nate Silver’s election forecasts, even though basically all he did was average poll data, it seems like this antipathy to statistics is pretty widespread; statistical laws explain exactly where, and under what conditions, you’d expect individual chaos to yield collective order, yet people really seem to recoil from the thought that their collective actions might be rule-governed.

It really does often feel like I’m possession of a kind of occult knowledge that everyone could learn but few choose to. And I’m nowhere near the level of statistical knowledge that I want to attain. Even just the bit of probability and statistics that I know is enough to resolve a lot of mental muddle.

One of the clearest things about measure theory I’ve ever read

Check out Terry Tao’s measure-theory book, starting with ‘let us try to formalise some of the intuition for measure discussed earlier’ on page 18, through to ‘it turns out that the Jordan concept of measurability is not quite adequate, and must be extended to the more general notion of Lebesgue measurability, with the corresponding notion of Lebesgue measure that extends Jordan measure’ on p. 18.

I’ve understood for some time that there’s a notion of “non-measurable set”, and that you want your definition of ‘measure’ to preserve certain intuitive ideas — e.g., that taking an object and moving it a few feet doesn’t change its measure. I didn’t understand that there was any connection between non-measurability and the axiom of choice. Tao’s words here are some of the first that have properly oriented me toward the problem that we’re trying to solve, and the origins of that problem to begin with.

My partner is taking a biostatistics course, which is reminding me of how much I loved this stuff at CMU. I’m inclined to find a course in measure theory around here. We have a university or two.

Oscar Wilde, The Picture of Dorian Gray

A painting of a young gentleman

This was my first foray into Oscar Wilde, and it was delightful. The book is an excellent meditation on the relation between art and life; but if it were only that, it would be boring indeed. So it’s about equally split between that and scenes of building tension that culminate in some scenes of jaw-dropping horror. I was not expecting the latter. I was expecting mostly Victorian material of the sort that Eddie Izzard described (summarizing the Merchant-Ivory movies) as “Room with a view and a staircase and a pond.” To the contrary, it was actually a page-turner. I wasn’t expecting that.

The basic story is that Dorian Gray is the sort of exquisitely beautiful creature that Plato would have taken as his sexy boy-servant and taught the ways of the world; the earlier parts of the book feature a fair bit of innuendo around Dorian’s ruby-red lips and so forth, which I imagine were fairly titillating when Wilde’s book came out in 1891. The painter Basil Hallward, when we meet Dorian, has seated the boy for a number of sessions, taking Dorian as his muse. Basil’s friend Harry Wotton, being one of those English gentlemen of leisure who spend their days careening from luncheon with the duchess to a cocktail party to the opera, hangs out with Basil and Dorian and drops apothegm upon apothegm about the proper conduct of a life. Should a man be ethical and good and decent? Harry generally finds decent people the most boring, and advocates for sucking the marrow out of life: when you’re young and beautiful, as Dorian is, sin as much as you can. You’ll have time enough to be decent when you’re dead. Harry rejects conventional morality; he’d much prefer to live every moment to its fullest, consequences be damned. Dorian takes this to heart.

One moment Dorian is engaged to be married to a young, exquisite actress. The next moment is just perfectly framed: the next night after he’s proposed to her, he goes to see her on stage, and all the art has drained from her performance; she is atrocious, and most of the audience has left by the time she’s done. When he confronts her about this after the show, she gushes that she now sees that all art is fake, and she wants only to live a beautiful real life with Dorian. He, meanwhile, has sworn himself to a life that is nothing but art; seeing his formerly beloved as the wretched actor that she’s become, he casts her aside, rending her heart in two. You might say that he’s in pursuit of truth through the Platonic forms, and has given up on vulgar reality, while she’s done just the opposite. His rejection of her leads her, that very night, to kill herself in one of the ghastly ways that women in 19th-century novels did (see Anna Karenina and Madame Bovary).

Initially Dorian is shocked. In his shock, he goes to examine the portrait that Basil had painted of him, and he sees that the portrait has ever-so-subtly changed. The mouth has become noticeably more scornful and … evil, while Dorian himself remains as perfect as he ever was. And as he ages throughout the novel, descending into a more and more hedonistic life, paying less and less attention to the destruction he wreaks on everyone around him, the painting becomes more and more grotesque while real-life Dorian still bears the physical perfection of a naïve and unsullied 17-year-old. He jealously hides the painting where no one will see it, in a locked attic to which only he has the key. His soul, which is on display in the painting, blackens, while the man himself is physically as flawless as ever.

There are interesting bits in here that you might call “philosophical” if you were into labeling such things. For instance, the moment when Dorian decides to worship art over life is the moment when the art depicting Dorian comes to be the only source of reality in Dorian’s life. What is art, anyway? And what does the artist depict? What should the artist depict?

Hard to know how much to blame Dorian’s descent into metaphysical ugliness on his friendship with Harry, and his absorbing Harry’s sinful teachings. Harry appears throughout the book, watching Dorian’s debauchery with (we envision) a slight smirk. Harry somehow seems above the fray. He can’t be too upset about anything, because his cynical eye has already foreseen the decline and fall of everything, and the true grotesque nature that lies inside most men. Dorian becomes the sort of dissolute, revolting creature whom respectable people cross the street to avoid, while Harry remains admitted to all areas of polite society. That may be the part of the book that mystifies me the most: Harry is Dorian’s teacher, and to all appearances Harry is satisfied with the progress of his student. Yet the student turns evil in ways that the teacher never would.

All told, it’s an engrossing book: thought-provoking and absolutely gripping. After 100-some years, you don’t really need me to tell you to go read Wilde’s novel; nonetheless, you really should.

I’m confused about what sin Amazon is supposed to have committed

I don’t have time to write about it right now, but Matt Yglesias’s post today on why calls to fight Amazon’s ‘monopoly’ are misguided did hit the mark. I wanted to write something the other day when John Gruber predictably snarked in favor of the Justice Department fighting Amazon’s ‘monopoly’.

There’s no there there, seriously. I’ve been waiting patiently for someone to make a good case that Amazon has done anything wrong. Seems to me that their worst sin is … negotiating very hard against publishers? And using their market power to demand lower prices? This is good for readers, isn’t it? It makes books cheaper. Maybe you could argue that something which is good for readers is bad for authors, but that requires argument; it can’t just be asserted. I had this same problem with George Packer’s argument against Amazon a few months back.

To put it in perhaps a few words: whatever Amazon is guilty of, Wal-Mart is guilty of too. And I don’t see anyone pushing to break up Wal-Mart. They’re both just large retailers pursuing high volume and low profit margins, perhaps at the expense of their suppliers. That’s all. What am I missing?

George Packer, The Unwinding: An Inner History of the New America

Tattered American flag

I’m familiar with two George Packers. On the one hand there’s Condescending, More-In-Touch-With-The-People-Than-Thou George Packer, who came to us in Central Square and Assassins' Gate. In Assassins' Gate we see Packer very publicly agonizing over his support of the Iraq War, lecturing at the rest of us who knew from the very beginning that it was a lie delivered to us by criminals. In Central Square, Packer works with the homeless in my beloved neighborhood, and spends a couple hundred pages telling upper-middle-class white people that they’re doing it wrong.

His heart is in the right place. At his best — in Blood of the Liberals, for instance — he wants to understand why people have turned away from liberalism, and why they would support something like the Iraq War. At his best, he spends his time with people who disagree with him. At his best, he tries to remind the rest of us what the real problems are that liberalism needs to solve (rampant income inequality, the disappearance of good jobs), and explains why ordinary people believe that liberalism has lost touch. At his worst, he doesn’t realize that we’re already thinking about this, and spends his time lecturing us while we all reply, “We know, George, we know.”

The Unwinding is by Good George Packer. While it’s actually impossible for him — for anyone — to avoid inserting an authorial voice into a book like this, Packer basically stays out of the way and lets his characters talk. He interviews a single mother in collapsing (collapsed) Youngstown, Ohio; an entrepreneur (who’s also, maybe, possibly, kind of a crank) in the South who’s trying to combat peak oil with his Next Big Thing based on canola oil; a whole host of folks in Tampa, who ride home prices up and fall down just as catastrophically when the bottom falls out of the market; Jay-Z (sic); Oprah; Elizabeth Warren; and Jeff Connaughton, self-described one-time Biden Guy and author of The Payoff: Why Wall Street Always Wins.

Each of these people has something to say about the structure of today’s United States. While Packer is a little angry at the Oprahs and Jay-Zs of the world for their unbridled materialism, I think he sees them more as instances of a bigger problem. There’s such desperation in the U.S. to get a good job and do right by your family, and there seem to be so few opportunities to make it, that people latch onto whatever impossible roads to riches they can find: flip homes, have Oprah toss some baubles your way, and be a big player like Jay-Z who can raise his middle finger at everyone while the money rolls in.

It’s sort of a bleak story, with no real good answer at the end. There are bits of hope, like Elizabeth Warren, or the entrepreneur who, despite all evidence to the contrary, jumps out of bed every day convinced that today’s the day he strikes it rich and changes the world for the better. It was sort of a half-hearted optimistic ending for Packer; I think he’s actually pretty sad about the state of the world. And I don’t know that he has any answers, other than to find people who love their country and who want to do right by it.

I don’t get any great morals out of The Unwinding. In fact I find the exact opposite of great morals: Packer tries hard to let everyone speak without interruption, to the extent that he even lets their verbal tics (e.g., “frickin'”) slip through. And every time someone says something that’s probably false, Packer lets it through. These are just individuals, speaking their minds. This is a book about a problem; it’s a portrait of a country. If you’re into that sort of thing, this one is quite good. In 50 years, people will read this and get a very sad — though very true — portrait of what life was like for a lot of Americans.

Steve Martin, Picasso at the Lapin Agile and Other Plays

Steve Martin's head poking out between what look like wooden-floor slats, with red curtains opening around his head and a pencil leaning against his right temple. He's a playwright, i.e.

Do you not know Steve Martin? You should know Steve Martin. He is a brilliant comedian; go listen to Comedy Is Not Pretty! or Let's Get Small.

Once you’ve listened to his albums, maybe you’ll want to know how he came up with material that still feels fresh and weird 37 years later. For that, go read his fascinating essay in Smithsonian Magazine.

Not enough? Okay, go listen to him play banjo; you get some of that on his comedy albums. Or you can listen to a full album of songs.

Or see him in funny films like L.A. Story, or somewhat dramatic ones like The Spanish Prisoner.

Or, finally, read the genuinely moving plays in Picasso at the Lapin Agile. Or see them on stage, if you’re that lucky (I’ve not yet been that lucky).

I don’t understand how one man can be that talented in that many things. He probably just works really, really hard. Also, he’s a genius.

Fuck that guy.

In lieu of a proper review of The Lives of a Cell, I hereby include a few beautiful quotes from this book, which you should most definitely read

Sort of conceptual drawing: what looks like the outlines of a cell, with everything floating in it -- everything from whales to amoebae.

(I transcribed everything below with Siri, by the way. Siri is amazing. She occasionally makes mistakes, so blame her for them, then blame me for not doing a better editing job.)

Fascinating:

From time to time, certain termites make a convulsive movement of their mandibles to produce a loud, high-pitched clicking sound, audible ten meters off. So much effort goes into this one note that it must have urgent meaning, at least to the sender. He cannot make it without such a wrench that he is flung one or two centimeters into the air by the recoil.

Beautiful:

There are, of course, other ways to account for the songs of whales. They might be simple, down-to-earth statements about navigation, or sources of krill, or limits of territory. But the proof is not in, and until it is shown that these long, convoluted, insistent melodies, repeated by different singers with ornamentations of their own, are the means of sending through several hundred miles of undersea such ordinary information as “whale here,” I shall believe otherwise. Now and again, in the intervals between songs, the whales have been seen to breach, leaping clear out of the sea and landing on their backs, awash in the turbulence of their beating flippers. Perhaps they are pleased by the way the piece went, or perhaps it is celebration at hearing one’s own song returning after circumnavigation; whatever, it has the look of jubilation.

Funny:

My mitochondria comprise a very large proportion of me. I cannot do the calculation, but I suppose there is almost as much of them in sheer dry bulk as there is the rest of me. Looked at in this way, I could be taken for a very large, motile colony of respiring bacteria, operating a complex system of nuclei, microtubules, and neurons for the pleasure and sustenance of their families, and running, at the moment, a typewriter.

Arguably confusing software and hardware:

According to the linguistic school currently on top, human beings are all born with a genetic endowment for recognizing and formulating language. This must mean that we possess genes for all kinds of information, with strands of special, peculiarly human DNA for the discernment of meaning in syntax.

Fascinating:

Lymphocytes, like wasps, are genetically programmed for exploration, but each of them seems to be permitted a different, solitary idea. They roam through the tissues, sensing and monitoring. Since there are so many of them, they can make collective guesses at almost anything antigenic on the surface of the earth, but they must do their work one notion at a time. They carry specific information in their surface receptors, presented in the form of a question: is there, anywhere out there, my particular molecular configuration? It seems to be in the nature of biologic information that it not only stores itself up as energy but also instigates a search for more. It is an insatiable mechanism.

Lymphocytes are apparently informed about everything foreign around them, and some of them come equipped for a fitting with polymers that do not exist until organic chemist synthesize them in their laboratories. The cells can do more than predict reality; they are evidently programmed with wild guesses as well.

In an essay that asks, basically, where all the dead bodies are if everything is dying all the time:

If an elephant missteps and dies in an open place, the herd will not leave him there; the others will pick him up and carry the body from place to place, finally putting it down in some inexplicably suitable location. When elephants encounter the skeleton of an elephant out in the open, they methodically take up each of the bones and distribute them, in a ponderous ceremony, over neighboring acres.

Really? That is amazing.

Paul J. Nahin, An Imaginary Tale: The Story of √-1

This is just a delightful book, a prequel to the equally delightful Dr. Euler's Fabulous Formula. It’s got the same approach, and much of the same subject matter, as its sequel. The method is to show you a bunch of cool math, to experiment, to avoid complete rigor in proofs as a way to drive the reader forward. It’s an exciting book.

An Imaginary Tale tells the story of √-1 historically. At many points throughout history, people could have made great strides in mathematics if only they had been willing to treat √-1 as a real thing — as something no more or less real than, say, π or e. Recall that the existence of irrational numbers led to the dissolution of the Pythagorean cult: an isosceles triangle with two legs of unit length has a hypotenuse of length √2, and the proof that √2 is irrational is trivial. When your worldview is centered around the belief that integers and their ratios are fundamental to the structure of the universe, and yet you can easily demonstrate that some basic operations lead to irrational numbers, you’re going to have problems.

Yet we acknowledge that irrational numbers “exist”, even though in some sense they’re fictional. (This gets to a whole discussion about mathematical existence, which I’m going to bypass for now.) In part that’s because they’re useful. And it turns out that √-1 is incredibly useful as well; many theorems about real numbers (which is to say, rational or irrational numbers) turn out to have short, elegant proofs by way of the number i. Indeed, throughout Nahin’s book I wondered why I didn’t learn about complex numbers as a natural part of undergraduate or even high-school calculus.

The quickest way to appreciate √-1 as a useful tool is to treat it as a rotation operator. Multiplying a vector by i is, geometrically, equivalent to rotating the vector counterclockwise by 90 degrees. But to appreciate this, it’s helpful to first picture the complex numbers as a plane, with the real part on the x-axis and the “imaginary” part on the y-axis; we now call this the “Argand diagram”, though Nahin shows that any number of other mathematicians were very close to making the same discovery.

Having made the discovery, mathematicians were off and running, and so is Nahin. Just like Dr. Euler's Fabulous Formula, An Imaginary Tale is filled with fun little discoveries of both a mathematical and an engineering sort (Nahin is an electrical engineer by profession). He gives you just enough of an introduction to, say, the Euler product formula to not be scared of it, and to say, “Oh, that’s all that was?” And you feel always like you’re in the presence of childlike glee; Nahin is genuinely excited about what he’s teaching you.

There’s something of a thread connecting a few of the books I’ve read recently: the two Nahin books, The Mathematical Experience, and the biography of the geometer Donald Coxeter. Mathematics took a turn for the formal in the 20th century, under the influence of (inter alia) Bourbaki. All the formality probably purged mathematics of all doubt, but it might have lost something — intuition or geometric understanding or even “a sense of play” — along the way. This thread of popular math writing tries to restore that sense of play; Paul Nahin is one of its greatest practitioners.