Why most bands suck by the time you've heard of them

Let me a describe a situation that long characterised my music listening, and see if it applies to you.

The bands I like always had one great album that was inevitably in the past. Occasionally you'd get to see them live and they'd play their old stuff, but they seemed a bit tired and past it - you never got to hear their good stuff while it was actually current.

I think I figured out a rough model of why this should be the case.

Suppose that each band has a underlying quality distribution - each time they write an album, it's an independent draw from their base quality distribution. Sometimes it's a good one (i.e. The Joshua Tree), sometimes it's a bad one (i.e. How to Dismantle an Atomic Bomb). The distributions will vary across bands in both mean and variance.Some bands have a high mean (e.g. Radiohead), some have a low mean (e.g. Nickelback). Some have a high variance (e.g. Bob Dylan), some have a low variance (Red Hot Chilli Peppers)

Friends don't let friends listen to Nickelback

Now, suppose further that there's a fixed quality threshold required in order to achieve popular success. You've got to have at least one good album to become popular. Additionally, suppose that news about bands only diffuses slowly - not everyone finds out about a band at the same time.

So how will this play out? Well, let's take the set of all bands who cross the quality threshold for the first time. Some of them will be truly talented and have a high distribution mean. Being above the quality cutoff signals a long line of good future albums (e.g. Death Cab For Cutie).

A lot of the time, however, being above the quality cutoff signals the band just got lucky with that particular album. After that, you get reversion to the mean. Think Jason Mraz or Ryan Adams. They have a couple of awesome songs ('I'm yours' and 'Desire' respectively), and whole albums full of complete crap.

This sketch of a model generates a couple of predictions:

-The average band will have their best album as their first major commercial success. After that they'll never have another good album

-The bands that have their best album as one of their later albums (Death Cab for Cutie with 'Plans', The New Pornographers with 'Challengers') will be more likely to to release subsequent good albums. These are the high mean bands, not the bands that were lucky.

-Related to the above, if a band releases a second album that's better than their first breakout album, it's a very strong signal of band quality.

-The chances that you get to hear their good stuff depends on how connected you are. If you hear about new bands quickly, you'll get to see them at their prime. If you're late in the loop, you'll always find out about them too late.

The optimal strategy in this setup is to go to see bands as soon as you can after they become popular - that's the high point, and they'll rarely get any better.

Which is a sad conclusion, but certainly sounds like the world we live in, doesn't it? Dire Straits was probably the best band in the 1980s (bar maybe U2), and yet Mark Knopfler has never lived up to the early promise since.

Headline of the Day

From Zero Hedge:

Has The Chairman Stopped Ordering Ink? Hewlett Packard Plummets After Hours On Major Downward Guidance Revision

Comedy gold!

The New Democratic Strategy

Every time a Republican state legislature threatens to pass legislation stripping public sector unions of their right to negotiate, they flee the state, thereby depriving the Republicans of quorum.

First in Wisconsin, now in Indiana.

It looks like they're taking a page out of the play book of Monty Python's Sir Robin:

"Brave Sir Robin ran away.
Bravely ran away away.
When danger reared its ugly head
He bravely turned his tail and fled
As brave Sir Robin turned about,
And gallantly he chickened out,
Bravely taking to his feet,
He beat a very brave retreat,
Bravest of the brave, Sir Robin!"

Frankly, only good things can come from this strategy. It makes them look like cowards, and likely galvanises public opinion against the unions.

But even if this doesn't happen and the legislation doesn't pass, every day that a legislature is deadlocked is one less day that value-destroying legislation can be passed. If I were the voters of Wisconsin, I'd call off the search for them and instead offer to put them up indefinitely in a swanky hotel in Illinois - it's got to be cheaper than the alternative.

Good Advice

The best advice is often not the most eloquent. It's not necessarily the advice which most concisely summarises the tradeoffs to be made, and points you in the best possible direction. Sadly, it's not even the wisest.

Often, what constitutes the most useful advice is simply that which has a way of coming to your mind when you need it most, and when you are at risk of doing the wrong thing. And in that regard, a pithy formulation can be the most helpful of all.

One of the central problems that I (and I think lots of people) face is simply that they get too caught up in their worries of the present moment. Fortunately, this is a very easy problem to solve. All you need to do is mentally take a step back, reflect on the bigger picture, and realise that most of the time the things you're worrying about don't actually matter much in the scheme of things.

The trouble is that when you get too close to your problems, however trivial, it's just a hard thought to remember.

By this metric, I find one of the best bits of advice is from Bob Dylan's song, 'Someone's Got A Hold of My Heart':

"She said 'Be easy, baby, there ain't nothing worth stealing here.' "

And it's true. For the vast majority of things, you are just better off being easy.

For some reason, it tends to come to mind in a way that wiser versions of the same thing don't always seem to.

I couldn't find the original (from the Bootleg Series Volume 3), but here's a pretty decent cover:

Where to from Libya?

Not so much in the political sense, but in the limited sense of 'where would you go if you were a dictator fleeing the country?'.

So things in Libya are looking very dicey at the moment for the Gaddafi regime. Apparently protesters now control the city of Benghazi, which is remarkable since the armed forces are doing nearly everything they can (including Air Force bombing raids) to blast the hell out the protesters and it's still not enough to hold the cities.

Who knows if this is true, but there have been reports that Gaddafi may have fled the country already.

Now, in the past, Saudi Arabia was usually the ex-dictator's destination of choice. Repressive and fairly stable regime, hear-no-evil-see-no-evil approach to other people's human rights abuses, lots of luxury stores to spend your stolen billions on - what's not to love?

What I liked the most about the story (even if it's not true) is the way it implies that even crazy dictators understand conditional probability.

In other words, conditioning on the fact that Gaddafi and the army couldn't hold Libya after the fall of Tunisia and Egypt, it becomes an open question as to whether Saudi Arabia will be next. Saudi Arabia is a great place to go when regime collapses are independent events. But at the moment, we're observing correlated regime collapses. The first two were nasty, vaguely pro-American regimes with less willingness to shoot their own people. If you're Gaddafi and it's gotten to the point that you need to flee, it shows that being anti-American and happy to shoot your own people may still not be enough. In which case, Saudi Arabia looks much less appealing than in the past.

Hence the draw of Venezuela, which is where the story claims he was fleeing to. A long way from the Middle East, leadership (I use the term loosely) with visceral hatred of America and willingness to accomodate anyone else in the same category. Hell, it's certainly where I'd be fleeing to if I were Gaddafi.

Put it this way, the fact that they chose Venezuela raises my probability estimate that the story is true (although that's still not a high number necessarily), or at least a well constructed hoax or piece of misinformation.

A pledge for science

There is one action that any airline can take that will ensure my loyalty for years. Not just mine, but I'm sure lots of other customers too. In fact, I pledge to buy airline tickets (for any price difference up to $100 in excess of an equivalent fare) exclusively from the first airline that allows electronic devices to be used throughout flight, including takeoff and landing.

The rule against them is the most ludicrous superstition ever. It's just staggering how the advanced technology that puts a plane in the sky can persist with the cargo cult lunacy that thinks that an ipod can cause a plane to crash.

Can anyone, honestly, give me a halfway plausible hypothesis as to how a non-transmitting device is supposed to interfere with a plane's navigation systems? The closest I've ever heard is 'something about electrical fields and magnets'. But that's absurd - it's not like it's an industrial strength magnet being waved near the cockpit, it's PSP being used 30m away for crying out loud. Even the arguments about transmitting devices like phones are weak to the point of being pathetic. With non-transmitting devices, they're not even trying to make a coherent case.

Proponents claim that there's anecdotal evidence that phones can interfere with navigation systems. You know what else has 'anecdotal evidence'? Astrology. Teleportation. Alien spaceships giving people anal probes. In fact, I'd wager the anecdotal evidence for the last one is several hundred times more voluminous than that in favour of electronic devices interfering with planes.

Honestly, if this is the standard to ban something, how can you establish any scientific proposition ever? You're only allowed to use things that nobody has ever told a story claiming that it happened? Anecdotally, people praying to God has fixed faulty planes. Should we mandate that too?

Consider the following examples that demonstrate the lunacy of the current rule:

-Electronic wristwatches use circuits too, but apparently these aren't able to crash the plane. Don't ask me why. They're too small, but apparently the tiny noise-cancelling device in my Bose headphones isn't. Skeptics might claim this has something to do with the impossibility of getting people to not wear watches or to remove the batteries from their watch. What would they know!

-Very few computers are shut down when people travel, they're mostly in sleep mode, a low power state in which the computer remains on. But mysteriously, this is okay too.

-TVs in the back of seats contain electronic circuits, and often remain on during takeoff. I guess they're sprinkled with magic non-interfering pixie dust.

-Pacemakers contain circuits too. Better turn that thing off, Beryl! It's for the good of everybody on the plane, you understand.

not to mention my personal favorite:

-The average plane has, what, 100 passengers? Maybe 200? Assume that 95% of them have phones. Now, what are the chances that among them, not a single one of those passengers managed to:

a) forget to turn off their phone

b) leave their iphone on, having not figured out that pushing the top button on the iphone doesn't actually turn it off

c) leave it on intentionally as an act of defiance

The chances, in short, are basically zero. Which leads us to the conclusion that virtually every single flight probably has at least one phone on during takeoff and landing, and miraculously they're not all crashing.

It's time to strike a blow for science. Take the pledge to buy from science-friendly airlines, and in the mean time, leave your ipod on as an act of defiance.

Markets Cater to All Demands, However Stupid

One of the great things about markets is when they expose the dumb and contradictory things that people believe.

When it comes to relieving pain, people have a view that essentially any change in temperature is helpful. Reducing the temperature helps - add ice! No wait, heat helps too - take off the ice and add a heat pack!

People are deeply attached to both ice and heat as methods of pain relief - it's as if the worst possible temperature is room temperature, and anything other than that is an improvement.

Now, of course, as anyone with even a rudimentary understanding of thermodynamics will tell you, something cannot be both cold and hot. This isn't even the zeroth law of thermodynamics - it's like the -1th law of thermodynamics - the same point in space can't have two different temperatures. They didn't bother to write this down, as it can be approximated by the phrase 'Duh!'.

But the heart wants what the heart wants. Enter markets, which step in to cater to people's ridiculous simultaneous demands for both heat and cold as methods of pain relief.

I give you 'Icy Hot'.

Just pause and reflect on this for a second:

IT CAN'T BE BOTH COLD AND HOT AT THE SAME TIME YOU IMBECILES!!!

Because I object to actually giving money to companies attempting to flagrantly lie to me, I haven't actually invested in one to find out if it makes the area cold, hot, or neither (because as we've already established, it's clearly not making it both). But I imagine it probably just makes it tingly, which is sort of like being cold, right?

NO! NO IT'S NOT!

I can just see the idiots at the pharmacy thinking 'Gee, I was going to buy both a heat pack and an ice pack, but now I can just buy a single pack of 'Icy hot' and combine them into one!'.

LORD KELVIN IS SPINNING IN HIS GRAVE WHEN YOU FALL FOR THIS MORONIC MARKETING!!!

Ugh.

/rant

Metaphor of the Day

From the excellent new song 'Bloodbuzz Ohio', by The National.

To describe the process of compounding interest leading to a spiraling debt:

'I still owe money, to the money, to the money I owe.'

I imagine that very few finance types would think to characterise it in this way, but it's far more evocative than, say, 'my interest expense keeps compounding higher and higher'. It often takes someone from outside the field to express an idea in language that resonates with the common man.

As I said a while back now, I think that the best description of opportunity cost is by Bob Dylan.

Dispatches from California Government

It's never nice to make light of people's deaths, but some stories just write themselves:

Apparently when you work for California governments, you can be dead for over 24 hours before anyone even notices:

"An L.A. County employee apparently died while working in her cubicle on Friday, but no one noticed for quite some time.

51-year-old Rebecca Wells was found by a security guard on Saturday afternoon.

...

The last time a co-worker saw her alive was Friday morning around 9:00 a.m., according to Downy police detectives."

It places a fairly high upper bound on the importance of the work you're doing when you can be dead for 24 hours in your workplace and nobody even notices.

The article mentions that she was a 'longtime compliance auditor', whatever that is. I'm sure if we happened to fire some of them from LA county, the unions would scream bloody murder about how crucial the compliance auditing was to the functioning of LA.

The comments to the article are gold:

Will she get overtime pay for being at her desk for 24 hrs straight without a break?

Ironically she had the highest productivity of any employee that day.

Will this be on her next performance review? Dying on the job will get you a 1 or 2 at most. This could affect her raise. Has Obama been informed? Where is the justice?

Government workers, I can hear it now, "We just thought she was taking a regular nap."

How much you want to bet she gets promoted?

Given that we can't seem to eliminate or even reduce these kind of positions in California, I guess macabre humour is the best we can hope for.

Wayne Swan - Too Stupid to be Treasurer

Wayne Swan continues to be a dangerous embarrassment to Australia.

The Labor Government tried to introduce a mining 'super profits tax' that can be best described as 'let's tax to death the one industry propping up the Australian economy.'

The original version of this tax was that if a mining company earned more than the rate of return on government bonds, the government would impose an additional 40% tax.

Students of Finance 101 everywhere thought to themselves, 'Wait, aren't mining company shares significantly more risky than government bonds? Why would anyone invest in a security with the same or lower return than a government bond, but more risk than a government bond? Won't that send Australian mining companies broke?'

Even former Labor Party ministers like Barry Cohen pointed out how dumb this was. Wayne Swan was too stupid to realise this basic economic point, but thankfully the Australian electorate (watching their superannuation funds crater) saw through it.

But in case you thought he might have learned his lesson, he's back at it again. Here's a great example of Swan-onomics.

Treasurer Wayne Swan has seized on BHP Billiton's 72 per cent jump in first-half net profit today, saying it showed why Australia needed a tax on resource company profits.

"But what you will see in terms of the future of the resource industry is that it is very strong, that's why Australia does need a resource rent tax."

Got that? Australian business is showing strong profits, ergo we need higher taxes.

There's so much stupid packed into that sentence that I don't know where to begin.

For a start, we could note that this parasite views all corporate profits as potential revenue for the government, rather than realising that it's not his damn money. 

We could next move on to the assumption that the government should tax successful businesses until they're no longer showing successful profits, and whether this is likely to produce more successful businesses or fewer.

We could wonder about what incentives it will create for economic growth when companies that do well are hit with unpredictable taxes, and how business will respond in terms of investment and job creation.

We could take a detour via the observation that setting different tax rates for different industries based on which ones seem to be doing well is the favored policy of banana republics, corrupt autocracies, and communist kleptocrats.

But honestly, what's the point? It's just whistling into the wind with this moron in charge.

The Labor Party seems to have completely abandoned its reputation for solid economic stewardship so carefully (and deservedly) built up under the Hawke and Keating governments.

Predictable Preference Reversals in Snowboarding Holidays

So I spent the last few days going snowboarding. I had arranged to go for three days, and head back early afternoon on the 4th day, on a 2:30 flight.

Now, when I started on the first day, I immediately regretted not booking for a fourth day and flying home in the evening. I started to think about whether I could change my flight, or alternatively whether I could at least cram in a morning of snowboarding before my flight.

But I'd been through this game before, and the three day choice was a deliberate one. Because every snowboarding holiday goes exactly the same way. On the first day, you're so stoked to be there that you immediately regret not booking for longer. But as you get to day two and three, you're pretty exhausted. Due to poor technique and male pig-headedness in not getting enough lessons, my front knee began to feel like an arthritic cripple in the days before knee surgery, when tearing an ACL was described as you having a 'dicky knee' for the rest of your life.

And by the end of the third day, it was a positive relief to be going home the next day. The prospect of a fourth day seemed exhausting. I slept in instead, and cheerily got on the plane.

The point is that it's a mistake to think about the question ahead of time as 'Do I want 3 days of snowboarding, or 4?'. The answer to that question is 'Aw hell yeah, snowboarding is so cool, let's do it for as long as possible!'

The correct way of thinking about it is 'When I've already been snowboarding for 3 days, will I feel like I need a 4th day?'. And the answer to THAT question, at least for me, is 'Almost certainly not - in fact you'll barely be able to walk.'

Because if you don't think this way, you'll predictably reverse yourself - you'll book for 3 days because you think you didn't have enough time, pay lots of money to change your flight after the first day because snowboarding is teh awesome, and then get to the third day and wish you hadn't changed you flight after all.

But for some reason, people who are craving an experience find it hard to put themselves in the position of having already enjoyed a good chunk of that experience and deciding whether to have a little more. It's the same reason that when I'm hungry, I always think I'll need the large quiznos sub, and that the regular surely won't be enough. Of course by the time I've eaten a regular-sized portion of the large sub, I'm feeling mostly full. But this never seems to instinctively occur to me at the time of ordering.

In the spirit of overcoming bias, even though I always feel like I need a large, I restrict myself to the rule of 'no matter how hungry you feel right now, just order the regular anyway'. And it works. Same with the 3 day snowboarding holiday.

In case you're wondering what Shylock snowboarding looks like, it's a combination of this:

and this:

A good proxy for estimating the skiing ability of random people on a mountain

Count the number of colours they're wearing, excluding black and white. The higher the number (and the brighter the colour) the more skilled the person is.

Predictions

I'd call this result a win for my prediction, except that I walked my initial estimate back a bit.

Overall I'm going with a partial win.

Good luck Egypt, and Sandmonkey for President!

Hippy Parents Make Me Rage

What is it with lame new-age parents and their refusal to impose any discipline on their children?

I was stuck on a plane with this couple with three young kids. One of them, apparently 2 years old, kept unbuckling her seat belt as the plane was taking off. The dad, some shlumpy herb beta type, kept insisting that she be able to sit on his lap. The air hostess (who, truth be told, was a bit of a bitch) told him that she was over two, and had to be in her seat.

Takeoff went okay, but then we got a repeat performance at landing. This guy got in a big argument with the hostess demanding to speak to the captain (who, the hostess pointed out, was in the process of trying to land the plane). But amazingly, at no point did he actually make any serious attempt at trying to stop the bratty kid unbuckling her seat belt, other than cooing type stuff. Forget raising your voice, even just a simple stern talk discussing why it was really important that she stay seated would have been welcome. But no, no inconvenience was too small for his bundle of joy. It was just the responsibility of the airline to bend to his child's wishes.

The hostess pointed out that if she wasn't seated and buckled up, they wouldn't be able to land the plane. The guy turned around to the cabin, announcing that she was buckled in and looking for moral support.

The lady behind me (who had a child of about 10) said to him and the rest of the cabin, 'you don't want to know what I think.'

Amen, sister.

A Tale of Two Cities

It's an underappreciated fact in political discourse that often the most successful points are made by telling a lot of narrative, and adding relatively little in terms of explicit commentary. Find a story that tells the main point, and just recount the story. The point gets made, but you come across as more detached. Even better, don't even tell the story, just show pictures.

In the case of leftist viewpoints, the strongest arguments are those that merely portray suffering and appeal to human compassion.

A great example is this story. It tells the story of the squalor of Harlem in New York City in the 60s. There are a few appeals to explicitly leftist agitprop about 'The Man' and such. But the essay is far more successful when the author is just showing photos of horrible living conditions and describing the people there.

http://www.skyscrapercity.com/showthread.php?t=402544

It's a very powerful essay.

On the other hand, it's possible to do this equally successfully with right wing ideas too. Here, the ideal depiction is that of moral squalor - the debasement that occurs when men live for no ideal higher than themselves. The most eloquent of these is Theodore Dalrymple (read here for some great examples).

Another example was a description of Birmingham in Standpoint magazine. The author, an anonymous wife of a pastor, describes the hostility she received from the local, predominantly Muslim, population. It's far more compelling call for immigration restriction than just praising the good old days.

Both of these stories describe complete decay of the urban environment, but suggest very different ways of dealing with the problem.

Interestingly though, they both agree on the need for police to enforce the law. This is a point too often missed by the left - the main perpetrators of crime may be poor and black, but the main victims of crime are also poor and black, and there's a lot more of the latter group than the former. This of course isn't lost on someone who actually spends time around these areas, as Jakob Holdt did in the first essay.

How Long Stuff Takes To Produce

In 2010, a 15 storey hotel in China was erected in 6 days:

In 1775, over the 9 months from April to December, Wolfgang Amadeus Mozart wrote 5 exquisite violin concertos, each one increasing in musical sophistication.

In 1905, over the space of 12 months, Albert Einstein published 4 groundbreaking papers in physics, covering the photoelectric effect, brownian motion of small particles in liquid, special relativity and matter and energy equivalence.

In 1931, the Empire State Building was constructed in less than 14 months - 410 days, to be precise.

In 2011, after 18 months of waiting since my application, the Department of Motor Vehicles in the State of California finally saw fit to issue me with a driver's license.

Truly, we live in remarkable times.

Who wants to be average looking?

The answer is (or should be), "everyone".

Roissy links to the following amazing picture, which is a computer composite of the faces of women from a lot of different countries. By combining lots of traits, the average traits of each group are highlighted. The full size image is here. There's such a gold mine of stuff to talk about here, but I want to focus on one big point.

I title the picture below:

Attractiveness(Average) > Average (Attractiveness)

Let's face it, these are fairly attractive pictures. So what's the reason?

It's related to the analysts' consensus forecast problem. It's a well known fact that the average opinion of a bunch of analysts (sports commentators, stock analysts, forecasters of some sort) is generally better that the median person in terms of accuracy. In other words, take 10 guys who forecast football game outcomes. Each game, find the average prediction (i.e. if 7 guys favour the Fremantle Dockers to win, pick the Fremantle Dockers). An algorithm that always picks the same as the average of the 10 guys will do significantly better than the 5th guy, and many times will actually perform better than any of the 10 did individually.

The reason this holds is that each individual guy is subject to mistakes. But if those mistakes are independently distributed, then taking an average will cause the noise in the estimate to cancel out. Hence, better forecasting. The key point is that errors are symmetrically bad. Being too optimistic or too pessimistic will both screw up your forecasts. As long as the distribution of forecasts is centred on the true value, the average of all forecasts will tend to improve on the median forecast.

So how will this help in faces? The answer is that it improves any trait where deviations in either direction are bad. If your nose is too big or too small, both are bad. If your eyes are too close together or too far apart, both are bad. If your face is too wide or too thin, both are bad.

There's a certain number of traits that fall into this category, because as Roissy notes, facial beauty favours symmetry. Any individual will have variation in these traits. The average of all people is much more likely to have symmetry across all of these symmetrically distributed traits, and thus look better. The average person, on the other hand, might score well on face shape, but have a nose that's too big (or too small). Hence they lose points relative to the composite image.

The key to the improvement of the average is that being in the middle of that trait is the best point. This is strongest in facial traits, where slight asymmetries in things like the height of each eye can quickly reduce attractiveness.

So where won't this hold? In other words, in which traits will the average woman merely look average, rather than good?

The answer is cases where the middle of the distribution isn't the best place to be. In terms of typical male perceptions of looks,  for a start I'd say weight and breast size. Most men favour bigger breasts and skinnier women, and the number of women who are too skinny or too buxom is likely to be small. To a first approximation, more is always better.

In these cases, the attractiveness of the average will be similar to the average of the attractiveness, because the middle of the distribution is not the optimal point. On these traits, the average woman will only be average.

Moving past looks, the average woman would probably only score average ratings in terms of being funny, being athletic, being a good cook - pretty much anything where more is generally better, they'll only be average.

But she will have a prettier face than average. Because there, symmetry is king.

I don't know if there's an equivalent image for men, but I'd bet a large amount of money that the average man is attractive too. Even without looking at the picture, I'd trade in my own looks (which, let's face it, are considerable) for the average.

The other posts that I considered writing for this picture (and for which a sentence will have to suffice) is this: I wonder what the tendency of the average woman to be smiling says about the likely personalities of the people in those countries? My guess is it says a lot. Average hairstyle says something too (as do the wisps of other hairstyles - fringes, hair up, that kind of thing). Lithuanians love fringes - who knew?

Requiem

Mama Holmes was in town this weekend, and we went for a long drive to a state park. Upon returning, my windscreen was totally filthy, in the way that no amount of wipers and spray can seem to put any dent in. Hope springs eternal in the human breast, however, and for some reason, one always feels internally convinced by the logic, 'sure, it did nothing the last 5 times I used the spray and wipers button, but maybe it's loosened it up for try number 6!'. Which it never has, of course, and eventually it became obvious that I'd need to clean it properly at a petrol station.

I filled up the car and began wiping. I was re-doing one section and feeling mildly peeved at the whole thing, when I was struck by the thought that every mark on my windscreen was due to some insect that got splattered on my drive. It made me think of the scene like that from the start of one of the Men in Black movies - you're a bug, flying around, going from flower to flower, and then one day for no reason at all you get squashed by a car travelling at you at 60 miles an hour, resulting in instant death. The car bore you no malice, of course - the driver was travelling some place for some purpose that wouldn't mean anything to you, even if you were able to comprehend it.

And in the face of this catastrophe, in response to this microcosm of the tragedy of the universe that is writ large across all of our fates, what response does your senseless death engender in the mind of the driver? What is the reaction of this representative of the highest intellectually developed and most morally sympathetic species on the planet?

Irritation that he will have to spend an extra 5 seconds cleaning your remains from his car.

It is difficult to bear too much of the world. If one descends too far down the rabbit-hole of wondering about the negative effects of all one's actions, it becomes impossible to live a life. One would become a naked ascetic, eating only the fruit that fell from trees, and obsessed about whether he stepped on any ants today by accident.

And yet...

Suffice to say, I felt very ashamed.

It may be impossible to avoid harming other creatures, but at least compassion is the debt we can owe to the world.

I'm sorry I ran you over, little insects. May you have gone to a better place.

Egypt Right Now

One post from Egyptian blogger 'the Sandmonkey' is worth 6 hours of al jazeera, 2 days of the bbc, and a month of everything else.

http://www.hyscience.com/archives/2011/02/rantings_of_a_s.php

His account has been suspended, but you can follow him on twitter. Apparently he was ambushed and beaten by police, according to his twitter feed.

Conservatives seem ambivalent about supporting the overthrow of Mubarak, for the justifiable fear that having the Muslim Brotherhood running things would probably be worse. If you believe Sandmonkey's account, it's hard to see how Mubarak hasn't forfeited what little legitimacy he once had. If it were me, I'd roll the dice.

Or for conservatives, here's an apt comparison. If dictatorship wasn't viewed as a bargain deal with Saddam and the Taliban, why are you willing to support it here? What exactly about Egypt makes you think it's less suited to democracy than these places? Now, you may think that the democratic project in these places has been a failure (and you wouldn't be short of evidence to support that view). But either way, it's hard to know how you can support Mubarak without thinking that it's high time we got the hell out Iraq and Afghanistan. Egyptians really are turning out en masse to demand Mubarak's removal, which is more than we can say happened in either of the other two places. Personally, I think it's high time they got a chance to determine their own national future.

Sadly, I'm revising down my estimate of the chances of Mubarak leaving in the short term (pace The Greek, who argued that my earlier prediction was a gimme). The army may not be willing to shoot the protesters, but they seem plenty happy standing by as the police do that job for them. Which is a very different proposition from what seemed to be the case a few days ago.

Generating truly random numbers...

...is always harder than you think. Always, always, always.

This comes under the category of "lessons that people generally know are true, but frequently have to re-learn the hard way". Such is the case of Mohan Srivastava, who successfully created algorithms to predict winning scratch lottery tickets.

One thing that's not discussed in the article, however, is why this isn't necessarily arbitraged. Mr Srivastava says that he figured out that his hourly wage from doing it just wasn't high enough. This confirms in my mind that he's a smart dude.

But let me suggest another reason, which relates directly to mispricing in financial markets.

It would be damn hard to convince a newsagent to let you come in day after day, look at 50 lottery tickets, buy only 7 of them, cash in the winnings and repeat. In fact even just doing it once marks you as highly suspicious. They might let you do it one day, suspecting that you're just a kook. But once they see you doing it multiple times, they'll do one of a couple of obvious things:

1. Stop selling to you.
2. Wait until you pick the tickets you want, then keep those tickets for themselves.
3. Call the lottery ticket office and tell them that something funny is going on.

Why would they do this? It's obvious - if one customer is taking all the winning lottery tickets, then selling only losing tickets to the other customers will cause fewer of them to come back to your store, hence less cash for you.

It's not enough to find mispriced assets. You've got to find mispriced assets that people will keep selling to you at the wrong price, even as you increase your volume of purchases.

Lottery tickets, sadly, are not such a case.

The story also tells you something else very good about mispricing. Arbitrage is likely to exist in markets where there are the fewest people looking for arbitrage. That's Grossman and Stiglitz (1980). So what's an example of a market that's populated only by imbeciles?  The lottery! The classic stupidity tax.

As a result, if you're the smartest guy playing the lottery by an order of 2 standard deviations (as Mr Srivastava probably was), there might actually be mispricing. In S&P 500 stocks where you're trading against Goldman? Yeah, not so much.

OKCupid - Masters at Data, Dumbasses at PR

OKCupid is is a dating website that ran what seemed to me to be an awesome business model - free signups, and supporting itself with revenue from ads. If the central problem of relationship-finding is a lack of liquidity, they did a great job of solving it. Everyone wants to go to a dating website that everyone else is already at, but you don't necessarily know which site that will be. When it's free for everyone, the co-ordination problem doesn't exist, as you may as well sign up and see if it works because it doesn't cost you anything. But because everyone is willing to sign up, it actually does work. Presto!

Their blog gives some of the best data-driven analysis of relationship trends I've seen. These guys seem very solid in terms of working with data, and seemed like they'd really thought about how to run a dating site. In short, they got the Holmes seal of approval (although I haven't actually used their site).

On their blog, they also wrote this excellent post a while ago talking about why the revenue model of paid dating websites like eHarmony and Match.com is broken. Essentially they have big incentives to make lots of profiles of inactive members visible, as it increases revenue when new people signup. But this means that there's very little chance the person you're talking to will actually respond.

I'm convinced! I'm not paying a cent for dating websites!

One small problem though...they just got bought out by Match.com. Which makes their earlier post a trifle inconvenient. Uh-oh, spaghetti-os!

So, if you've got such a great data driven approach, surely you can write a new post explaining why the earlier reasoning no longer holds and you'll still run a great site, right?

Or you can just try and delete the post, thereby starting the Streisand effect where trying to hide the information actually makes it more visible. Case in point, the cached version of the post above is currently the top-rated item on Hacker News. Quick, our left foot is still attached at the ankle! Reload the shotgun and fire again!

They have done a great service though - they significantly increase my estimate of the probability that their earlier post is actually correct, and that they can't write a follow-up post to explain why their new site at Match.com will be awesome. It won't.

You know who benefits from this? The guys who originally solved the co-ordination problem in online dating sites, and still operate their dating section for free. Shylock says that when the problem is liquidity, you can't beat a price of free. The guys at OKCupid are smart enough to know that this still holds true, even though they've cashed in to Match.com. You should be smart enough to realise it too.

Conan on Egypt

Ah, Conan! Truly the funniest man on TV.

The best signal of cafe quality...

... is the quality of art drawn in the foam at the top of your cup.

This:



is made by a barista who takes pride in what they do.

In my experience, I've encountered plenty of false negatives from this signal (good coffee without foam art), but so far I've never had a false positive (every foam art coffee has been good).

End Game for Mubarak

Most people don't make public, verifiable predictions. Everyone is an expert after the fact, and they're sure that they knew what was happening, but try to pin them down at the time and they're much more hesitant. This is the appeal of prediction markets - if you ain't betting, then you don't actually believe what you're saying.

So here's my prediction -  this is the end game for Mubarak. The army has announced that they're not willing to shoot at protesters. My guess is that Mubarak will be out within the week, probably within the next 48 hours. My estimate of the probability for this is between 0.7 and 0.8. I'm well aware that this is perhaps already late in the game and the prediction might be at the point of not being controversial, but it's my two cents anyway.

Revolutions happen in very quick succession, and the break point is always the same. 100,000 people turn up on the streets. President gives order for army to shoot. At that point, the only question is how much internal discipline the president and top commanders loyal to him have over rank and file troops.

If the army actually shoots, game over for the protesters. See Rangoon, Tienanmen Square. Unarmed protesters against live ammunition is an outcome that's not seriously in doubt. And not many people keep turning up to the streets to get mowed down by machine gun fire - the instinct for self-preservation kicks in, and the dictator stays.

If the army doesn't shoot, that's the end for the president. See the Philippines, Czechoslovakia. When people begin to suspect that the president doesn't have the power to stop them, they start descending on wherever the president is, and he goes from radio announcements that all is well to hopping on a plane for Switzerland. .

We'll see soon how well calibrated I am.

Update: In case I end up being right and it looks like this was a gimme prediction, here's the top suggested Google news article for 'Mubarak', from 40 minutes ago:"Analyst: No Sign Egyptian President Mubarak Will Step Down". So just remember, this was posted when the outcome was at least partly unclear. If I'm wrong, well so much better for the 'Analyst'!

Self-recommending Wikipedia Entries

"List of Animals with fraudulent diplomas"

Pure gold. (via Hector Lopez, who is on fire at the moment).

Drink driving - as funny as being punched in the kidney while watching Schindler's List

Cultures change slowly, but they do change. Sometimes, even more rarely, they can be changed from without by a deliberate effort.

One of the cases where this was done was in Australia with drink driving. Traditionally, drink driving was viewed as a bit of a laugh - a kind of blokey, she'll-be-right type affair where everything was kind of funny as long as you made it home okay and avoided the cops.

Some time during the 80s, the government began a concerted public relations campaign to de-legitimise it, much the same way that smoking used to be publicly acceptable everywhere and is now largely shunned. They ran a series of quite effective campaigns with the slogan 'If you drink and drive, you're a bloody idiot'.

The idea was to get  people to view drink-driving for what it is - an activity where in expectation you kill people because you were too cheap to pay for a taxi.

Which, when viewed in that light, is disgusting. Saying that you didn't kill anyone this time is like saying that playing an involuntary game of Russian Roulette with the general public is okay, as long as it landed on an empty chamber. If you actually kill someone while drunk, I'd charge you with murder.If you crash or get picked up, I'd send you to the slammer for at least some period of time, no matter the excuse.

As you can perhaps tell from my attitude, the campaign was largely effective. Australians generally now view drink driving as not something funny, but something that makes you a scumbag.

Which is why it's strange to see people who clearly don't think that way. Witness Ludacris, in his song 'Move, Bitch':

"I'm D.U.I., hardly ever caught sober,

And you're about to get ran the fuck over."

Moron

It's bizarre to me that he would think this makes him sound cool. I take it as given that rappers are generally a) dumber than a bag of hammers, and b) about as good role models as the hobo down the street. But I do expect them to have a sense of what makes them sound cool - after all, appearing cool is their main job description. Since Ludacris is fairly successful, I'm guessing he probably has his finger on the pulse of public opinion better than me. If so, so much the worse for public opinion.

I guess you've got to do something to make yourself sound like a big man, especially when you're only 5'7". Little Man Syndrome claims another victim.

It Takes Brass Balls...

...to be a double agent working against the Nazis in WW2.

It takes even bigger brass balls to be so good at it that you manage to receive both the MBE and Iron Cross.

For services to the Allies in fighting the Nazi menace and generally having enormous cojones, Juan Pujol Garcia is posthumously awarded honorary membership of the Shylock  Holmes order of "Guys who kick ass".

The Good Old Days...

...of The Economist, circa 1843.

Check out how eloquent, well-reasoned and thoroughly grounded in economics their arguments against slavery-based sanctions are:

We firmly believe that free labour, properly exercised, is cheaper than slave labour; but there is no pretence to say that it is so at this moment in our West India colonies; and we undertake to show, in an early number, in connexion with this fact, that the existence of the high protecting duties on our West India produce has done more than anything else to endanger the whole experiment of emancipation.

But, moreover, our West India monopoly,—the existence of the high prohibitory differential duty on sugar, is the greatest, strongest, and least answerable argument at present used by slave-holding countries against emancipation. The following was put strongly to ourselves in Amsterdam a short time since by a large slave owner in Dutch Guiana:—"We should be glad," said he, "to follow your example, and emancipate our slaves, if it were possible; but as long as your differential duties on sugar are maintained, it will be impossible 

...

But now were it otherwise:—have the professors of these opinions ever considered the huge responsibility which they arrogate to themselves by such a course? Let these men remember that, by seeking to coerce the slave-labour producer in distant countries, they inflict a severe punishment on the millions of hard-working, ill-fedconsumers among their fellow countrymen; but they seem always to overlook the fact, that there is a consumer to consider as well as a producer;—and that this consumer is their own countryman, their own neighbour, whose condition it is their first duty to consult and watch;

Note the lack of polemical assertions on economic matters far in excess of what economics can actually establish ("Stock markets are clearly overvalued, and the Fed should raise interest rates to fix this").

Examine how they deal with the anti-slavery trade restrictionists who share many of their aims, but who they feel are incorrect on some matters:

We must, however, in doing so, make a great distinction between the two classes of persons who are now found to be joined in an alliance against this application of free-trade principles; two classes who have always hitherto been so much opposed to each other, that it would have been very difficult ten years since to have conceived any possible combinations of circumstances that could have brought them to act in concert: we mean the West India interest, who so violently opposed every step of amelioration to the slave from first to last; and that body of truly great philanthropists who have been unceasing in their efforts to abolish slavery wherever and in whatever form it was to be found. To the latter alone we shall address our remarks.

We trust we shall be among the last who will ever be found advocating the continuance of slavery, or opposing any legitimate means for its extinction; but we feel well assured that those who have adopted the opinion quoted above, have little considered either the consequences or the tendencies of the policy they support.

Compare this with the sneering condescension of the modern magazine towards the Tea Party, who ostensibly share many of the aims of the magazine:

For all the talk about practical electioneering, some of [the Tea Partiers] teetered on the edge of the extreme and wacky. Mr Tancredo denounced the “cult of multiculturalism” ... Andrew Breitbart, the founder of a news site (Breitbart.com), railed in a speech against the hostile “mainstream media” in hock to the far left. 

Wacky! Extreme!

I'd definitely renew my subscription to the 1843 Economist.

(via Marginal Revolution)

Make Child Tax Rebates A Percentage of Income

If I had to nominate a fact in the 'most significant under-appreciated truth about western society', it's that western societies have ridiculously low birth rates by historical standards. The United States has 2.06 births per woman, just under the replacement level of around 2.1. Australia is 1.78. Italy is 1.32. Singapore is 1.1. As Mark Steyn is fond of pointing out, this has huge potential consequences. Social security is a Ponzi scheme that only works in a growing population, so lots of workers support each retiree. It collapses when you have negative population growth, and more retirees than workers.As most first world countries are rapidly finding out. The alternative is mass immigration, which is fine as long as you're willing to accept that your country at the end will probably look completely different in many cultural respects from how it started. If you're a self-loathing leftist who sees nothing but greed and oppression in the history of the west, this is a big plus. If you have half a brain in your head, it is not.

Another alternative is of course eliminating these programs, which I'm fully on board with, but try selling that at the Iowa caucuses and watch the AARP vote stampede towards your opponents.

For one reason or another, most educated people in the first world don't seem to want to have kids. While kids may have negative externalities when you're on an airplane, they probably have positive externalities in the medium term.  Like any good economist, we want to incentivise people to have more kids.

And here's where current policy gets tricky. Most tax credits are in lump sum forms, or tax rebates that phase out at higher incomes. The US gives you $1000 per dependent child. The Australian government gives you a lump sum 'baby bonus' of $5294 when you have a kid.

Here's the problem. It costs hundreds of thousands of dollars to raise a kid. Who is actually going to be incentivised to have a kid by the promise of five grand?

Idiots, that's who. Absolute drooling innumerate fools who can't figure out that this is a drop in the ocean relative to what that kid will cost them. Morons with ridiculous hyperbolic discount rates fuelled by their crack habits. Or (more likely) people who will collect the 5 grand, but weren't planning on paying for the kid anyway as they live on welfare.

Now, you don't have to be a genetic determinist to have misgivings about this scheme. A relatively larger proportion of the children in the country will now be raised in households where the parents thought it was a great idea to have a kid for the princely amount of five grand. If you're confident in the ability of Australia's robust public schools to undo this kind of home-instilled stupidity, then it's no problem. I also suggest you probably haven't had much exposure to Australia's public schools.

If I were designing a scheme, I'd make each child generate a tax credit equal to $2000 or 2% of income, whichever is larger. That way the middle class and the rich are encouraged to have children as well. It also reflects the fact that like it or not, rich parents will spend more on their children, and so you have to offer more dollars to incentivise them to have kids. While 2% may sound like a lot, I doubt people are going to start popping out 10 kids to save 20% on taxes. Although if they did, those kids would be being brought in a family that really hated taxes, which is fine by me too. Either way, the future is looking brighter.

Your Daily Roundup of Schadenfreude

-Western Australian police invade clubhouse of outlaw motorcycle gang using a battering ram. Bikies cry like babies about 'police vandalism'. (via Hector Lopez)

-Staff of Lefty New York magazine 'Harpers' join a union, magazine owner suddenly discovers the virtues of non-unionised labour (via Ace of Spades)

-Rahm Emmanuel booted off ballot in election for Chicago Mayor, based on minor technicality that he doesn't actually live in Chicago as the law requires. Court reverses decision of earlier officials, including super transparent and disinterested "hearing officer, and the full Chicago Board of Elections". (Sadly, the Illinois Supreme Court has stayed the ruling until it considers it)

Every Insurance Company Can Save You Money!!!

One of the most entertainingly useless statistics touted by insurance companies is the average amount of money that customers save by switching to their insurance.

The first hint that this information is useless is that every company seems to mention it. Now, they can't all be cheaper than each other, although that's certainly the impression that their ads convey. At a minimum, you might think there's some subset of non-advertising ripoff insurance companies that have only imbeciles as customers, and this is the segment that's being targeted in the ads.

The reality is different. It's obvious that customers who actually switch will save money, because nobody switches to a more expensive insurance company. The relevant statistic is what percentage of customers who actually get a quote from you end up choosing your insurance. Because that tells you whether it's worth spending the 15 minutes typing in information to find out if they're actually cheaper. Of course, they never tell you that, because the number is probably pretty low, and probably wouldn't actually motivate you to visit their website. It's much better to just tout the savings of the guys who switch, hoping that idiots won't realise that it's useless information.

Let's assume that all insurance companies are ex-ante identical, and that when a customer comes along, they form an estimate of the premium required for that customer that is drawn from the customer-specific random variable.- e.g., each insurance company quote for you is drawn from N(Shylock mean, Shylock Std Dev). A customer gets their first quote, then goes to the second company for a second draw from the distribution and chooses company #2 if it's cheaper. They repeat this process until they estimate that the time cost of drawing another quote is greater than the expected saving.

Now, in this model all insurance companies are equally good ex-ante. But there'll be some people who start with company A (who had an abnormally high quote), then get a quote from company B that's lower and switch. This will be offset on average by the same number of people whose first quote from B was at the high end, then got a draw from company A that was cheap and switched.

In other words, every company will be able to tout the same pointless statistic about how much their customers saved, even though it actually tells you nothing about the average cost of insurance from that company.

All it tells you is that insurance companies have different premiums for the same customer. Which, when you think about it, is a very interesting point. But somehow I don't think they're spending all this advertising money to raise interesting questions about microeconomics.

Update: The Greek notifies me that he in fact was able to write down a proper version of the model sketched above which generate the conclusions listed. Now that's intuition you can take to the bank!

High Pitched Voices

One thing I find surprising in modern America is the number of women with high-pitched voices. Now, I may be imagining this, but to my ear the average Australian girl speaks at something closer to the alto rather than soprano end of the scale like here. And this puzzles me, because we're talking about white people drawn from fairly similar Anglo-Saxon and European genetic stock. I don't know of any obvious 'high-pitched' races, and certainly not any that also would explain (by virtue of differing demographic representations) the average female pitch in the two countries.

But here's where it gets weird - we normally think that voice pitch as something you're born with. It wouldn't be high on my list of culturally determined things, and certainly not something that people deliberately change.

And yet that's where the data seems to point me. Either consciously or unconsciously, some fraction of women are deliberately speaking in higher pitched voices. My guess is that part of the appeal is that of appearing more girly and youthful. Women's voices drop too when they hit puberty, but not by nearly as much as men. By talking like an 8 year old, it has the same appeal as getting a Brazilian wax, but visible for the whole world.

Consistent with this, having a high-pitched voice is a fairly strong negative signal on my 3 second judgments of personality. This is not because I find the actual voice intolerable, or because it's a massive moral or character failing. But it's about correlations - a desire to make yourself look artificially girly and innocent is likely to be correlated with you being superficial and annoying (conditional on the voice being artificially high, and not just naturally high).

The signals to respond to the most are those that are the most informative about personality in general, not necessarily those that are the biggest problems in themselves. Can you think of any other signal of superficiality that you can identify within half a second of a person talking, regardless of what they're talking about?

Me neither.

Thoughts while wandering around California yesterday

"Man, it smells like another person suffering from glaucoma! It's like there's a glaucoma epidemic going around or something".