LondonAnybody who says they saw this coming is lying. When the London bureau went out to cast its ballots at lunchtime yesterday, the sun was shining, the birds were singing, and 11 separate polls were confidently predicting a photo finish with neither Labour or the Conservatives winning enough seats to form a government on their own. And since the same polls also all agreed that the Scottish National Party, whose leader, Nicola Sturgeon, had already pledged to “lock out” the Tories from Westminster, were on course for a historic victory in Scotland, it seemed as if the only question was whether Ed Miliband would have the stomach to claim the prize whose legitimacy David Cameron and the right-wing press were already beginning to attack.

When the polls closed at 10 last night, and the results of the BBC’s exit poll showed not just a larger-than-expected Labour collapse in Scotland, but also a failure to gain any significant number of Tory seats in England or Wales, my first thought was “This must be wrong.” To steady my nerves, I went on Nate Silver’s web site to remind myself that the wizard of odds had predicted 278 seats for the Tories, 267 for Labour, and 53 for the SNP—easily enough, with a little help from the Greens and the Welsh nationalists to put together a left-wing majority. If the exit polls were right, even my gloomy Labour insider, who on Saturday told me he thought a minority Tory government a real possibility, was too optimistic.

Of course, the reality turned out to be much, much worse. When I went to bed last night it looked like the Tories and Lib Dems together might just have enough seats to govern. I woke up to a Tory majority, the certainty of five more years of cuts to social services, and the very real prospect—especially given the 4 million votes across the country for the UK Independence Party—that Britain will leave Europe. By lunchtime the former Nation intern Ed Miliband was also the former leader of the Labour Party, though he was beaten in resignation by former Nation intern Nick Clegg, who saw his party reduced to just eight seats in the new Parliament, where in any case the Tories now have no need of Lib-Dem support.

How did this happen? Though some pollsters have apparently already identified the “shy Tory” factor—personally I’d have thought “shame” entirely appropriate—the more searching debate ought to be about what happened to the Labour Party. Since the recrimination and finger-pointing has already begun, let me sketch out my own sense of what went wrong—and begin by saying that although Miliband was far from a natural campaigner (and the Tory press ruthlessly underlined every awkward gesture) Labour’s defeat did not stem from any of the candidate’s personal failings. Indeed in the past few weeks the conventional wisdom was that the Tories had fatally underestimated Miliband’s quirky charm.

There are, however, a few factors which, taken together, account for much of Labour’s difficulty, not just in this election but going forward:

Globalization has hollowed out the British working class. Thirty years after Margaret Thatcher broke Britain’s miners’ strike, the Labour Party still has not come to terms with the loss of the industrial unions who founded the party and remained its base. Tony Blair briefly offered a neoliberal vision of a consumer paradise, but the global financial crisis put an end to politics as an aspirational lifestyle choice, and Miliband—perhaps out of a desire to postpone the conflict that a real debate would require—never came up with a convincing alternative. The novelist James Meek offers much the most sensitive, and acute, diagnosis of this issue here.

Given the choice between austerity-lite and austerity, voters opted for the real thing. Among the many sickening sequelae of this latest defeat one of the most egregious has been the parade of freshly exhumed Blairites complaining that Labour was too slow to defend the party’s record in power, thus allowing the Tories and the right-wing press to embed firmly within the body politic the notion that it was Labour’s profligacy—money squandered on schools and hospitals—rather than the contagion from Wall Street, that caused Britain’s deficit to balloon. Like many half-truths, this is precisely half-true; under Miliband the party never effectively challenged the Tory narrative about the deficit. How could it, since it clearly accepted the Tory premise that, despite Britain retaining sovereignty over its currency, and even though Britain can currently borrow at historically low rates, the country was still somehow so much at risk of going the way of Greece or Ireland as to require a “triple lock” on spending. Every time Ed Miliband hummed a vaguely populist tune on the economy, Ed Balls, his shadow chancellor, would break into a rousing chorus of “No new spending.” (Watching Balls go down to defeat was one of the day’s few compensations.) The only politician to confront the austerity con head-on was Nicola Sturgeon—and look where that got her.

Scottish voters felt betrayed. For decades the Labour Party has treated Scotland like a plantation, where constituency parties of a few dozen members could be easily bullied—or bought—by London. And when the Scottish National Party began to challenge that docility, both in Westminster and in Edinburgh, Labour responded by making a Faustian bargain with the Tories to put down the cause of Scottish independence. In a referendum debate notable both for the unseasoned optimism of the “Yes” side and the naked appeals to fear by the “No” camp, Labour never made the positive case for a solidarity across nations, or for the sharing of risk and reward as part of a commitment to the common good. Instead, in a grim lockstep with the Tories—and the big Scottish banks—the party put its money on fear. That, it turned out, was the deal that broke Scottish Labour—and with it the party’s hopes of achieving a majority, perhaps in my lifetime. Labour may well come back into power in five years—but only as part of a coalition. Unless the Tories, who ruthlessly stoked up English resentment as a weapon against the prospect of a Labour-SNP coalition, drive Scotland out of the Union.

Mobilizing and organizing are not the same thing. In the anthology Blue Labour, the veteran American community organizer Arnie Graf writes that “building public relationships precedes power.” For a time last year it seemed as if Labour, which hired Graf as an organizer (without bothering to apply for a work visa) was going to really do the hard work of reinventing itself. But the key to Graf’s method is finding, identifying, and empowering new leaders—enough of a threat to the party leadership that someone leaked his visa status to the Sun. And until this morning, that leadership had no reason for regret, since there were plenty of bodies to be mobilized—as I saw myself. Mobilization is task-oriented: stuffing envelopes, canvassing voters, handing out literature. Organization involves listening, and responding, and being willing to change—a process anathema to Labour’s apparatchiks.

Fear is the right’s home-field advantage. Asked why, after five years of grinding austerity, real-wage stagnation, and an economy that was outperformed in Europe not just by Stakhanovite Germans but by the joie-de-vivring French, so many English voters lined up for more of the same, one politician quoted Hillaire Belloc:

And always keep ahold of nurse
For fear of finding something worse.

That was a strategy Labour never seemed able to counter, whether out of failure of political imagination or fear of appearing fiscally irresponsible. As in Scotland in September, Labour let the Tories define the contest as a battle between hope and fear. Fear won, both times.

In the next few days the media here will be filled with arguments between those who insist that Labour needs to lurch left and those who urge a swerve right. It will be ugly. And there will be blood. But this argument, which Ed Miliband worked so hard to avoid, needs to happen. Not because either side is correct—although my own sympathies are mostly with the left—but because some time in the Blair era Labour stopped standing for anything, and defeat on this catastrophic scale may just prompt a renewal from the ground up, rather than (as happened after 2010) from the top down. Nicola Sturgeon’s triumph in Scotland shows what can happen with idealism and energy and a genuine openness to change. But before Labour can again serve as a vehicle for change, the party itself will have to change, to decide not just where it stands but who it claims to speak for. And to do that, it will first have to do a lot of listening.