As I write this sentence, Donald Trump has a 14.8 percent chance of becoming president. At least, such is the verdict of the “polls-only forecast” issued by Nate Silver and his team at FiveThirtyEight. Switching to the site’s “polls-plus forecast,” which accounts for factors like economic performance and electoral history, raises Trump’s odds to 18.3 percent. The New York Times offers a more reassuring estimate: According to its model, the Republican nominee has a 11 percent chance of moving into the White House next January.

These predictions would be more comforting if their creators hadn’t spent much of 2015 insisting that Trump would never make it this far. In September of last year, Silver estimated the likelihood of Trump’s winning the nomination at “about 5 percent.” That was up from the 2 percent chance Silver gave him a month earlier, when he outlined the “six stages of doom” awaiting the billionaire’s campaign. FiveThirtyEight was just as skeptical of Bernie Sanders’s candidacy, writing after his spring 2015 announcement that the Vermont socialist had “pretty much no shot of winning.”

The data connoisseurs at FiveThirty­Eight were far from the only figures who failed to see the support for Trump and Sanders coming, but the limits of their prognostications were particularly glaring because of their previous records. Silver had racked up an impressive set of accurate predictions since his 2008 debut as a political forecaster. He accurately called all 50 states in the 2012 presidential race, and while his performance was shakier in 2014, it was still imposing: He correctly projected 34 of 36 Senate elections and 31 of 36 gubernatorial races.

Silver’s predictions and the complex mathematical models that generate them are sterling examples of a new kind of political journalism and analysis that has flourished as newspapers across the country have shut down. Where the old school prizes shoe-leather reporting—interviews with voters, scoops from campaign staffers, leaked documents from party elites—this new generation looks for truth in numbers, supplementing qualitative musings with quantitative rigor. Drawing inspiration and data points from the more statistically inclined branches of the social sciences, these writers aim to dive beneath the froth of endless anecdotes to expose the underlying forces that drive our politics.

Even outlets that don’t share FiveThirty­Eight’s statistical ambitions have taken up the cause of using the social sciences to supply a deeper appraisal of current events than reporters hurrying to meet a deadline have typically attempted. Ezra Klein’s Vox is the standard-bearer for this brand of political analysis. A typical article in this style doesn’t have a mathematical model of its own, but it nonetheless reminds you of its social-science credentials, bristling with charts and links to peer-reviewed scholarship. Economists and statisticians make frequent appearances in the citations, but so do political scientists more concerned with state institutions and historical narratives than with regression analyses. Even the occasional humanist receives a nod.

The politics of this turn to social science is buried just beneath the surface. Most of the leading members of this new generation of policy analysts and data journalists came of political age under George W. Bush, when the White House turned an acquiescent media into its court stenographers in the run-up to the Iraq War. While Bush’s top aides dismissed “the reality-based community,” pioneers of this new form of political journalism took refuge in the techniques of social science that could make some claim on empirical reality. Knowledge wasn’t power in the Bush years, but it could expose the hubris of the powerful.

Barack Obama’s tenure in the White House—premised as it was on the hope of a postpartisan politics—offered the ideal conditions for this new interest in social-scientific knowledge. When Obama took office, he recruited his administration from the highest ranks of the technocracy, making Larry Summers, Timothy Geithner, Christina Romer, and Peter Orszag generals in his war on the Great Recession. Conservatives had Sean Hannity and conspiracy theories; liberals had Paul Krugman—and loads of graphs.

But Republicans had their own budding pop social scientists. Although a right-wing mediasphere dominated by Fox News and talk radio had little use for charticles, Obama’s presidency ushered in a boom for conservative think tanks. Despite their ideological commitments, researchers at these institutions had a lot more in common with their supposed opponents than they did with the fire-breathing birthers and Tea Partiers: They graduated from and taught at the same schools. They lived in the same neighborhoods and spoke in the same jargon. Whatever their political differences, they recognized a fellow wonk.

Statistical wizards, journalists brandishing white papers, and denizens of various social-science faculties were too busy looking at each other to see the populist tsunami rushing toward the Capitol. Scientific ambitions and empirical dexterity were supposed to give them a particular insight into how politics worked. Instead, these very qualities pre­vented this generation of policy wonks from being able to foresee the incoming electoral revolt.

Both Sanders and Trump, from the social-democratic left and the ethno-nationalist right, provided a voice for this insurgency against an expert class that seemed to know everything except how to improve the lives of Americans outside the Northeast’s Acela corridor. Trump’s supporters proved especially receptive to this message, rallying around a candidate who refuses to accept basic facts about his own biography, let alone the latest study to come out of the Brookings Institution. Experts dismissed Trump’s pledge to build a wall, but they had already constructed one of their own.

* * *

Yuval Levin has spent his career on the experts’ side of this wall. The type of conservative that a wonkish liberal could happily share a debate stage or an Uber with, Levin writes with the eminently reasonable tone of a David Brooks in training. He holds a doctorate from the University of Chicago’s Committee on Social Thought and has also done tours in the trenches of the American right, working for Newt Gingrich and George W. Bush. The leader of a cohort on the right whose members style themselves as “reformicons,” Levin has spent the Obama years urging his allies to ask more from their government than tax cuts and deregulation. He’s the founding editor of National Affairs, a quarterly journal that publishes articles with titles like “Resisting Abuses of Benefit-Cost Analysis,” and he does not tweet.

Levin prefers to express himself with books—four of them so far, though he is still shy of 40. The Fractured Republic, his latest, is a characteristic example of his approach. Even the most policy-oriented conservative thinkers often have a weakness for the metaphysical; Levin has a love for it. Eager to put his Chicago PhD to use, he is careful to mix a reference to St. Benedict of Nursia into his musings on teen pregnancy rates. But it is the spirit of Alexis de Tocqueville—­perhaps the greatest metaphysically inclined social thinker of his era—who presides over The Fractured Republic. Weaving together observations on the state of democracy with meditations on such hazily defined forces as “atomism” and “collectivism,” Levin argues that American politics has become the prisoner of baby-boomer nostalgia: It is stuck trying to re-create a supposed golden age in which the country was united by shared values, a rough degree of income equality, and respect for its major state institutions.

This golden age was more of a fable than Levin allows. But he rightly notes that its life span was short. It peaked just after World War II and began unraveling almost immediately. Though his diagnosis is conventional, the lesson he draws from it is more controversial. After a brief period of consensus, Levin argues, American society became splintered society, a fact that won’t change. The question, from Levin’s perspective, is how we come to terms with it.

* * *

The links between the abstractions that drive Levin’s analysis and the more concrete subjects that occupy his attention are murky at best. His explanation for the collapse of the midcentury order hinges on a relationship between “over-centralization” and “hyper-individualism.” Levin, like most conservative thinkers, believes the two feed off each other. Big government crowds out smaller institutions—Bible studies, PTAs, neighborhood watches, and the other standbys in this familiar catalog—leaving isolated citizens to stand against the state.

One baleful consequence of this dynamic of fragmentations created by over-centralization and hyper-individualism is mounting economic inequality: An atomized society cannot restrain demands from those at the top for higher executive pay and compensation, accelerating a growth in income inequality that globalization had already set in motion.

Here, the costs of Levin’s fondness for abstractions become evident. While a symbiotic union between centralization and individualism sounds like a plausible thesis, it does little to explain the world we live in. If a breakdown in social cohesion brought about by a powerful central government was the root cause of the 1 percent’s rise, then the gaps between rich and poor in the United States should be dwarfed by the vast chasms in Europe’s more centralized social democracies. But as any dutiful social scientist could have reminded him, the opposite is true.

This oversight reflects a larger weakness in Levin’s account. He is at ease with the minutiae of the legislative process and with metaphysical musings, but his grasp of political economy is more tentative. Perhaps this is because Levin’s chief concern is ultimately political, not economic. He worries that Americans, constrained by the government and separated from each other, have lost their capacity for self-government.

For a book so preoccupied with the dangers of nostalgia, The Fractured Republic puts a curious amount of emphasis on notions about a virtuous citizenry threatened by an overweening state that would be more at home in George Washington’s age than in our own. Levin’s solution to the problems created by America’s “fractured republic” are equally out of sync with his own time. His program—essentially, Compassionate Conservatism 2.0—dodges the question of why centralized programs like Social Security and Medicare remain overwhelmingly popular and does nothing to address some of the most serious issues facing us today. The index for The Fractured Republic contains multiple entries for “isolation,” but none for “climate change.”

Levin frequently portrays the Democratic Party and the doyens of centralized administration as divorced from the wider world. But The Fractured Republic suggests that if anyone is cut off from his fellow citizens, it is Levin. The candidates who leaned toward his positions in the Republican presidential primary—Jeb Bush, Marco Rubio, John Kasich—received a total of just under 8 million votes. Donald Trump had 14 million; Bernie Sanders, that tribune of a social-democratic tradition supposedly buried in the sands of time, won more than 13 million. Levin insists that Republicans and Democrats are united by their concerns about “the most vulnerable Americans.” Trump offers reason to think otherwise. His supporters have found a cure for the ailments of a fractured age not by resisting centralized political and economic power, but by forming an exclusionary idea of national community around their candidate.

* * *

Members of this growing wing of the GOP are everywhere. A video of a promotional event for The Fractured Republic“Beyond a politics of nostalgia: Yuval Levin on moving the country forward,” hosted by a prominent conservative think tank— currently has a little under 3,000 views on YouTube. The first comment is “Boring”; the second is “Trump 2016.”

Members of the liberal expert class are often more comfortable with position papers than ruminative treatises, but the political scientists Jacob S. Hacker and Paul Pierson manage to combine the ethic of the briefing book with the vigor of a manifesto in their new book, American Amnesia. They embrace the nostalgic baby-boomer politics that Levin bemoans, insisting that there truly was a golden age of American liberalism that should be remembered—and, with the right programs, could be revived.

With admirable boldness, they argue that a mixed economy—an economic system characterized by a strong state and robust markets—is not just a desirable policy regime but perhaps “the greatest invention in history.” The 20th century, they note, witnessed a rise in living standards that was arguably greater than all of the previous eras of human history combined, and much of the credit for that accomplishment belongs to the centralized governments that so trouble Levin. For him, the veneration of a useful liberal past has prevented a fractured republic from recognizing that it cannot return to the state-centered politics of the New Deal; for Hacker and Pierson, the glories of balanced government and shared growth make the distance that the United States has fallen since the mixed economy of the 1950s and ’60s all the more damning.

Employing a comparative perspective sorely lacking in Levin’s analysis, Hacker and Pierson demonstrate just how steep this fall has been for the United States. Once at the top of global rankings for personal well-being, Americans are now in the middle. Income, health, and education are the standard metrics, but America’s decline has also manifested itself in such incongruous areas as height. American men were once an average of three inches taller than their European counterparts, but today a typical Dutch man of six-foot-one will tower over his five-foot-nine American counterpart. From this perspective, the greatest toll of a politics that hasn’t kept up with changing times is not an increase in social anomie, as Levin would have it, but a generation of children who are sicker, poorer, and less educated than in other wealthy countries.

American Amnesia pins the blame for this deterioration on a retreat from the mixed economy led by two “Great Enablers”: radicalized Republicans and a misguided economic elite. The pair, Hacker and Pierson argue, worked in tandem to undermine the midcentury order, dragging policy rightward and giving rise to a “twenty-first-century feudalism” that converts clout on Capitol Hill into corporate profits.

To Hacker and Pierson, the irrationality of this new system is almost as deplorable as its consequences. They identify its characteristic defects not so much with policies that wield state power on behalf of special interests (although those are certainly a factor) but with the re­fusal to respond when private decisions exact costs on the public good.

Rescuing banks deemed “too big to fail” is justifiable during a financial crisis, but the implicit subsidy these institutions continue to receive from the government’s tacit backing of them after the crisis is over benefits the banks’ shareholders at the expense of the general welfare. This subtler mode of corruption is the preferred tactic of today’s robber barons, in Hacker and Pierson’s view, and is as common in health care, energy, and a host of other industries as it is on Wall Street.

In the language of economics, which today is also the language of policy-making, these are cases in which negative externalities are ignored and public goods neglected. In the more colloquial vocabulary of a book intended to reach a popular audience, “There is money on the table just waiting to be picked up.” However, change will not be easy, as Hacker and Pierson admit: “The problems are too complex, too interdependent, and too entrenched to make any single ‘solution’ plausible.” But step by step, incremental victory by incremental victory, Americans can reclaim their lost patrimony.

* * *

The ease with which Republican voters this year shrugged off the demands posed by this supposedly fearsome juggernaut of “Great Enablers” suggests that mobilizing a coalition in favor of the mixed economy might be easier than Hacker and Pierson recognize. But the returns from 2016’s primaries raise a deeper puzzle for their analysis: While public opinion hasn’t shifted dramatically rightward over the last generation, distrust in government has been rising. This is especially troubling for the authors, who reserve their greatest displays of enthusiasm for those experts in government who deploy the power of the state on behalf of the greater good.

“He didn’t believe in democracy; he believed simply in government,” said H.L. Mencken of Theodore Roosevelt, one of his era’s great prophets of the mixed economy. That charge has dogged aspiring technocrats ever since. Although American Amnesia emphasizes the important and destructive role that business leaders and conservative activists have played in undermining the mixed economy, corporate and political elites could not have succeeded without a complicit electorate that helped make their ambitions possible by electing these elites’ chosen representatives.

According to Hacker and Pierson, this is not because Americans are venal, but because they are ignorant: “Voters have a limited understanding of government performance, receive scant guidance from the media, and are encouraged by a barrage of negativity to assume the worst.”

At times, Hacker and Pierson sound like parents disappointed with a child who, after being given a priceless family heirloom, shrugs her shoulders and goes back to her smartphone. The egalitarian ambitions for the policies outlined in American Amnesia are undeniable. But in the authors’ view, these prescriptions are primarily the work of government, not democracy.

* * *

But what if, in our moment, democracy, as we understand it, isn’t worth defending? Hacker and Pierson’s fellow political scientists Christopher H. Achen and Larry M. Bartels take up this question in their new book, Democracy for Realists.

The pair take this question into consideration by setting their sights on what they call the “folk theory” of democracy: the idea that democracy is a system for translating the people’s will into government policy. They describe this belief as a 21st-century “divine right of kings” and consider it just as deluded as its medieval predecessor. For them, American democracy is not the embodiment of a popular will but the endless struggle between warring tribes motivated by cultural, political, and religious alle­giances, plus a hefty dose of self-interest. All politics, by this reasoning, is identity politics.

And that’s when voters know what they’re doing. Drawing on more than half a century of scholarship, Achen and Bartels conclude that “the political ‘belief systems’ of ordinary citizens are generally thin, disorganized, and ideologically incoherent.” These citizens routinely fail simple tests of political knowledge and base their votes on a sloppy mixture of group loyalties and shortsighted assessments of their own well-being—assuming they bother to vote at all, which in the United States, most of the time, they don’t.

That assessment provides a shaky foundation for would-be defenders of democracy. In Achen and Bartels’s telling, politicians win office by appealing to primal instincts within the electorate, while the work of governing is done by elites who know how to organize, build coalitions, and pressure the right legislators. “Policy-making,” they write, “is a job for specialists.” Voters don’t set the agenda; they merely help to elect those who do.

Lenin would have recognized this vision of a politics where power congeals at the top; but in our time, Achen and Bartels argue, the most skillful practitioners of this craft have come from the right. While the folk theory of democracy has encouraged liberals to assume that the popular will triumphs in the end, clear-eyed strategists in the conservative movement have recognized that ostensibly democratic institutions can be manipulated to serve their interests. Recalls, referendums, and other measures undertaken in the name of direct democracy chiefly benefit those who can afford to mount expensive campaigns promoting their pet causes—a group that, unsurprisingly, is dominated by the wealthy.

Meanwhile, right-wing activists have concentrated the rest of their attention on the top, lobbying policy-makers with enormous success. Grover Norquist, the Washington power broker who made signing a pledge to oppose tax hikes a virtual requirement for Republican politicians, is a representative example. He is also, from this perspective, the most effective Leninist in the history of American politics.

* * *

Where Hacker and Pierson place their faith in an expert class administering a mixed economy for the good of all, Achen and Bartels are readier, in Democracy for Realists, to depict politics as a battle for power. They also believe that to fix the imbalances of American democracy, liberals need to side with those who enter the conflict with fewer weapons. The powerful already have enough advantages; having a monopoly on the best tactics for effecting change should not be among them.

Yet a zeal to champion their “group theory” of democracy leads Achen and Bartels to push their arguments beyond what the historical record can sustain. Voters are not the perfectly rational individuals of academic mythology, but that doesn’t mean they—and by “they,” I mean we—are merely potential recruits for an unruly mob. Take, for example, what is arguably the central event of recent American political history: the once solidly Democratic South’s flight to the party of Lincoln. Challenging the interpretations that attribute this shift to a backlash against Democratic support for civil rights, Achen and Bartels argue that the GOP’s rise wasn’t a product of white racism, but of Republicans’ increased effectiveness at presenting themselves as spokesmen for white Southern identity.

The tangled relationship between racism and identity is only the most glaring difficulty raised by this account. While Achen and Bartels are right to note that the Democrats’ troubles in the South preceded Lyndon Johnson’s monumental legislative victories of the 1960s, that doesn’t eliminate the possibility that racism catalyzed the region’s embrace of the GOP. The iron link between the national Democratic Party and white supremacy had already been broken during the New Deal. A generation later, Barry Goldwater became the first Republican presidential candidate since Reconstruction to carry Alabama, Georgia, Louisiana, Mississippi, and South Carolina not because he embodied Southern identity, but because white Southerners knew he was their best chance to halt the civil-rights movement.

In their less doctrinaire moods, Achen and Bartels acknowledge the interplay between ideology and identity, but they too often succumb to misleading juxtapositions of the two that make it all but impossible to explain how a given set of policies becomes associated with a group, and how those associations can change over time. That problem confronted them again in May when, after the publication of their book, they turned their attention to the San­ders campaign. The Vermont socialist’s run became, in their telling, an ideologically incoherent protest against the Democratic establishment. Achen and Bartels correctly note that part of Sanders’s constituency was driven by opposition to Clinton and the elites who run (or so these voters believe) the Democratic Party. But these typically older white male voters had different motivations than those of the diverse coalition of millennials who gave Sanders landslide margins with the under-30 crowd, just as the affluent suburbanites who turned out for Clinton had a different ideological profile than the African Americans who also backed her by commanding margins.

Achen and Bartels are right to emphasize that majorities can be cobbled together for a diverse array of policies, but that conclusion should be liberating. The boundaries of our debates are wider than the enforcers of Washington’s conventional wisdom recognize. At its best, and its worst, the art of politics consists of channeling inchoate sentiments into platforms that give larger meaning to diffuse anxieties. Social identities deserve a crucial place in any explanation of this system, but so do the beliefs that are bound up with those identities. American democracy is shaped by many hands: activists searching for issues that can spark a movement; elected officials trying to manage conflicting imperatives from voters and donors; bureaucrats charged with making the government function; and, ultimately, the decisions of the electorate. To grasp this system, a realistic theory of democracy needs more than unwavering insistence on the primacy of the tribe.

* * *

Whether the way we imagine democracy requires the bracing corrective prescribed by Achen and Bartels is open for debate. Evidence is accumulating that support for democratic institutions has begun to fade across the world. As political scientists Yascha Mounk and Roberto Foa have shown, polls from an array of countries reveal mounting support for more authoritarian approaches to governing. In the United States, this trend is most pronounced among millennials, with only around 30 percent willing to describe living in a democracy as “essential.”

This disillusionment with democracy has been a long time coming. In 1840, five years after Tocqueville’s Democracy in America called for “a new science of politics” to understand the new world that democracy was creating, more than 80 percent of the country’s eligible voting-age population took part in that year’s presidential election. Only once during the remainder of the 19th century did the total dip below 70 percent. In no presidential election over the last 100 years has the turnout exceeded the lowest rate from this earlier period. The number of potential voters has expanded, and talk of politics saturates our culture, but judged by the simplest of standards—who casts a ballot—American democracy is deep in a century-long slump.

Tocqueville’s successors—policy and data journalists, Beltway wonks, and political scientists among them—can draw on more sophisticated tools for analyzing contemporary democracy than anything the French aristocrat could have imagined. But some truths can’t be found in a spreadsheet. “Effective democracy,” Achen and Bartels write, “requires an appropriate balance between popular preferences and elite expertise.” Fair enough—except that for decades this balance has tipped more and more in the elites’ favor and against popular control. With public power increasingly concentrated in agencies outside immediate democratic oversight, and with national governments professing themselves unable to defy global markets, the value of a vote has decreased. The technocratic bargain was attractive during an earlier era of peace and prosperity, but it has become more suspect after the experts have proved incapable of holding up their side of it.

What is most striking is how quickly this fatalism about democracy can give way to hope. Barack Obama’s 2008 campaign, following the despair that liberals felt in the Bush years, supplied a vivid illustration of this dynamic. At first, it appeared that Obama had touched off a passing democratic mania that would subside once the realities of governing set in. Then the same excitement surfaced again in 2011 with Occupy Wall Street, and again around Black Lives Matter, and then again with Bernie Sanders. For almost a decade, American politics has coursed with a new sense of possibility—and not just on the left. From Sarah Palin to the Tea Party to Donald Trump, Republicans have been singed by their own encounters with rekindled democratic spirits. Data are most useful when past behavior predicts future performance—­but when those relationships break down, as they have in the last year, so do the models. Call it democracy’s revenge.

The interweaving of these trends—diminished confidence in democracy today, coupled with rising optimism about what democracy might become—gives contemporary politics its bipolar quality. A rising generation of voters is both more skeptical of democracy than its forebears and more willing to support a democratic socialist for president. Activists often feel as if they’re flinging their bodies against locked iron doors, but there are times when it seems like these doors might swing open. The question now is what we’ll find on the other side.