Donald Trump boasted that his election reflected the “voice of the people,” but it was mostly the voice of fear. And while ingrained racism and misogyny drove much of Trump’s support, his popularity reflected a massive sense of loss: real economic loss, perceived cultural loss, and anticipatory loss for their children’s generation. Just how “real” this decline actually has been, however, depends on where you stand, and where you’re falling from.

A recent study of stratification and eroding quality of life across generations, between races, and between socioeconomic classes sheds light on how America’s so-called “middle class” perceives itself.

Social-inequality trends over the past half century indicate that class divisions are growing more rigid, most are getting worse off, and those at the bottom are falling further, faster by the day. It’s the momentum of change that is causing much of the pain and anxiety, as many self-identified “middle-class Americans” are realizing the truth only now: They were never as well-off as they thought they were.

“Overall, if you look back 30 years, most of the distribution [of wealth] is lower than where it was in the ’80s. So…the typical American family today has less wealth than the typical American family in the ’80s,” says University of Michigan sociologist Fabian Pfeffer, who co-published a new research collection on trends in inequality. And yet, Pfeffer observes, higher on the economic hierarchy, affluent households experienced “the mirror image,” accruing riches and power at others’ expense.

For households losing wealth, Pfeffer found that social insecurity hurts from many different angles: not just in the evaporation of housing and retirement wealth but also through declining health, diminished prospects for their kids, and ensuing despair and anger.

“One of the things wealth gives you is safety and security,” Pfeffer says, but when insecurity becomes chronic “as labor markets become more insecure…and as public safety nets become more porous…that role of private safety nets, such as in the form of wealth, may become even more pronounced.”

Take the case of a working-class, jobless white youth in a marginal postindustrial suburb. He hovers in the same social status as his blue-collar parents, but his life is markedly harder than theirs were. He is priced out of higher education in a community with few living-wage jobs and has virtually none of the health or retirement benefits his parents attained through their now-vanished industrial vocations.

But white anxiety about middle-class precarity is only part of the picture because the middle-class was always built on structural inequality and social exclusion. The anxiety Trump manipulated so deftly on the campaign trail expresses real agony that working people are feeling. Yet the people who aren’t represented in Trump’s support base are in many ways suffering the most from long-term economic polarization.

Compared to whites, the downward trajectory has been steeper in communities of color. A typical low-income black kid has even dimmer college prospects, having been deprived of early education and decent housing and health care from birth. She grows up with greater exposure to traumas like mass incarceration or foreclosure. Racial discrimination limits her career opportunities and she moves from teenage poverty into inescapable, lifelong debt after getting hit by predatory lenders. While her parents were also poor, they benefited from public welfare and education programs, and retired with modest savings instead of an underwater mortgage.

Towering high above both these young people is the affluent white youth who inherits her parents’ educational advantages, graduates debt-free, and has the social capital that accrues over her formative years spent in a privileged community network. Her segment of society is pulling away from both low-income black and white communities.

Pfeffer’s analysis shows that “the probability of becoming part of the wealthiest 20 percent of Americans is seven times greater if your parents were also in the top 20 percent [of income earners] instead of the bottom.” And social mobility could decline further in the coming years. This is in part perhaps why Trump was able to win votes with his vision of restoring “greatness,” but such a restoration is a return to a fundamentally unfair social structure built on a zero-sum economy that has been cratering on itself for decades.

Such political polarization can poison democracy. As Princeton economist Angus Deaton contended, since economic elites are inclined to push policies that impoverish public social infrastructure and distribute wealth upward on the income scale (and into their pockets), the political system becomes increasingly undemocratic—and plutocracy becomes even more entrenched.

The 50-year trajectory of class polarization isn’t likely to end anytime soon. A political solution probably isn’t forthcoming under Trump’s impending reign of chaos. But for the rising generation, there may be some promise in the debate around rebuilding “infrastructure.” Rather than roads and bridges, however, a more radical approach would develop our social infrastructure. An emphasis on social programs that would directly benefit people’s lives could be a redistributive, comprehensive corrective for the wealth gap, and would carry real populist appeal.

“The one public infrastructure that I see as the most in need of public investment is education,” Pfeffer says, “and that is also because the private investment [has grown] more unequal, partly thanks to this growth in the wealth gap.”

In coming years, there will be a chance for a grassroots movement to emerge and press for a stronger public-education system as an intervention against further decline in the next generation—it could stop kids today from falling through the gaps that ensnared their parents.

Trump found support among people who supposedly lacked formal education. Yet they keenly understood how it felt to be cheated by the system. The blame for middle-class decline, however, which should be aimed at the top, was tragically misdirected at others on the bottom, whom Trump and right-wing media inaccurately portrayed as having an unfair advantage. Overall, a huge chunk of the country is sliding down; some have just been slipping a little faster in recent years. The more crowded it gets on the bottom rung, the harder it is to see those climbing at the top and leaving the rest of us behind.