Friday’s jobs report seemed to grab headlines for one aspect in particular: the labor force participation rate, i.e., the number of people either working or looking for a job, fell to 63.8 percent, the lowest level since 1981. That means more and more people are dropping out—retiring, turning to something else like grad school or just giving up on the prospect of a job altogether. But there was a debate about how much of a bad sign this is. Is it because the recession has made people lose hope of finding gainful employment? Or is it just because baby boomers are hitting prime retirement age and moving to Miami?

It’s likely a combination of factors. But there seems to be a big difference in what’s driving men and women to leave the labor force.

What do the numbers look like for both genders? According to the Bureau of Labor Statistics’s Current Population Survey, men’s participation rate—the ratio of men working or looking for work versus those who have dropped out—has fallen 3.1 percentage points since the beginning of the recession, and women’s has fallen 1.8 points. The dip looks more troubling for men than for women. The last time women’s labor force participation rate was this low “was in June 1995,” Joan Entmacher of the National Women’s Law Center told me. But her colleague Katherine Gallagher Robbins noted that this year has been pretty steady for women’s rate, while men are starting to experience a real decline.

Yet interestingly, a recent paper from the Federal Reserve Bank of Kansas City (hat tip to Mike Konczal) finds that the forces behind those numbers are very different. For men, 60 percent of the drop from 2007 to 2011 has been due to a decline in “trend participation,” meaning things that were on course to happen whether we were in an economic crisis or not. That’s because the rate for men “has been falling steadily for 60 years,” in part due to things like increased access to Social Security benefits and an aging population that make retirement look like a pretty good option. In contrast, the paper “attributes essentially all of [women’s] decline to the cyclical downturn of the labor market”—in other words, the fact that we hit the Great Recession.

Why would a recession drive women out of the labor force so much more strongly than men? Because when the labor market looks shoddy, the Kansas City Fed paper says, “nonmarket work can become relatively more productive for many women.” That’s a fancy way of saying that domestic work—and very likely childcare in particular—becomes more valuable. “The difference between the benefits of working or not working may often be fairly small” for women, it says, while “the human capital of men is often more specialized toward market activities,” in other words, jobs outside the home.

When parents are trying to cut back on costs, it’s unsurprising that childcare could top the list. According to the National Association of Child Care Resource & Referral Agencies, the average annual cost of putting a 4-year-old in full-time care can be as much as $14,050 a year. No wonder that nearly 40 percent of parents worry that their income won’t be enough to cover it. And most parents evaluate that cost against a woman’s salary. Many mothers are deciding that it’s more cost-effective to stay home and focus on domestic work than to go out and try to get a job in a terrible economy.

This is even truer for low-income women. Stay-at-home mothers are more likely these days to be young Hispanic women with low levels of education who may be unable to get jobs that will pay enough to outweigh the cost of childcare. The economy has made that decision even harder in another way: state budgets that were all but decimated by the recession led thirty-seven states to pull back on childcare support.

Women who leave the labor force for care duties may be hard to categorize, however. “For women it’s a little tricky because those who are out for family-related reasons may not call themselves discouraged workers,” Entmacher said. While more men are counted in the category of workers who have given up entirely on looking for a job—a troubling group —women who can’t find jobs that work with their care-giving responsibilities may be just as discouraged but not counted as such.

On the brighter side, it’s also possible that women’s higher inclination to get a college degree is playing a role. As Catherine Rampell reported at the end of last year, the high number of young women dropping out may not be doing so indefinitely, but instead are leaving to get more education. While demographic trends may be leading older people to drop out of the labor force and into retirement, there are a lot of young people leaving as well. Evan Soltas calculates that of the millions of “missing” people who should be in the workforce if we hadn’t entered a recession, the young are seven times more over-represented. That could mean that rather than giving up altogether, young women are going back to school in the hopes of upgrading their prospects once the job market really rebounds.

I hope more women are dropping out for the latter reason instead of the former, because taking a break from the labor force to care for children can have a huge impact on women’s earning capacity. As a report from Rutgers notes, women who take maternity leave often “pay a penalty for leave-taking in wages and earnings long after [their] child’s birth,” a portion of which is likely due to salary increases that would have happened had they stayed in their jobs. Similar penalties will apply to women who are being squeezed out of the labor force and into the home by the recession.