With tragic photos of bodies on beaches surfacing in the news, the refugee crisis seems to be concentrated at the borders of Europe and the United States. But the desperate exiles who have fled to the West actually represent a minority who have managed to make it across borders. Most of the world’s refugees are not even officially labeled as such by international standards.

From New Orleans to Bangladesh, millions of people have been forced out of their homes without crossing an international border. They’re labeled “internally displaced people,” and international humanitarian laws grant them virtually no protection, despite the fact that they often live under worse conditions than global refugees.

Globally and internally displaced populations overlap heavily, but the plight of “IDPs” is often overlooked because they are rarely counted. Fresh research by the International Displacement Monitoring Center (IDMC) attempts to track this massive, floating demographic by curating international and national data sets: More than 40 million people were internally displaced as of late 2016, with three-quarters of them having fled home in the past year. The chief causes were environmental disasters such as storms and floods, while conflicts and violence triggered about 7 million forced migrations.

Violence-driven internal migration is concentrated in war-torn countries like the Democratic Republic of the Congo, with about 920,000 displacements, followed by Afghanistan, with 652,000. But below the media radar, internal communal violence has displaced many communities—even in relatively “stable” countries like India and the Philippines, where conflict displaced more than 800,000 people combined. (Nonetheless, the overall rate of displacement is more intense when viewed in the context of DRC and Afghanistan’s relatively smaller populations.)

The emerging data on IDP flows expose how international humanitarian law offers only a partial solution to a global displacement crisis that transcends nation-state boundaries. Refugee law emerged as a political response to modern warfare, as a way to provide temporary relief to victims of turmoil and political repression. While international refugees are afforded certain legal protections (like the right to claim asylum in the court system of their host country, or to access humanitarian aid like emergency shelter and medical care), IDPs are generally subject to domestic laws, and often live under oppressive regimes with policies that forced them to migrate in the first place. A lack of relief at home is a key factor that drives international refugee migration, whether inside a given region or from Global South to North. Moreover, the IDMC notes, “the exact push and pull factors that explain how someone who is an IDP one day can become a refugee, an asylum seeker or an international migrant the next are still unclear.”

The crisis will deepen as climate-change destabilizes environments and livelihoods, as the IDMP acknowledges: “[Increased] frequency and intensity of extreme weather events and environmental degradation will increase displacement risk further.” Intersecting with spiking violence and entrenched inequality, environment-related social stressors and public-health crises will aggravate insecurity and resource competition. It could also breed even more conflict, as appears to be happening in the rapidly desertifying Middle East.

The link between violence and scarcity is illustrated today in the Horn of Africa, where intersecting stressors are spurring “recurring droughts, poor access to basic services and infrastructure, lack of livelihood options and ongoing conflict and insecurity,” leaving “highly vulnerable and exposed people with no other option but to move.” Famines in Somalia and territorial battles in rural Brazil show the impossibility of disentangling “man-made” and “natural” violence in regions that become simultaneously ecologically and socially uninhabitable.

The needs of specific subgroups of IDPs exacerbate the humanitarian challenge. Women may be especially disadvantaged in mass evictions, because of political disenfranchisement and discriminatory property laws. Indigenous and minority groups face heightened risk in land-rights clashes, and the poverty linked to their social marginalization leaves them more exposed to environmental disasters. In Colombia, a country plagued by climate change and civil war, Afro-Colombian and indigenous people make up three-quarters of victims of mass-displacement events, even though they constitute less than a fifth of the total population. As with environmental disaster, discrimination exacerbates the effects of displacement.

In reality, the chaotic refugee camps dotting the Mediterranean are the far edge of the refugee crisis. In many cases, the poorest and most traumatized cannot afford or survive the farthest-ranging smuggling routes, and the worst-off are forced to stay close to home. For example, the bulk of Syrian refugees now languish in camps just outside the border or in Turkey.

In extremely violent contexts, IDPs are forced to bounce across regions, surrounded by chaos. In Afghanistan, which shares porous (if hostile) borders with Iran and Pakistan, decades of global and civil war have meant that displacement is often permanent. Whether populations migrate to Europe or stay in the next province, much of the country has endured displacements across generations, constraining the government’s capacity to rebuild. A 2016 survey of Afghan refugees who entered through Greece revealed that one in four “were first or second generation refugees who had never lived in Afghanistan.”

Despite the fact that the internal-displacement crisis is even larger than the refugee crisis in numerical and humanitarian terms, the plight of IDPs remains shamefully neglected.

Yet every year this nation-sized diaspora grows, while borders harden and suffering deepens both within disaster-stricken societies and refugee-host countries. It is up to international institutions to prevent refugee crises by developing protections for the people adrift in the purgatory of humanitarian law. Whether inside or outside their home country’s borders, the displaced are a global responsibility.