Once again the rainbow folks are skirmishing again over that tedious question: Was I born this way, or did I choose to be gay? You know the party line: we homos were born this way and can’t change even if we wanted to, even if we tried. Our celebrity allies, from Lady Gaga to Macklemore, hew to this line, as do many gay advocates and activists. It’s often gay men who are more insistent on the innateness of sexuality, whereas many lesbian and bisexual women have pushed back at this argument, since we’ve often (not always) had different experiences with sexuality. But here’s a happy turn: the latest challenge to that orthodoxy comes from a gay man. At The New Republic, Brandon Ambrosino is thinking this orthodoxy through for himself, and concluding that “I can’t help myself” is an insulting narrative:

I see no reason to believe that the only sexualities worth protecting are the ones over which one has no control…. It’s like saying, “You can’t help it, sugar. You were born this way. Me? I was born with astigmatism and a wonky knee. We can’t change our limitations even if we wanted to.” (As if homosexuality was taken out of the DSM only to be written into the ADA.)

No wonder he’s getting corrected by the LGBT thought police. Gabriel Arana, a writer whom I respect enormously, takes issue with Ambrosino by saying, “the idea that being gay is a choice is precisely the grounds on which conservatives seek to deny gays and lesbians civil rights.” Sure, that’s true. But that’s no reason to agree to their terms. As Ambrosino notes, we’ve come a long way, baby, in a very few years—and it might be time to let our conversation get a little more sophisticated.

Ambrosino gets some things very wrong—he’s clueless on trans identities, for one thing, and has trouble distinguishing desire from identity. No one this side of the rainbow flag is arguing that people choose the direction of their romantic and sexual desires in the way that someone might, say, choose between different brands of toothpaste. Desire happens unbidden. Exactly what you’re supposed to do with a deviant flicker has varied over time and space: become a nun or priest; get slotted into a third gender between male and female, called “hijra” or “fa’afafine” or “two-spirit”; marry heterosexually and rear a family, while having “special friends” on the side. But if you get that flicker, you’re faced with a choice, whether or not you choose consciously. Do you suppress the flicker, or follow its lead? If you follow, do you do so openly or secretly?

According to Kinsey, up to 37 percent of men had some sexual experience with another man. “Lesbian until graduation” is by now a well-known phenomenon, and plenty of lesbians and gay men have tangled with those who come to visit for a few nights but head back to heterosexuality once vacation is over. And of course, many gays and lesbians who now identify as 100 percent homosexual have had opposite-sex relations when they were younger. Many adults, in other words, have a little internal leeway to choose which of their desires they will pursue.

Which means that in times and communities where the pressure against being gay is harsh—during the 1950s, say, or in certain religious communities—the few who “come out” and embrace being gay are often the ones with no leeway whatsoever, for whom denying or ignoring that overwhelming internal direction would be emotional suicide. That’s especially true for men. In our culture, masculinity is fiercely patrolled. Being a faggot, until now, has been a gender deviation that got bullied, beaten and badly humiliated. Many of the men fighting for LGBT rights spent years dealing with people who wanted to scour away the gay: dads who taunted them to “man up,” high school athletes who beat them up between classes, church group prayer circles that hounded them to be cleansed by “reparative therapy.” Those are the ones hating on Ambrosino’s argument—because they had no choice, except the choice of whether to kill themselves or come out.

But the others who sense that flicker and choose to walk away from it aren’t secretly gay. Gay isn’t the desire; it’s the social identity we layer on top of the desire—and it’s only yours if you claim it. Even men who have sex with men (MSMs, in the lingo) are not gay unless they say so. Midcentury evildoer Roy M. Cohn is one infamous example: he slept with men but insisted he was not a homosexual. Fine. He was not on my team—because he didn’t sign up.

But having teammates who refuse to wear the T-shirt is only half of the problem with wrapping an identity around desire. Here’s the other: desire is not a fixed or reliable compass. It varies, usually not in a willed way. I know many women who say that they were once so identified with one direction of desire that it never occurred to them it would ever vary—and yet fifteen or thirty years later, are stunned to find that their attractions take a U-turn. Some women who spend their younger adulthood identifying as lesbians shift in midlife and get involved with men. Some straight women have serious, meaningful relationships with men—and are stunned when they fall hard for a woman, not with a mild crush, but with full-on love and desire. These women often don’t call themselves “bisexual.” They say: at one point I was straight and then became gay (or vice versa). And that involves choice. Undoing a marriage or a household, leaving behind the way your social circle imagined you, rewriting your identity: all that takes effort and has social costs. Tackling that is a choice.

So let’s start talking about it that way. In much of the world—from Argentina to New Zealand—the rainbow faction is winning. And as the stigma fades—as we progressively win acceptance in law and in life—I betcha that more men will stop hiding the fact that their desires vary, just as women’s do. They’ll start saying that, yes, they made a choice to follow one sex or another, one person or another, because either way, there’s nothing wrong with it.

That’s the half of the “born this way” argument that often gets forgotten in these internal skirmishes. You can be born lots of ways that society demands you suppress. If someone could prove that being a child molester or serial rapist or homicidal sociopath were genetically predetermined, would we welcome those desires into our public square? Hardly. They fail the “I’m not hurting anyone” test. Which means the argument is really “I’m born that way and there’s nothing wrong with that.”

Because this one is the best argument we have: “Love makes a family.” That’s been our movement’s real contribution to the social discussion—that insistence that the building blocks of love needn’t be confined by sex or gender or reproduction, that how we care for each other is more important that who. And we’re winning on that one too, whether the social conservatives like it or not.

Our society protects chosen identities. One’s being a Seventh-Day Adventist, Sufi Muslim or Hasidic Jew may be strongly influenced by the culture one is born into—but it’s not genetic. People convert in and out, in a way that involves new conceptions of their core identity. In some parts of the world, and in large swaths of Western history, choosing the “wrong” religion can be a death-penalty offense. But in our era we protect your freedom of religion. It’s time to be neutral about orientation in just the same way, protecting personal freedom of choice. Because really, who cares?