OpenAI’s latest system, Sora, creates videos on the basis of text prompts. You write a prompt and Sora does the rest. The result: highly photorealistic videos and endearing animations. From mammoths sauntering through a snowy meadow to a market full of people in Lagos, Nigeria, in the year 2056. Nothing seems too wild for Sora’s imagination.

Sora is not yet available to the wider public, because OpenAI is first letting “red teamers” assess the product in regard to its potential for abuse. OpenAI also asked creative professionals for feedback on how Sora can benefit them, framing Sora as a tool that could enhance their output.

At the same time, however, technologies like Sora may render many creatives irrelevant. Just last month, famed director and producer Tyler Perry halted an $800 million studio expansion after seeing what Sora is capable of. It would eliminate the need for him to build sets or travel to film on-location. “I can sit in an office and do this with a computer, which is shocking to me,” he told The Hollywood Reporter.

It should come as no surprise that advances in AI have sparked debates about the future of work as well as concerns about misinformation, bias, and copyright infringements. Responding to the backlash, tech companies have (virtue-)signaled consideration for these issues by stressing the importance of Ethical AI, Responsible AI, or Trustworthy AI and developing guidelines for these noble goals. Indeed, OpenAI’s website includes the disclaimer that despite extensive testing, it cannot predict all possible benefits and harms of its technology, and that (ironically) it is critical to release (potentially harmful) AI into the real world to increase AI’s safety over time.

Policymakers are also actively discussing how to regulate AI in order to prevent or, at least, limit the aforementioned problems. In fact, earlier this month the EU passed the landmark AI Act, which includes bans and restrictions related to biometric identification systems, manipulative uses of AI, and artificial general intelligence (AGI, or AI with human-level or above intelligence and the ability to self-teach).

While the widespread attention for AI ethics is to be applauded, it is guiding attention away from a deeper issue. Focusing on ways to regulate and improve AI confirms a techno-deterministic narrative, one that does not question the overall desirability of technological advancements and assumes that AI and AGI are inevitable. However, we as a society must question whether generative AI systems like Sora really bring about progress and whether these systems should be welcomed in the first place.

OpenAI claims “to ensure AGI benefits all of humanity,” but in reality it threatens the very concept of humanity. We are beyond AI just beating us at chess and Go, optimizing pricing, and recognizing faces. Although these previous milestones were impressive, shocking, and maybe even demeaning to those with a strong sense of human superiority, that technology “merely” automated routine and rule-based tasks. Now, as our preemptive concerns about super intelligence have slowly dampened, AI has begun to master a realm many long thought was uniquely human—the realm of creativity and imagination.

Creativity and imagination are related but different phenomena. To create or be creative means to materialize a vision or idea into a painting, a movie, a song, and so on. To imagine goes a step further. Imagination requires the ability to develop that vision in the first place—to see what has not yet been manifested. Sora fills the gaps in our prompts. The model not only understands prompts (a point OpenAI repeatedly emphasizes on Sora’s site), but also imagines what the scenes described could and should look like. Yes, that imagination is based on training data, but so too is ours.

For humans, imagination takes time and space to develop. But in our capitalist society, which determines our value based on productivity and volume, artists are told that AI will be helpful to them because it will allow them to create more content so that they can work faster. These promises of productivity make it easier for people to overlook the fact that AGI-driven intelligent systems are on track to have capacities equal or better than humans’. AI’s imagination will likely become more grandiose in years to come.

We should not let the promise of productivity or narrow debates about AI’s ethical implications distract us from the bigger picture. Under the guise of improving humanity by increasing productivity, we risk releasing our ultimate replacement.