Gareth L. Powell on AI in Fiction and Our Precarious Position With AI Development Today

Gareth L. Powell’s Descendant Machine is set about 200 years in the future, and yet the recent explosion in A.I. technology suggests Powell’s imagined future—in which the minds of humans and A.I.s are symbiotically enmeshed—is just around the corner.

The Bristol author’s new novel centers around a mysterious machine called the Grand Mechanism, an impenetrable black sphere, which, about two thousand years ago, replaced a star in a binary system. The system is home to a humanoid, multi-armed species known as the Jzat, who are divided among those who want to crack open the Grand Mechanism, believing it contains a wormhole to connect them with a more advanced Jzat civilization, and those who want to leave the mechanism alone, fearing it contains a black hole or other existential danger.

“I got a bit satirical with the way the faction is appealing to nationalism to get the power they need to open this thing by promising sunlit uplands and making Jzat great again,” Powell says. “It’s like any scientific experiment, any scientific knowledge that sentient beings see. It’s a process of just poking stuff to see what happens. Chimpanzees do it, and crows do it. You find something you don’t understand, you poke it and try and break it and see what it can do. And that’s how we learn. And that’s what’s basically happening on a massive scale in this story with this ancient machine that nobody knows what it does, but they want to poke it and see what happens.”

From the episode:

Rob Wolf: What do we know about the Grand Mechanism?

Gareth L. Powell: The Grand Mechanism is the mystery at the heart of the novel. It’s the locked room. If you’ve ever seen Ronin … it’s all about all these people trying to get hold of this briefcase and nobody actually knows what’s in it, but everybody wants it… The Grand Mechanism is the engine that drives the plot because there’s one faction that wants to open it to see what’s inside, and there’s the other faction who thinks that could be a really, really bad idea. For most of the book, we’re not sure which it is. It’s like a Schrodinger’s cat experiment of catastrophe.

RW: It reminded me a little bit of debates we have about technology. Everyone’s so nervous about A.I. right now. It’s happening so fast, and some people are saying “slow down, don’t do it,” and others are saying, “we have to go forward, this is good.”

GP: Yeah, it’s like any scientific discovery. From when we first learned how to harness fire, we could use it to cook or we could use it to set people we didn’t like on fire. It’s the beginning of 2001: A Space Odyssey, when the ape picks up that antelope thighbone and immediately starts whacking stuff with it. Anything can be used for good or bad.

RW: Descendant Machine is framed as an investigative report about what transpires with the Grand Mechanism. The author is the Frontier Chic, a sentient spaceship, who decides to write it as a work of fiction because humans find it easier to absorb fiction than a dry summation of events. Are you trying to tell us storytelling illuminates truth better, or just holds our attention better, but we can’t rely on it necessarily for truth?

GP: Some of that and some of the fact we are a narrative species. We pretty much assume language evolved when one Homo erectus wanted to tell the other one, “You should have seen the size of that tiger than nearly ate me.” Everything—our whole society, our history—is stories. Every tradition, every social change has got a story attached. We need a beginning, a middle and end, we need a why, we need a motivation in order to understand things. And if we just, as you say, get a dry recitation of facts, we seem to find it very hard to put that into the right kind of understanding. But if we get told a story, then we get it. We empathize with it, we understand it.

. Brenda Noiseux: I thought it was cheeky that it was a report written in novel format, and I thought, “Okay, this is fun. I can get behind this.” I’m curious though, why did you even feel to go that route? Because you are writing a novel. The novel itself, you know, stands on its own without that the introduction and ending that make it a report.

GP: I was messing around, and it struck me as it would be a fun thing to do. It’s been about 50/50 well received and not well received as a framing device by reviewers. But I just thought it would be quite fun to have the ship at the beginning going, “This is a report into why what we tried to do was a complete disaster.” And so the reader sets out thinking, “Oh, how is this going to be a disaster?” and it sets some expectations.

RW: So you’re a writer who reads reviews? Some writers don’t go there.

GP: There are writers who read reviews and then there are liars. I can’t believe there are writers who don’t read any reviews at all. I don’t go on Goodreads or anything because that place is just not designed for writers, really. It’s  more for readers, and they should have a space where they can sound off and talk about things.

But I do read the ones in online blogs or review sites, ones written by reviewers as opposed to ones written by readers, if you see what I mean, just to get an idea of how it’s being received, because those are the reviews that probably my editor, my publisher, other editors and publishers will see and take note of. So if there’s a review in a magazine, I’ll read that one, but I won’t go trawling through the comments on Amazon because, you know, that can be a bit like Forrest Gump‘s mum’s box of chocolates. You just never know what you’re going to get.

The ones that are written by critics and reputable review sites and stuff I’ll take a look at because it’s always good to get some feedback because otherwise I’m just writing in a vacuum and I don’t think that’s healthy because I’m trying to write stories that people enjoy and people want to read and want more of. So if I’m kind of disappearing up my own fundament, then I’m going to, at some point, lose the audience.

BN: I was curious if you were inspired at all by the conversations we’ve been having around artificial general intelligence. The A.I.s in your book, in a lot of ways, feel more like what we’re aiming towards—a human component working with an A.I. component, and both are benefiting each other because neither can excel or go to the next level without the other.

GP: Obviously this was written before all this kind of huge take off in A.I. talk hit the mainstream. We need to distinguish between things like ChatGPT and artificial intelligence, which are two completely different things. ChatGPT is just like the predictive text on your mobile phone, just a bit more complicated; it will scan through what it’s being fed off the Internet and predict sentences based on what you ask it to do. And I find that really annoying. I know other writers who have lost jobs already because copyright companies are thinking, “Oh, we can save a bundle on copywriting,” … which I think is ridiculous because these things do contain mistakes.… Eventually we’re just going to have these A.I. machines writing articles to get clicks from other machines who will write articles to get clicks from other machines, and it’ll just be this automated click thing and humans will be bored out of our tiny minds.

Actual thinking, conscious machine intelligence, I am completely torn on. On one hand, I’m really, really excited about the idea. On the other hand, I’m really, really frightened about the idea. It depends what we do with it. And the trouble with human beings is, as I said earlier, we’re curious and we’re a little bit impulsive. We tend to do stupid things when we invent something new. So are we going to hook it up to the nuclear launch codes to see what happens? …

There’s the perfectly rational argument that the A.I. might look around and see how chaotic we are and think, “I am stuck on a planet with chaotic, heavily-armed monkeys. The best thing I can do for my survival is wipe them all out.” Or it might be super-intelligent and realize that cooperation and stewardship and compassion are the way forward because it realizes that in the whole of eternity, life is incredibly rare and precious. So who knows? It could go any way.

__________________________________

Gareth L. Powell is known for using fast-paced, character-driven science fiction to explore big ideas and themes of identity, loss, and the human condition. He has twice won the coveted British Science Fiction Association Award for Best Novel and has become one of the most-shortlisted authors in the 50-year history of the award, as well as being a finalist for the Locus, British Fantasy, Seiun, and Canopus awards.

Brenda Noiseux and Rob Wolf are co-hosts of New Books in Science Fiction.