Ken Liu’s 2012 short story “The Perfect Match” is a disturbing premonition of a world where Facebook is everyone’s personal assistant. What struck me most forcefully, though, was that although a Turing-test-capable personal AI is a key character, and feeding the algorithm drives the plot, the story is told in entirely human terms. There’s no sense, let alone a depiction, of the super-human technology that I imagine is driving everything. (Spoilers.)
“The Perfect Match” was previously published in Lightspeed, and collected in The Paper Menagerie and Other Stories (2016); h/t Olivia Chadha for alerting me to Ken Liu's work. It’s set in a world where Centillion, a Facebook-Google mashup, makes AI personal assistants called Tilly that help people optimize every part of their lives, from morning wake-up tunes to romantic recommendations. The protagonist Sai comes to see Centillion as an evil, manipulative corporation and agrees to take it down. In the climax, Centillion’s founder and executive chairman says to Sai, “You thought Centillion was just an algorithm, a machine. But now you know that it’s built by people—people like me, people like you.”
I think it’s exactly the other way around. Our media show Big Tech in terms of the people that make it work or who fight against it (human interest FTW), obscuring or ignoring the needs and drives of the underlying tech-business systems.
The story, first published in 2012, is prophetic given that behavioral tracking, intention-manipulating algorithms and virtual assistants that are now pervasive. The outcome is dark. However, telling the story in terms of human agency – admittedly, probably the best way to tell a story humans will enjoy – left me unsatisfied. The insatiable appetite of algorithms for data and their symbiosis with advertisers’ interests shape the humans’ behavior as much as (and I think more than) the people in the company shape the machine.
By analogy, a queen bee certainly is pre-eminent in a hive but it’s the collective behavior of all the bees that lead to evolutionary success. See, for example, this marketing video about “swarm AI” that describes how a bee colony decides how to choose a new hive location (h/t Gabor Molnar).
Tilly, the pervasive personal assistant, is a rather thin character with a slightly wheedling tone – probably as both Centillion and the story’s author intended. I was hoping, though, that Ken Liu would help me understand what Centillion the organism, and/or the business ecosystem in which it’s embedded, wants.
Consider this passage from the story, when the protagonist Sai meets the CEO: “The man was barely in his forties and looked fit and efficient—kind of like how I picture a male version of Tilly, Sai thought.” The implication is that Centillion’s founder made the AI in his own image. That’s surely part of the truth, but I suspect the CEO probably also molded his own persona to fit Tilly’s over time; and Tilly evolved to be what Centillion’s customers were most likely to respond to.
It’s very unlikely that there would just be one Tilly in the world: would all Centillion’s customers really resonate with exactly the same character type? That’s just one aspect of the underlying premise that Centillion is a monopoly. It’s a common trope; in the movie Ron’s Gone Wrong, the corporate protagonist Bubble is a monopolistic Facebook-Google-Apple mashup (see Bubble Inc's Dream). I don’t know whether monopolies make storytelling simpler, and/or whether auteurs (like their politicians) see monopoly when they look at Big Tech. There’s certainly massive market power, and monopoly is a convenient simplification. However, just as there are many human characters in a plausible story, there are many interacting greater-than-human entities built on groups of people (see O-gregores: Perhaps the aliens are already here).
In an interview for Lightspeed magazine associated with the first publication of this story, Ken Liu says, “Ultimately, I am not afraid of machines and databases, but those who hold the keys to the databases.” He's evidently focused on people, not the machine systems; the danger lies with people, not systems.
I’m less sure. I’ve been influenced by Heidegger’s essay “The Question Concerning Technology” (pdf) and take from it that the essence of technology is that it turns people into a resource. Technology transforms people into things (just as it turns things into other things), and thus I think can be treated as having agency. Here are a few excerpts from Heidegger’s essay:
What is modern technology? It too is a revealing. Only when we allow our attention to rest on this fundamental characteristic does that which is new in modern technology show itself to us. . . .
What kind of unconcealment is it, then, that is peculiar to that which comes to stand forth through this setting-upon that challenges? Everywhere everything is ordered to stand by, to be immediately at hand, indeed to stand there just so that it may be on call for a further ordering. Whatever is ordered about in this way has its own standing. We call it the standing-reserve [Bestand]. [footnote 16, see below] . . .
Only to the extent that man for his part is already challenged to exploit the energies of nature can this ordering revealing happen. If man is challenged, ordered, to do this, then does not man himself belong even more originally than nature within the standing-reserve? . . .
As soon as what is unconcealed no longer concerns man even as object, but does so, rather, exclusively as standing-reserve, and man in the midst of objectlessness is nothing but the orderer of the standing-reserve, then he comes to the very brink of a precipitous fall; that is, he comes to the point where he himself will have to be taken as standing-reserve. . . .
[Footnote 16. Bestand ordinarily denotes a store or supply as “standing by.” It carries the connotation of the verb bestehen with its dual meaning of to last and to undergo. Heidegger uses the word to characterize the manner in which everything commanded into place and ordered according to the challenging demand ruling in modern technology presences as revealed. He wishes to stress here not the permanency, but the orderability and substitutability of objects. Bestand contrasts with Gegenstand (object; that which stands over against). Objects indeed lose their character as objects when they are caught up in the “standing-reserve.” Cf. Introduction, p. xxix.]
P.S. A centillion is a number equal to 1 followed by a very large number of zeros (303 in the US, 600 in the UK). It’s 3.03 (or six, in the UK) orders of magnitude larger than a googol.
No comments:
Post a Comment