Facebook’s umwelt

Our CU Boulder tech & mythology group recently discussed Facebook’s umwelt, that is, the aspects of its environment that the company perceives. These are my post-meeting notes on the topic.

My thanks to all the participants: Cheryl De Ciantis, Paul Diduch, Jill Dupre, Mark Gross, Dale Hatfield, and Gabor Molnar. Their insights have shaped my understanding and this post (though the opinions and errors are all mine).

Umwelt

The term umwelt was coined by biologist Jakob von Uexküll and refers to what an organism perceives and what is meaningful to it (Von Uexküll pdf; Wikipedia). In an Edge.org piece, David Eagleman says that each organism presumably assumes its umwelt to be the entire objective reality ‘out there’. Pamela Paresky in Psychology Today calls it the slice of our ecosystem an organism can detect. 

Different organisms with different sensory apparatuses perceive the world very differently. For example, Von Uexküll’s 1934 monograph A Stroll through the Worlds of Animals and Men: A Picture Book of Invisible Worlds analyzes how a tick uses just three stimuli to find a mammal whose blood it can suck. As Clark (1996) paraphrases it: “Butyric acid, when detected, induces the tick to loose its hold on a branch and to fall on the animal. Tactile contact extinguishes the olfactory response and initiates a procedure of running about until heat is detected. Detection of heat initiates boring and burrowing.” [1] 

The umwelt is thus what an organism can perceive. It’s an interaction between the outside world, sensory organs, and brains. It doesn’t speak to subjective experience (if any), such as whether there’s anything “it is like” to be a tick. Humans have subjective experiences, though they might differ significantly such as in people with blindsight where there is visual perception but no subjective experience of it.

Cheryl De Ciantis used the term “sensorium” responding to a photo I showed of Phelim McDermott and Julian Crouch’s set design for Philip Glass’s Satyagraha. (The image represents the puzzle of social explanation: we need to account for the ordinary people, the inspiring leader, and looming behind him, greater-than-human things.) The large puppets led her to muse about how putting on masks affect one’s sensorium.

Von Uexküll’s umwelt sounds a lot like sensorium as defined in Wikipedia: “the apparatus of an organism's perception considered as a whole, the ‘seat of sensation’ where it experiences, perceives and interprets the environments within which it lives.” Wikipedia continues, “In medical, psychological, and physiological discourse it has come to refer to the total character of the unique and changing sensory environments perceived by individuals.” In a different sphere, the differing sensoria in cultures and individuals sounds like the different umwelten of different animals (Wikipedia).

Facebook

Our discussion of Facebook’s umwelt was a case study of a Big Tech, social media ogregore. Facebook (Meta) has 70,000-odd employees and 10 million customers (i.e., advertisers). (For company data, see the 2021 annual report) Even though Mark Zuckerberg has a controlling shareholding, he doesn’t know everything Facebook knows, and can't determine all its actions. Facebook runs about 100 million lines of code. It has 2 billion daily active users and 18 data centers worldwide. The data centers cover 40 million square feet, equivalent to 600+ football field-sized server farms. 

What does Facebook perceive?

When I consider Facebook’s umwelt, I’m thinking about the organization’s sensorium, not the perceptions (let alone subjective experiences) of its managers or its users. 

Facebook’s umwelt includes both raw and processed information in the same way that our visual system detects different elements in different brain areas, such as velocity tagging in the lateral geniculate nucleus, edge detection in V1, depth in V2, global motion in V3, simple shapes in V4, and complex shapes, objects, and faces in the inferior temporal gyrus. The point of the umwelt is 

Von Uexküll argues that that there isn’t an objective reality that our perceptions reveal. He claims that it is a “premature conclusion that objects by themselves are autonomous realities, having an existence of their own, independent of the subject.”[3] I would therefore not make fine distinctions between the low level data that Facebook collects and the inferences it draws from them.

Users and Advertisers

Facebook is a social media company. As such, it collects information about users and advertisers by observing (i.e., collecting data on) their behavior and their content. A 3-part project by the Share Lab research group about the “Facebook Algorithmic Factory” mapped and visualized the complex and obscure processes of its social network in four stages: data collection, storage, algorithmic processing, and targeting. (H/t Gabor Molnar for the reference. The work dates to 2016 but I assume the underlying architecture hasn’t changed much.) Share Lab identifies two main categories of collected information: (1) within the Facebook domain, the interactions, created or uploaded content, pages visited and other user activity on Facebook, as well as the information users provide about themselves; and (2) the digital traces harvested by Facebook from activity beyond their sites. This information (user activity, content, social connections) is stored and then processed to determine user interests and demographics in order to match them to ads. 

Facebook will have detailed information about advertisers, too, including what they’re advertising, where, and their willingness to pay for advertising slots. In addition to the engagement-related information on users and advertisers it will also collect aggregate financial information like revenue per user and advertiser, churn, and acquisition cost.

Other outside entities

Other entities in Facebook’s environment include:

• Social media competitors like TikTok, Snap, and YouTube, as well as other advertising channels, on which it collects user sentiment, and market share information.

• Hardware channels, notably Apple and Android, about which it can collect installed base and activity information.

• Adversaries like hackers, hostile governments, and criminal organizations.

• Governments (including legislators, regulators, and courts) for which it tracks proposed and in-effect legislation, rules, and litigation.

• Civil society and users as a group, on which it tracks sentiment and actions.

• Markets and investors, observing share price, trading volume, etc. 

• It can observe its microeconomic climate through cash flow, and the macroeconomic  climate not only through publicly available statistics like inflation and hiring but also via user sentiment and behavior that only it has access to. (For example, one can track inflation expectations in real time by analyzing tweets.)

Internal milieu

I consider employees to be both part of Facebook and part of its umwelt, in the same way that an organism tracks its internal milieu in order to maintain homeostasis. [2] Like any company, it tracks data about employee productivity, sentiment, salary & retention, recruitment, etc. 

Similarly, it observes the status of its operations, from lines of code, bug counts, number of servers, and server load to cybersecurity metrics like breach counts, Mean Time to Discover, and Mean Time to Resolve.

Offline world

I believe Facebook can observe offline activity. It gathers information about user location (Facebook documentation; NYTimes 2018). It can infer users’ household incomes and financial transactions. It is also aware of its impacts on mental health and society. According to the Facebook Files investigation by the Wall Street Journal, there were groups in Facebook researching such questions (“Facebook Knows Instagram Is Toxic”; “Is Facebook Bad for You?”]. Data was collected, distilled, and pushed up to executive levels. 

The company didn’t always act on the information, of course. On polarization, Zuckerberg and senior execs shelved the research (“Facebook Tried to Make Its Platform a Healthier Place”). However, action and perception (or subjective experience) aren’t the same. The fact that I don't act on something doesn't mean that I don’t perceive it. Conversely, in the case of blindsight, acting on something doesn’t mean the subject has a subjective experience of it.

Does Facebook have subjective experience?

The data that Facebook collects and processes are very different to those that we process in creating a human umwelt. From a human perspective, it’s exotic. However, Von Uexküll’s point was that one animal’s umwelt could be very different from another’s. We can still reason about another animal’s umwelt even though we may not know what its subjective experience might be. (An entity’s umwelt is not the same as its subjective experience, if any.)

However, we can, and often do, talk about the subjective experience of other animals even if it’s at second hand. One of the perennial topics in consciousness studies is whether, or to what extent, non-human animals have subjective experience. My impression is that a substantial number of scholars believe that they do.

It’s possible that Facebook as an entity has subjective experience. I don’t know how one would demonstrate that, and I’m not going to attempt a plausibility argument here. 

Speculating about what its subjective experience might be if it had one, I suspect Facebook might feel conflicted because it wants both its users and its advertisers to be happy. That's one of the reasons it got into trouble on teen body image, polarization and so on: content that that drove engagement also drove mental illness and aggression. 

It might also feel vulnerable. The C-suite is feeling under pressure (cf. Zuckerberg’s “Year of Efficiency” memo) and many employees must, too, given the past and promised layoffs. It’s been losing advertisers and users to TikTok, and advertising spend as a whole has been under pressure. That affects average revenue per user, cost per gross addition, and churn [cf. Tren Griffin’s Five Horsemen of the Business Apocalypse: ARPU, COGS, CPGA, churn, and WAAC].

What Facebook doesn’t perceive

In the same way that different animals have different umwelten, different organizations perceive different slices of the world. Even tech companies that are just as data intensive as Facebook will process different kinds of information. For example, Amazon tracks torrents of transactional and inventory data that Facebook doesn’t, and Google sees a different slice of user interests through search data than Facebook sees through posts and reactions. Moving to other industries, WalMart’s brick & mortar business has a different perspective on sales and inventory than Amazon does; Exxon tracks raw and processed material flows in great detail; and Southwest Airlines sees travel and equipment data Facebook doesn’t track. One could keep going indefinitely. It would be fascinating to do a comparative study that includes, say, the IRS and the Red Army. One can probably categorize organizations by similarities between their umwelten. [4] 

While Facebook does perceive activity in the offline world, primarily that of its users, it is not sensitive to many aspects of the world perceived by humans. Those include astronomical objects, the weather, and water- and airborne chemicals (aka tastes and smells). As I write that, however, I hesitate: the platform collects and processes user-generated content about all these things. Many human perceptions are technologically mediated—I can see galaxies only after they’ve been processed by telescopes and rendered on a page or screen. Are galaxies part of my umwelt? I don’t know what Herr Doktor von Uexküll would say…

Facebook doesn’t have a spatially centered umwelt the way animals do. Our perceptions come through our sensory organs on a body that’s located in quite a small volume (small compared to the volume through which that body can range). Facebook’s sensors and actuators are spatially diffuse; many, like social graph and financial metrics, are not really tied to spatiality at all. [5] It’s processing is also decentralized, for example with programming, legal, marketing and operational functions operating semi-independently. This reminds me a bit of octopuses; according to New Scientist, “Two-thirds of its neurons are located in its arms, and there is some evidence that each limb operates semi-autonomously.”

Notes

[1] Clark, A. (1996). Being There: Putting Brain, Body, and World Together Again. The MIT Press. https://doi.org/10.7551/mitpress/1552.001.0001

[2] In Feeling & Knowing: Making Minds Conscious (2021: 78), Antonio Damasio defines homeostasis as “the process of maintaining the physiological parameters of a living organism (for example, temperature, pH, nutrient levels, visceral operations) within the range most conducive to optimal function and survival.” For him, feelings are “the mental experiences that follow and accompany varied states of organism homeostasis.” The tracking of homeostasis is, for Damasio, closely tied to consciousness.

[3] Von Uexküll, J. (2001). An introduction to Umwelt. Semiotica, 134, 107–110 (pdf)

[4] An umwelt classification of organisms would cut across taxonomic categories as convergent evolution of very distinct animals leads to similar umwelten. I couldn’t find examples of animal species being categorized by their umwelt. However, one can categorize animal species by their behavior, which is influenced by their umwelt.

[5] If one ties consciousness to experience in space, as Jennifer Mather does, the decentered nature of Facebook’s umwelt argues against its consciousness. Mather is quoted in a 2011 New Scientist article about cephalopod intelligence as saying that “if you consider it to be a notion of self in space and an ability to make decisions based on information from previous experience and the current situation, cephalopods pass with flying colours.” Facebook would check the “make decisions” box, though.