Privacy Implications of Social Media Manipulation

The ethical debate about Facebook’s mood manipulation experiment has rightly focused on Facebook’s manipulation of what users saw, rather than the “pure privacy” issue of which information was collected and how it was used.

It’s tempting to conclude that because Facebook didn’t change their data collection procedures, the experiment couldn’t possibly have affected users’ privacy interests. But that reasoning is incorrect.

To simplify the discussion, let’s consider a hypothetical social network that I’ll call Wo. Wo lets people set up accounts and establish mutual friend relationships, as on Facebook. Rather than letting users post detailed status updates, Wo just lets a user set their status to either Smiley or Frowny. Users viewing the service then see photos of their friends, which are either smiling or frowning depending on the friend’s status. Wo keeps records of users’ status changes and when each user views their friends’ statuses.

Wo learns certain things about their users by collecting and analyzing these records. They can tell how often a user is Smiley, how long a user’s Smiley and Frowny states persist, and how a user’s status correlates with the status of each friend and with the statuses of their friends in aggregate.

What’s interesting is that if Wo manipulates what its users see, it can learn things about individual users that it couldn’t learn by observation alone.

Suppose Wo wants to study “emotional contagion” among its users. In particular, it wants to know whether seeing more Frowny faces makes a user more likely to set their own status to Frowny. Wo could measure whether a user’s status tends to be correlated with her friends’ statuses. But correlation is not causation—Alice and Bob might be Frowny at the same time because something bad happened to their mutual friend Charlie, or because they live in the same town and the weather there is bad.

If Wo really wants to know whether seeing Bob’s Frowny status tends to cause Alice to set her status to Frowny, the most effective method for Wo to learn this is a randomized trial, where they artificially manipulate what Alice sees. Some random fraction of the time, they show Alice a random status for Bob (rather than Bob’s actual status at the time), and they measure whether Alice is more Frowny when the false Bob-status is Frowny. (There are some methodological issues that Wo has to get right in such an experiment, but you get the idea.)

This kind of experiment allows Wo to learn something about Alice that they would not have been able to learn by observation alone. In this case, they learn how manipulable Alice’s emotional status is. The knowledge they gain is statistical in nature—they might have, say, 83% statistical confidence that Alice is more manipulable than the average person—but statistical knowledge is real knowledge.

A notable feature of this hypothetical experiment is that Alice probably couldn’t tell that it was going on. She would know that she was revealing her emotional state to Wo, but she wouldn’t know that Wo was learning how manipulable her emotions were.

The key point is that the privacy impact of an interaction like this depends not only on which types of information are gathered, but also on which prompts were given to the user and how those prompts were chosen. Experimenting on users affects their privacy.

Now: What does this hypothetical teach us about the privacy impact of Facebook’s experiment?

What it tells us is that Facebook did learn some non-zero amount of information about the manipulability of individual users’ emotions. Given the published results of the study, the information learned about individual users was probably very weak, in the statistical sense of being correlated with the truth but only very weakly correlated, for the vast majority of users or perhaps for all users.

To be clear, I am not concluding that Facebook necessarily learned much of anything about the manipulability of any particular user. Based on what we know I would bet against the experiment having revealed that kind of information about any individual. My point is simpler: experiments that manipulate user experience impact users’ privacy, and that privacy impact needs to be taken into account in evaluating the ethics of such experiments and in determining when users should be informed.