Trolling for Truth on Social Media – Scientific American

During the 1999 World Trade Organization (WTO) meeting in Seattle, tens of thousands of protesters took to the streets with banners and puppets to push back against economic globalization. They were met with a violent militarized suppression. At the same time, a small group of artist-activists called the Yes Men created a parody Web site pretending to be the WTO. Cloaked in its official logos and design, they made critical claims about the organization. This hoax was so successful it landed the Yes Men speaking engagements as the WTO at several conferences around the world. As the absurdity grew, viewers began to question what they saw—which was the point.

Realizing that they could pull off similar pranks using mimicry of official Web sites, the Yes Men made a career out of punching up, posing as the National Rifle Association, the New York Times and Shell, among many others. In an eerie foreshadowing of today’s disinformation campaigns, these activists poked fun at George W. Bush’s gaffes as a presidential candidate at GWBush.com.

Through spoofs, the Yes Men understood the power of the Internet as a new networked terrain where battles over truth could be fought. They played with the ambiguity of authenticity at a time when most Internet users were already skeptical of online content. The “culture jamming” tactic used by the Yes Men took its cue from Guy Debord and the so-called situationists of the 1960s, who advanced social and political critique. In the 1980s and 1990s culture jamming unified activists around a common cause and set of tactics, such as making minor edits of an advertisement to drastically change its meaning. This form of “artivism” was championed by Adbusters, a Canadian magazine that ran numerous anticorporate campaigns, most notably initiating the call to Occupy Wall Street in 2011. Simply modifying the slogan under a Nike swoosh to read “Just Buy It” was an effective way of reorienting consumers’ ideas about what it meant to wear corporate logos as fashion.

Crucially for activists such as the Yes Men, the big reveal was the raison d’être for the hoax. The cognitive dissonance experienced by the reader or viewer was a clever strategy that opened the way for critical thinking. Once they produced that chasm of the mind, the real work would begin: convincing new audiences that these corporations were the real -enemies of democracy and justice.

The Yes Men’s tactics were a kind of media manipulation. For them and other activists, the Internet provided a means of knowledge transmission and a way to counter the credulity of the mainstream press and hold corporations to account. But the ingenuity of using the Internet as a canvas for mischief and critique worked a little too well. Just a couple of decades later technology companies have created a media ecosystem that allows governments, political operatives, marketers and other interested parties to routinely expose Internet users to dangerous misinformation and dupe them into amplifying it. There is mounting evidence of foreign operatives, partisan pundits, white supremacists, violent misogynists, grifters and scammers using impersonation on social media as a way to make money, gain status and direct media attention. How did we end up in a situation where lies travel farther and faster than the truth?

The answer involves the promise of networked communication technology, the new data economy and the spiraling deluge of profitable strategic misinformation. We must admit that the truth is often emotionally boring and that the motivation to take action online through, say, sharing a video requires some combination of outrage, novelty and hope. But rather than proposing a set of solutions that involve tweaking social media systems, hunting down bots or insisting on verified identities, we can look to activism to understand how we got here—and how we could get out.

These days it is difficult to remember that there was a time when what happened online was not so world-shattering. Back in the mid-1990s heyday of America Online, Internet users protected anonymity via screen names and cryptic profiles littered with song lyrics. Most would never have thought it safe to type a credit-card number into a Web site or share personal identifying information. Because bandwidth was limited and you were yoked to the telephone wire, going online meant stopping everything “in real life” and forming bonds out of shared interests or a desire to play backgammon on Yahoo Games. The Internet was a place you could go to be yourself or someone else. John Perry Barlow and other early Internet pioneers cheered that it was the “home of the mind” where neither bodies nor laws much mattered.

Because news was still expensive to create and its distribution largely remained in the hands of media moguls, the Internet was not considered a place to seek authoritative information. In the mid-1990s news organizations were contemplating “going digital,” which mostly meant putting print articles online. There was no widespread fear that local news would disappear. If anything, it seemed like networked communications would produce the opposite outcome: any person with a connection could write about their community and interests and publish it for the world to see, anonymously or not.

I often joke that the Internet died the same day someone figured out how to get users to pay online for pizza delivery. As the Internet developed into a digital economy, verifiable identity was indispensable to the flows of commerce. Of course, the first widespread online commodity was pornography, which illuminates an important point: it is often not the whizbang of disruptive innovation that drives social change but the technological adaptation of the ordinary and mundane. As technology develops, so do humans; in adopting new technologies, people become part of a recursive circuit that changes themselves and the world around them.

Whereas the printing press was the platform that gave birth to a society of readers, the Internet fashioned everyone as a publisher. Early social-networking platforms such as LiveJournal, BlackPlanet, Friendster and MySpace were like self-service telephone books; they gave people the capacity to share stories and converse. Similarly, today’s biggest Silicon Valley tech companies began from modest intentions, a desire to connect people for specific reasons.

Facebook built its base by maintaining exclusivity. It was social networking but only for the elite colleges. Its earliest version included a misogynistic feature where users could compare and rate fellow women students’ attractiveness. YouTube began as an update on video dating, where the “you” in YouTube was an invitation for users to upload short videos talking about their perfect partner in the hopes of finding true love. Twitter was meant to function like group texts among co-workers but only seemed to find its purpose when the techno-elite of SXSW used it to enhance communication across an already technologically dense network. In that context, Twitter’s character limit was celebrated as virtuous microblogging, where small strings of text were favored over the long-winded diatribes of traditional blogs. Each of these tools has evolved not just technologically but also culturally, as society passed through a phase of excitement into one of disillusionment.

Since their inception, big questions loomed about how social media companies could become financially lucrative. The search for profit drove decisions about expanding the user base, remodeling advertising and converting users into market value. Mobile technology and broadband accelerated the capabilities of tech companies to expand their services in new areas, including data harvesting. Personal data were seen as an artifact of time spent on these services, and by simply interacting, online users sloughed off enough residual data to energize a digital economy ravenous for every click, like, share and mouse movement to be aggregated and monetized.

Social-networking sites transformed into social media, where the business model was no longer just to connect people to people and litter those pages with ads but also to connect people to “content”—information, pictures, videos, articles and entertainment. The result was a digital economy built on engagement, where content farms making “click-bait” became the watchword of the Internet economy.

But not just junk news sites make money. By creating a content-rich environment, tech companies turned advertisers into customers and users into cattle to be milked. Behavioral data could be repackaged for purposes from marketing to research to political campaigning. Profit-sharing models that made average users into content producers generated a so-called influencer culture, where entrepreneurial creators cultivated networks of followers and subscribers and then monetized them through donations, subscriptions or sponsored content. As personal data became a cash cow for social media companies, user experience could be tailored to prolong their time on sites.

The consequence, as we know well today, was the development of personalized information ecosystems. No longer did Internet users see the same information. Instead algorithmic echo chambers shaped individual news feeds and time lines to the extent that two people sitting side by side may receive very different recommendations based on their past behaviors online. Scams and grifts that would have been shut down if they were taking place on city streets, like selling counterfeit merchandise or running an illegal taxi company, flourished online.

Yet technology companies shield themselves from accountability by claiming to be a humble set of rails on which information is shuttled from one place to another. Largely because of an early ideological commitment that cyberspace was no place at all, tech companies leveraged a metaphorical mirage, where jurisdiction in cyberspace is murky and accountability is elusive. While scholars of gender, race and technology, such as Lisa Nakamura of the University of Michigan, Alice E. Marwick of the University of North Carolina at Chapel Hill and T. L. Taylor of the Massachusetts Institute of Technology, routinely wrote about the dangers of divesting the Internet of a material existence, politicians and regulatory bodies failed to treat the Internet as a place where real damage could occur.

Just prior to the rollout of broadband in the 1990s, a hyperlocal model of media justice took shape through activist use of the Internet. Jeffrey Juris, an anthropologist of networked social movements, ethnographically studied how the anticorporate globalization movement of the late 1990s and early 2000s used every technology at its disposal to organize large summits to protest meetings of the WTO and International Monetary Fund. As Juris has written, the Zapatista movement’s use of networked communication technology was the forerunner to large protest gatherings: insurgents used online networks to connect with other like-minded groups globally and to provide the international press with updates on the struggle for independence in Chiapas.

To plan the 1999 protest against the WTO in Seattle, activists relied on Web sites and e-mail lists to coordinate their tactics and to forge trust across borders. Juris wrote of this form of media activism in 2005 as he studied the development of a digital hub for citizen journalists calling itself. He wrote that “Indymedia has provided an online forum for posting audio, video, and text files, while activists have also created temporary media hubs to generate alternative information, experiment with new technologies, and exchange ideas and resources. Influenced by anarchism and peer-to-peer networking logics, anti–corporate globalization activists have not only incorporated digital technologies as concrete tools, they have also used them to express alternative political imaginaries based on an emerging network ideal.” This shared set of digital tools included Web site templates that could be quickly adapted and networked through a centralized repository. The rallying cry of Indymedia contributors became: “Don’t Hate the Media, Become the Media!”

It was this same techno-optimism that later led activists to adopt Facebook, Twitter and YouTube alongside e-mail lists, SMS text groups and livestreaming during the so-called Arab Spring, the Occupy Movement and the early iterations of Black Lives Matter. These networked social movements were multiplatform in several senses of the word: they existed on computational infrastructure that referred to itself as a tech platform where activists offered an alternative political platform geared toward social justice.

Because activists were using this infrastructure to create widespread change, technology companies envisioned a new purpose for their products. To capture this momentum, companies such as Facebook and Twitter began to rebrand their products as tools for free speech. In this new marketing scheme, social media companies were likened to the digital streets or public square, and their products were framed as synonymous with democracy itself. In truth, the slipperiness of the term “platform” permitted companies such as YouTube, Facebook and Twitter to sidestep regulation and public-interest obligations that are typically applied to broadcast media.

Then, in 2013, the Edward Snowden scandal revealed a deep paradox to the public: The same technology used by activists to foment social change was being used by governments to spy on their citizens and for corporations and political campaigns to carry out different kinds of experiments. (Shoshana Zuboff explored this theme in her 2019 treatise on surveillance capitalism.) Activists’ participation on tech platforms was largely about using any means necessary to achieve a more just society. As the platforms’ products changed, so, too, did their usefulness to other actors, such as police, news organizations, brands and politicians. By expanding their customer base to include all these types of professionals, tech companies diluted their reputation as a place for digital democracy and took on the sinister character of a panoptic media system—one bent on making a profit at the expense of users and anyone who threatened their growth.

Throughout the 2000s the Yes Men continued to pull pranks through their form of digital activism. They devised a political education program, where many folks contemplated the use of hoaxing as a mechanism for social protest. Shenanigans undoubtedly make a lasting and memorable impression, but hoaxes and impersonation can backfire by giving false hope. No one likes to feel manipulated or tricked, and the tactic received significant criticism from people who were truly victimized by corporations. In 2007, for example, the Yes Men impersonated Dow Chemical during an interview on the BBC where they took responsibility for the gas leak disaster in Bhopal, India, and promised $12 billion in reparations. This “news” was met with excitement that morphed into sadness and disappointment when victims found out that Dow actually did no such thing.

It was not only leftists creating convincing hoax sites in the 1990s. Jessie Daniels, a sociologist, has researched the myriad ways white supremacists have used “cloaked Web sites” to malign Martin Luther King, Jr., and other Black activists and groups, in ways similar to the tactics the FBI would use to plant stories about King. Networked factions of white supremacists are keenly aware that they cannot show up in their true form online. Whether they are remaining anonymous to avoid social stigma or evade hate crimes investigations, white supremacists continue to see the Web and social media as a political opportunity to convert new believers. As a result, they have innovated on strategies to hide their identities online to maximize reputational damage to their perceived opposition.

Now the field is open to any ideologically motivated group. Tactics include impersonating individual politicians, creating mass fake accounts, and coordinating the harassment of journalists and activists through the use of streaming platforms, chat rooms and message boards. Groups have also used automated posting to game algorithmic signals, as well as paid advertising tools to target vulnerable populations. Others have generated denigrating deep-fakes. They also have adopted techniques to influence trending algorithms, as well as to circumvent content moderation.

Many of these techniques, such as the use of bots, were pioneered by advertising agencies, which understood that data were money and that the creation of fake engagement data could produce real profit. Now the generation of fake accounts and manipulated engagement are the means by which hoaxes are carried out.

Unlike the artist-activists who used hoaxes to reveal deeper truths about capitalist exploitation, these imposters use cloaking and pseudoanonymity to attack journalists, politicians and average users. My Harvard University colleague Brian Friedberg and I have written about the impact of “pseudoanonymous influence operations,” wherein politically motivated actors impersonate marginalized, underrepresented and vulnerable groups to malign, disrupt or exaggerate their causes. Recently accounts run by white supremacists claiming to be antifascist activists were outed as impostors.

If and when operators of pseudoanonymous accounts are found out, there is no grand reveal of some larger social or political critique. Usually their goal is to trick journalists into smearing their opponent or to simply cause chaos. These disinformers quickly move on to the next potential media-manipulation campaign to advance their political agendas.

Eradicating these impostor tactics is possible, but it would require tech companies to admit that the design of their systems aids and abets media manipulators.

Indeed, we do not have an equal-opportunity media ecosystem. The anguish of seeing these tactics deployed time and time again to malign movements for justice illustrates that over the long term, they are effective only for those who want to advance short-term gains over long-term trust and safety. Many people believe they can spot false news and propaganda, but the reality is that it is much more difficult because the very design of social media and the incentives to plant misinformation are weighted in the favor of disinformers. In an environment where novel claims travel far and fast, the truth is at a serious disadvantage.

Because tech companies have been reticent to handle the information war playing out across their platforms, society at large pays the price. News organizations, as well as individual journalists, are investing huge amounts of resources to combat the problem. National security experts and academic research centers across the globe are creating content moderation software to monitor social media. Yet activists who have long endured damaging coverage by misinformed press are now called to defend their very existence from impostors—the disinformers who are deliberately stealing the moral authority and trust activists have built up over years of digital engagement.

For anyone who still cares deeply about the truth and people’s access to it, fighting back involves dispatching with the ideology that technological platforms are democracy in action. They have shifted from connecting people to people to connecting people to information, tilting power toward those groups that have the most resources. They are also fundamentally businesses that have scaled without a plan for mitigating the harmful effects they have on society.

Redesigning social media for timely, local, relevant and authoritative information requires a commitment to design justice, which sees technology not as a neutral tool but as a means for building the worlds we want. As communication scholar Sasha Costanza-Chock of M.I.T. has researched, the process of design must adhere to an ethic of “nothing about us without us.” For example, there would be no accountability on facial-recognition technologies without the activism of groups such as the Algorithmic Justice League, the research of AI Now, the political work of the A.C.L.U., and ordinary advocates showing their support online and off.

Activists are visionaries in the sense that they see materials not just as they are but for what they can become. In the early 2000s they transformed technology in new and exciting ways, but that era has passed. We can’t stay swept up in “techno-nostalgia” for what once was or could have been. If we are going to survive our ailing social media ecosystem, the truth needs advocates.