“Surveillance capitalism unilaterally claims human experience as free raw material for translation into behavioral data.”
― Shoshana Zuboff, The Age of Surveillance Capitalism“Throughout history, social control has been inseparable from the harvesting of personal information.”
—Josh Chin and Liza Lin, Surveillance State“If an individual is completely isolated, but most important, completely visible, says Foucault, then power functions automatically. “[T]he surveillance is permanent in its effects, even if it is discontinuous in its action; [ . . .] the perfection of power should tend to render its actual exercise unnecessary.”
― Kai Strittmatter, We Have Been Harmonized: Life in China’s Surveillance State
Been keeping up with China’s Xi and the CCPs use of technology and algorithmic governance for some time. A new book by Josh Chin and Liza Lin, Surveillance State came out today:
“Over the Communist Party’s seven decades in power, Xinjiang has been China’s most fractious region, riven by ethnic tensions between Uyghurs and Han Chinese migrants that have periodically exploded into deadly violence. Against the odds, the Party has brought the territory under total control using a combination of internment camps, brainwashing, and mass surveillance. The Communist Party’s offensive in Xinjiang ranks among the most unsettling political developments of the twenty-first century. Chinese leaders have revived totalitarian techniques of the past and blended them with futuristic technologies in an effort not to eradicate a religious minority but to reengineer it. The campaign is one part of a radical experiment to reinvent social control through technology that is forcing democracies around the world to confront the growing power of digital surveillance and to wrestle with new questions about the relationship between information, security, and individual liberty.”
Chin, Josh; Lin, Liza. Surveillance State (pp. 5-6). St. Martin’s Publishing Group. Kindle Edition.
It follows on my reading of The Perfect Police State: An Undercover Odyssey into China’s Terrifying Surveillance Dystopia of the Future by Geoffrey Cain, and We Have Been Harmonized: Life in China’s Surveillance State by Kai Strittmatter. Each covering different angles of surveillance and state control through AI, Police, and old and new Surveillance technologies.
Of course, we’re seeing aspects of it in the West as well, just not to the extreme extent that China has gone. I’ve read such works as The American Surveillance State: How the U.S. Spies on Dissent by David Hotchkiss Price, No Place to Hide: Edward Snowden, the NSA, and the U.S. Surveillance State by Glenn Greenwald, The Watchers: The Rise of America’s Surveillance State by Shane Harris, Democracy Betrayed: The Rise of the Surveillance Security State by William W. Keller, The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power by Shoshana Zuboff and others. It’s nothing new but the implications are there that with the newer technologies such systematic surveillance and social control are becoming easier to implement in even our own democracies. One imagines some new ‘state of emergency’ arising in which such technology will become not only prevalent but will become the bedrock of a new police state because of our fears and supposed needs for Security.
Kurt Vonnegut’s short story Harrison Bergeron was an early attack on surveillance state tyranny, a satire which takes place in a highly surveilled dystopian state where everyone is made physically and mentally “equal” by various laws that require anyone with any type of advantage to assume a handicap—if you are beautiful, for example, you must wear a mask; if you are intelligent, you must wear thought-distorting headphones. While some characters make an argument for the protections that these laws provide, the story shows a world devoid of any beauty, creativity, or love.
THE YEAR WAS 2081, and everybody was finally equal. They weren’t only equal before God and the law. They were equal every which way. Nobody was smarter than anybody else. Nobody was better looking than anybody else. Nobody was stronger or quicker than anybody else. All this equality was due to the 211th, 212th, and 213th Amendments to the Constitution, and to the unceasing vigilance of agents of the United States Handicapper General.
—Kurt Vonnegut. Harrison Bergeron
Other Dystopian tales on surveillance:
1984 by George Orwell (1949)
1984 reflects the author’s concerns about the dictatorships of his time, although it was also inspired by his activity at BBC radio during World War II, rewriting the news to make it conform to the propaganda needs of wartime. Orwell extrapolated the growing influence of electronic media—radio, movies, and TV—and their potential for misuse by power, from the broadcasting of propaganda rallies to televisions that can watch us back. As a classic awful warning tale, it established the parameters for surviving (or not, in this case) the surveillance state.
Shockwave Rider by John Brunner (1975)
Brunner anticipates cyberpunk in his portrayal of a character who can weave his way through an increasingly computerized society. Trained as a genius to serve the technocracy, the protagonist hides from, and indeed within, the system by periodically changing identities through his reprogramming of the database. Brunner mingles utopian possibilities with dystopian ones, showing how committed individuals can use the power of technology to thwart abuses of same.
Little Brother by Cory Doctorow (2008)
Little Brother is considered an adolescent novel, although it has been challenged as too mature and too anti-authority for young readers, especially by authority figures. A response to the contemporary War on Terror, it portrays a group of tech-savvy teens in a near future who get scooped up in the wake of a terrorist attack on San Francisco. They respond effectively with cyber-attacks on the Department of Homeland Security. As the title hints, the book offers an alternative to the pessimistic assumptions of Orwell’s classic.
The Circle by Dave Eggers (2013)
A polemical fable featuring one Mae Holland, a young woman who seems to land the perfect job at the high tech company The Circle. Its latest gadget is the SeeChange, a wearable camera that guarantees everyone perfect “transparency,” consistent with the company slogans: Secrets are Lies; Sharing is Caring; Privacy is Theft. Mae is very much with the program, to the point of betraying all other characters who express concern about the potentially dystopian consequences of this technology.
The Transparent Society by David Brin (1998)
The one non-fiction book on this list, The Transparent Society was written at the dawn of the internet era—before the proliferation of drones and camera phones—and is prescient is laying out the challenges for the twenty-first century. Brin counters the fears of the surveillance dystopia with advocacy of “sousveillance,” that is, turning the technology of transparency back on big institutions, private and public, as a guarantor of democratic civilization.
A Scanner Darkly by Philip K. Dick (1973)
The protagonist is Bob Arctor, member of a household of drug users, who is also living a double life as an undercover police agent assigned to spy on Arctor’s household. Arctor shields his identity from those in the drug subculture and from the police. (The requirement that narcotics agents remain anonymous, to avoid collusion and other forms of corruption, becomes a critical plot point late in the book.) While posing as a drug user, Arctor becomes addicted to “Substance D” (also referred to as “Slow Death”, “Death” or “D”), a powerful psychoactive drug. A conflict is Arctor’s love for Donna, a drug dealer, through whom he intends to identify high-level dealers of Substance D.
When performing his work as an undercover agent, Arctor goes by the name “Fred” and wears a “scramble suit” that conceals his identity from other officers. Then he is able to sit in a police facility and observe his housemates through “holo-scanners”, audio-visual surveillance devices that are placed throughout the house. Arctor’s use of the drug causes the two hemispheres of his brain to function independently or “compete”. When Arctor sees himself in the videos saved by the scanners, he does not realize that it is him. Through a series of drug and psychological tests, Arctor’s superiors at work discover that his addiction has made him incapable of performing his job as a narcotics agent. They do not know his identity because he wears the scramble suit, but when his police supervisor suggests to him that he might be Bob Arctor, he is confused and thinks it cannot be possible.
Dark Constellations by Pola Oloixarac (2019)
Argentinian Pola Oloixarac’s novel investigates humanity’s quest for knowledge and control, hurtling from the 19th century mania for scientific classification to present-day mass surveillance and the next steps in human evolution. Canary Islands, 1882: Caught in the 19th-century wave of scientific classification, explorer and plant biologist Niklas Bruunis researches Crissia pallida, a species alleged to have hallucinogenic qualities capable of eliminating the psychic limits between one human mind and another. Buenos Aires, 1983: Born to a white Argentinian anthropologist and a black Brazilian engineer, Cassio comes of age with the Internet, and demonstrates the skills and personality that will make him one of the first great Argentine hackers. The southern Argentinian techno-hub of Bariloche, 2024: Piera, on the same research group as Cassio, studies human DNA. When the Estromatoliton project comes to fruition, the Argentine government will be able to track every movement of its citizens without their knowledge or consent, using censors that identify DNA at a distance.
Decoded: A Novel by Mai Jia (2015)
Rong Jinzhen, an autistic math genius with a past shrouded in myth, is forced to abandon his academic pursuits when he is recruited into Unit 701: a top-secret Chinese intelligence agency whose sole purpose is counterespionage and code-breaking. As China’s greatest cryptographer, Rong discovers that the mastermind behind the maddeningly difficult Purple Code is his former teacher and best friend, who is now working for China’s enemy―but this is only the first of many betrayals.
As author Cory Doctorow puts it in a short essay on surveillance: “Some people think so. Today, there is a widespread belief that machine learning and commercial surveillance can turn even the most fumble-tongued conspiracy theorist into a Svengali who can warp your perceptions and win your belief by locating vulnerable people and then pitching them with A.I.-refined arguments that bypass their rational faculties and turn everyday people into flat Earthers, anti-vaxxers, or even Nazis. When the RAND Corporation blames Facebook for “radicalization” and when Facebook’s role in spreading coronavirus misinformation is blamed on its algorithm, the implicit message is that machine learning and surveillance are causing the changes in our consensus about what’s true.”1
As Shoshana Zuboff in The Age of Surveillance Capitalism tells us
Surveillance capitalism unilaterally claims human experience as free raw material for translation into behavioral data. Although some of these data are applied to product or service improvement, the rest are declared as a proprietary behavioral surplus, fed into advanced manufacturing processes known as “machine intelligence,” and fabricated into prediction products that anticipate what you will do now, soon, and later. Finally, these prediction products are traded in a new kind of marketplace for behavioral predictions that I call behavioral futures markets. Surveillance capitalists have grown immensely wealthy from these trading operations, for many companies are eager to lay bets on our future behavior.
As Jakob Howy in The Predictive Mind states: “the brain is essentially a hypothesis-testing mechanism, one that attempts to minimize the error of its predictions about the sensory input it receives from the world. It is an attractive theory because powerful theoretical arguments support it, and yet it is at heart stunningly simple. Jakob Hohwy explains and explores this theory from the perspective of cognitive science and philosophy. The key argument throughout The Predictive Mind is that the mechanism explains the rich, deep, and multifaceted character of our conscious perception. It also gives a unified account of how perception is sculpted by attention, and how it depends on action. The mind is revealed as having a fragile and indirect relation to the world. Though we are deeply in tune with the world we are also strangely distanced from it.”
Most of our AI Systems, our artificial intelligence machinic systems that are built of algorithms are being programmed to mimic human forms of intelligence so that such error prediction mechanisms are being built into these advanced intelligence gathering and collation systems. Built to study human behavior for purposes of social control and manipulation these systems may one day be turned back even on their makers as they gain that future singularity threshold of superiority over human thinking and thought. With such systems of algorithmic governance already at their disposal one can imagine the dark dystopian future that might entail.
One can imagine a totalitarian government that through extensive surveillance systems and detainment of knowledge successfully sustains its power and control over the public and private life of its citizens as it not only contributes to the internalization of surveillance which ultimately allows the government to transform its citizens into docile bodies, it also restricts and prohibits opposition to the state. The law enforcement further contributes to the crime it administers, by tolerating its existence among the smaller dealers and thereby utilizing the information it gains from them. Additionally, one can imagine advanced AGI utilizing various forms of perceptions of reality produced, recorded, and ultimately distorted through technological surveillance apparatuses. The technological advances the government applies in its extensive surveillance of its citizens is much in alignment with the theory of the Panopticon as a disciplinary concept that contributes to sustaining the constant fear and paranoia that most humans live in, because they may never know whether they are being watched or not.
- Doctorow, Cory. How to Destroy Surveillance Capitalism. Medium Editions (January 26, 2021)
- Shoshana Zuboff. The Age of Surveillance Capitalism. PublicAffairs; Reprint edition (March 3, 20
taken from here