BenBella Books

Excerpted with permission from T-Minus AI: Humanity’s Countdown to Artificial Intelligence and the New Pursuit of Global Power (BenBella Books, 2020)

---

With the advance of twenty-first-century technology, the watchful eye of the Communist Party’s authority has become even more penetrating. Digital methods of censorship, surveillance, and social control have become unavoidable, integral parts of Chinese society. Those methods provide the Communist Party, which essentially is the state, with powerful eyes, ears, and influence over most aspects of its citizens’ lives.

Again, and as stated, I am not criticizing the Chinese people themselves, nor suggesting that China is entirely alone in surveilling its population. The extent and unchecked degree to which China is doing so, however, is far beyond any Western notions of national security or local crime control rationales for doing so.

The AI-enabled surveillance state that is emerging in China results from the government’s embrace of AI at a speed, scope, and scale that is hard to imagine. Much of it is made possible by the sheer size and changing geographic characteristics of the Chinese population itself. As we discussed briefly in Chapter 8, China’s total populace of 1.4 billion is the largest in the world. More pertinent to the development and application of China’s evolving AI applications, however, is that tremendous concentrations of China’s population are being shepherded, for various reformation reasons, into massive, rapidly rising urban centers and cities. As of 2019, China has more than 65 cities populated by over a million residents each, and the number of such cities surpasses 100 when metropolitan areas are included. By contrast, the US has only 11 cities with populations that exceed a million. The largest US city, New York, had a 2019 population of 8.6 million. By comparison, more than 26 million people live in Shanghai’s overall metropolitan area — China’s largest city. All told, China has seven cities significantly larger than New York and 22 cities larger than Los Angeles, America’s second most populated metropolis.

The consolidation of Chinese citizens into enormous metropolitan areas is growing at an astronomical rate. Hundreds of factories and technology centers are now being built and relocated in concentrated city areas that, together, provide unparalleled numbers of new jobs and up-skill opportunities. Beyond that, the government is rapidly constructing housing complexes and offering a host of economic and lifestyle upgrades and benefits that encourage and motivate mass relocation. As a result of remarkably well-coordinated government funding, efficiently strategized engineering plans, and lightning-fast construction times, it is projected that by 2025, China will have more than 220 cities with populations of a million or more people.  Overall, the aim is that one billion people, a full 70 percent of China’s total population, will live in more than 400 enormous cities by 2030.

And it’s not just the number of Chinese megacities that merits comparative awe; it’s the advanced state of technology built into them and the purposes to which those technologies are being put that warrant the world’s attention. By 2018, more than 200 million government- monitored, closed-circuit cameras had been installed at intersections, street corners, pedestrian crosswalks, parks, recreation areas, commercial markets, shopping malls, office building entrances, museums, tourist attractions, entertainment venues, sports stadiums, banks, bicycle stands, bus terminals, railway stations, shipping docks, and airports. By 2021, the total number of surveillance cameras is projected to grow beyond 400 million — almost one for every three Chinese citizens. Fueled by machine learning facial-recognition programs, the cameras are linked directly to local governments, law enforcement, and other agencies, giving authorities the ability to electronically identify citizens, track and monitor them, and compile activity profiles on targeted individuals and common citizens alike. Reasonable policing and crime- prevention purposes might arguably justify, to some, the scope of such surveillance. But the Chinese government’s use of its vast camera system goes far beyond — at least from Western perspectives — any arguable legitimacy. While crime control, tickets, and arrests follow from what the cameras show, so too does a broad scope of government-imposed social control, social shaming, and citizen tracking. Huge digital billboards constructed next to pedestrian crosswalks, for instance, display the photos and names of individuals who jaywalk, get ticketed, or have outstanding parking fines. As a population that prides itself on reputation, the government’s unabashed strategy of socially stigmatizing its people is powerfully effective. And while many Chinese citizens undoubtably consider these aspects of the surveillance system an inevitable, or perhaps even positive, exchange for the benefits technology otherwise provides them, they ultimately have no say in the matter anyway—politically or otherwise.

Tracking physical activities through cameras, however, is only the beginning. China’s influence and control also invasively extend to people’s use of the internet and to their personal digital devices. China’s internet and digital market is controlled primarily by three corporate technology giants — Baidu, Alibaba, and Tencent (collectively referred to as “BAT”). Individually, they’re roughly equivalent to Google, Amazon, and Facebook. Baidu is a Chinese company with a significant global presence and reach. Based in Beijing, it specializes in internet services and AI, and it provides the second largest search engine in the world, only behind Google — which has been blocked in China since 2010. Alibaba is an enormous Chinese conglomerate based in Hangzhou that specializes in e-commerce, internet services, and technology.  And Tencent, discussed earlier in Chapter 7, is a Chinese giant based in Shenzhen that specializes in internet entertainment, social messaging services, gaming, and AI.

As of 2019, Tencent, Alibaba, and Baidu ranked as the third, fifth, and eighth largest internet companies, respectively, in the world. Combined, their power and range are colossal — particularly with respect to AI. It is currently estimated that more than half of all Chinese companies that are in any way involved in AI research, development, or manufacturing have ownership or funding ties that relate directly back to one of the three.

In China, a true separation between the public and private sectors doesn’t exist, at least not to any degree approaching the Western concept of separation between government and commercial enterprise. Regardless of the formal structure of their ownership, Chinese companies are subject to a mandated and direct influence from the Communist Party. 

Its largest enterprises, including the large tech giants Baidu, Alibaba, and Tencent, are required to have Communist Party committees within their organizations. This gives both the party and central government the ability to obtain and influence all strategies, research, intellectual property, and data the companies generate. Also, pursuant to China’s 2014 Counter-Espionage Law and its 2017 National Intelligence Law, all companies are formally required to “support, assist, and cooperate” with the state’s intelligence network, effectively making them unable to protect any data and information that the government demands. We’ll return later in this chapter to the implications this has on Chinese technology used outside of its borders.

Westerners often mistakenly assume that the content they can access on the internet is essentially the same as what’s available to residents of other countries. But that’s entirely untrue, and China’s control of its internet is one of the most glaring examples. Often referred to as the Great Firewall of China, the government not only blocks websites that don’t conform to its allowed content and messaging, but also systematically monitors, and even manages, individuals’ use of the internet in general. Many Chinese circumvent the government’s censorship and accessibility restrictions by logging onto the open internet through secure VPNs (virtual private networks) that connect them to computers outside of mainland China. VPNs are becoming increasingly difficult to obtain, however, and Chinese citizens risk significant government reprisal if caught using one.

Beyond censoring and monitoring the internet, China also surveils its masses by collecting data from their personal devices—most notably their mobile devices and the apps they rely upon to conduct their daily affairs. Since 2015, China has been developing a “social credit system” powered by AI that is expected to be a unified, fully operational umbrella covering all 1.4 billion of its people by 2022. The system is meant to collect all forms of digital data in order to calculate the “social trustworthiness” of individual citizens, and then reward or punish them by either allowing or restricting various opportunities and entitlements based on their scores. The formal and publicly stated aim of the system is to “allow the trustworthy to roam everywhere under heaven while making it hard for the discredited to take a single step.” An additional party slogan for the system is “Once discredited, limited everywhere.” The analogies to George Orwell’s novel 1984, and its themes of government overreach and Big Brother’s regimentation of social behavior are hard to deny.

Through AI programs built into internet platforms and mobile applications, the social credit system is intended to eventually track and collect data related to most everything an individual does. In almost all aspects, Chinese citizens orchestrate their lives through their phones. By and large, China has become a cashless society, and almost all transactions are executed through mobile, digital technology. Tencent’s WeChat app is almost unknown outside of China and Southeast Asia, but within China it has a mobile user base of over one billion people. Often described as the world’s super app, WeChat is used for everything from text, audio, and video messaging to information searches, purchases, banking, personal finances, and medical records management. It’s a compilation, in one app, of Google, Facebook, Amazon, PayPal, Instagram, Uber, and any number of other social and transactional applications you can think of. Cumulatively, it provides Chinese citizens an easy method of managing almost all aspects of their lives. But it also provides state-controlled companies, local governments, and the Communist Party a means of looking into the details of individual citizens’ lives that, by any democratic standard, would never be condoned.

Chinese bank accounts are linked directly to WeChat, and transactions are accomplished by direct, mobile pay (now using face verification). The users’ transactional histories and banking details are accessible to the app provider… and, consequently, to the government itself. Even in the conduct of their offline daily activities, almost all purchases are made via phone. For purchases and payments of every kind—including food from grocery stores and restaurants, goods and products at retail stores and markets, bike and car rentals, entertainment and transportation tickets, monthly mortgages, apartment rental payments, utility bills, and even government taxes — payments are made the same way, usually by scanning square two-dimensional barcodes, commonly called QR codes. All of it, transacted digitally and immediately traceable and forever recorded, makes the information regarding individual finances, spending habits, and financial status available and collectible by the systems that feed the government’s calculation of “social trustworthiness” scores.

And the social credit scores of China’s citizens aren’t affected only by their online activities, offline purchase and payment histories, and apparent fiscal responsibility. Offline, non-financial behaviors — both personal and social — at home, at work, and in the community are also increasingly being tracked and calculated. Social offenses that diminish social credit scores include smoking in public places, playing music too loud, texting while driving, taking drugs, purchasing alcohol, being publicly intoxicated, arguing with spouses, spreading information considered false or unacceptable, espousing religious beliefs, loitering, littering, and even walking pets without leashes. In essence, any violation of “acceptable” social norms can adversely impact one’s social credit score.

As a result, Chinese citizens can find themselves blacklisted or otherwise restricted from renting cars, buying train or airplane tickets, obtaining favorable loan rates, acquiring insurance, purchasing real estate or otherwise obtaining affordable housing, making financial investments, and even attending preferred schools or qualifying for certain jobs and career opportunities. Again, many Chinese consider these restraints fair exchanges for other government and technological benefits. It’s important to realize, however, that most of China’s population has never had the opportunity to enjoy anything approaching the sort of privacy rights generally expected and considered fundamental in democratic societies. As a consequence of its long history of authoritarian rule, many basic freedoms have never been granted in China as absolute entitlements. They’ve only been provided, when and if at all, from government authority and only in exchange for conformity or social silence, a quid pro quo. What citizens of democratic governments consider and demand as undeniable rights, Chinese citizens have only enjoyed as trade-offs for behaving consistently with Communist Party ideology. This is, after all, a government that effectively imposed a one-child policy on all married couples from 1979 until 2015—although, for some of those years, couples were allowed to have a second child, but only if their first was a girl. In any event, there’s little the Chinese can do to alter their reality. As has long been the Communist Party strategy, conformity is the only real option.

And just as technology is used to inform authorities of nonconforming behavior, it’s also now being used to actively measure proof of conformity—in some cases by tracking citizens’ consumption of loyalist Communist Party information and propaganda. In early 2019, a mobile app called Xuexi Qiangguo was released by the Communist Party’s publicity arm that requires users to register by providing their mobile phone numbers and full names. Translated roughly as “study Xi to strengthen the nation,” the app allows users to earn “study” points by logging onto it, reading articles, watching videos and documentaries about Xi Jinping, and taking multiple-choice quizzes on what they’ve learned about the party’s policies and doctrines. Government directives were issued following the app’s release that instructed party staff (or cadres) and members, of which there are 90 million, to download the app — with strong suggestions that they use it every day to increase their points, exhibit their loyalty, and earn benefits. Xuexi Qiangguo quickly became the most downloaded app in China, and yet another tool of state information efforts and social control. Before long, workers and users of the app found themselves spending long hours logged on, each day — feeling forced by the party, their employers, and colleagues to earn certain point levels. Reports surfaced that individuals’ scores were being posted as another way to cause social shaming of those not showing or proving sufficient proof of party loyalty and communist ideals.