The Inevitable: Understanding the 12 Technological Forces That Will Shape Our Future (2016)
10
TRACKING
We are opaque to ourselves and need all the help we can get to decipher who we are. One modern aid is self-measurement. But the noble pursuit of unmasking our hidden nature with self-measurement has a short history. Until recently it took an especially dedicated person to find a way to measure themselves without fooling themselves. Scientific self-tracking was expensive, troublesome, and limited. But in the last few years extremely tiny digital sensors that cost a few pennies have made recording parameters so easy (just click a button), and the varieties of parameters so vast, that almost anyone can now measure a thousand different aspects of themselves. Already these self-experiments have started to change our ideas of medicine, health, and human behavior.
Digital magic has shrunk devices such as thermometers, heart rate monitors, motion trackers, brain wave detectors, and hundreds of other complex medical appliances to the size of words on this page. A few are shrinking to the size of the period following this sentence. These macroscopic measurers can be inserted into watches, clothes, spectacles, or phones, or inexpensively dispersed in our rooms, cars, offices, and public spaces.
In the spring of 2007 I was hiking with Alan Greene, a doctor friend of mine, in the overgrown hills behind my house in northern California. As we slowly climbed up the dirt path to the summit, we discussed a recent innovation: a tiny electronic pedometer that slipped into the laces of a shoe to record each step, then saved the data to an iPod for later analysis. We could use this tiny device to count the calories as we climbed and to track our exercise patterns over time. We began to catalog other available ways to measure our activities. A week later, I took the same hike with Gary Wolf, a writer for Wired magazine, who was curious about the social implications of these emerging self-tracking devices. There were only a dozen existing ones, but we both could see clearly that tracking technology would explode as sensors steadily got smaller. What to call this cultural drift? Gary pointed out that by relying on numbers instead of words we were constructing a “quantified self.” So in June 2007 Gary and I announced on the internets that we would host a “Quantified Self” Meetup, open to absolutely anyone who thought they were quantifying themselves. We left the definition wide open to see who would show up. More than 20 people arrived at my studio in Pacifica, California, for this first event.
The diversity of what they were tracking astounded us: They measured their diet, fitness, sleep patterns, moods, blood factors, genes, location, and so on in quantifiable units. Some were making their own devices. One guy had been self-tracking for five years in order to maximize his strength, stamina, concentration, and productivity. He was using self-tracking in ways we had not imagined. Today there are 200 Quantified Self Meetup groups around the world, with 50,000 members. And every month, without fail, for eight years, someone at a Quantified Self meeting has demo’d an ingenious new way to track an aspect of their life that seemed unlikely or impossible a moment before. A few individuals stand out for their extreme habits. But what seems extreme today will soon become the new normal.
Computer scientist Larry Smarr tracks about a hundred health parameters on a daily basis, including his skin temperature and galvanic skin response. Every month he sequences the microbial makeup of his excrement, which mirrors the makeup of his gut microfauna, which is fast becoming one of the most promising frontiers in medicine. Equipped with this flow of data, and with a massive amount of amateur medical sleuthing, Smarr self-diagnosed the onset of Crohn’s disease, or ulcerative colitis, in his own body, before he or his doctors noticed any symptoms. Surgery later confirmed his self-tracking.
Stephen Wolfram is the genius behind Mathematica, a clever software app that is a math processor (instead of a word processor). Being a numbers guy, Wolfram applied his numeracy to the 1.7 million files he archived about his life. He processed all his outgoing and incoming email for 25 years. He captured every keystroke for 13 years, logged all his phone calls, his steps, his room-to-room motion in his home/office, and his GPS location outside his house. He tracked how many edits he made while writing his books and papers. Using his own Mathematica program, he turned his self-tracking into a “personal analytics” engine, which illuminated patterns in his routines over several decades. Some patterns were subtle enough, such as the hours when he is most productive, that he had not detected them until he analyzed his own data.
Nicholas Felton is a designer who has also tracked and analyzed all of his emails, messages, Facebook and Twitter postings, phone calls, and travel for the past five years. Every year he generates an annual report in which he visualizes the previous year’s data findings. In 2013 he concluded that he was productive on average 49 percent of the time, but most productive on Wednesdays, when he was 57 percent productive. At any given moment there is a 43 percent chance he is alone. He spent a third of his life (32 percent) sleeping. He used this quantitative review to help him “do a better job,” including remembering the names of people he met.
At Quantified Self meetings we’ve heard from people who track their habitual tardiness, or the amount of coffee they drink, their alertness, or the number of times they sneeze. I can honestly say that anything that can be tracked is being tracked by someone somewhere. At a recent international Quantified Self conference, I made this challenge: Let’s think of the most unlikely metric we can come up with and see if someone is tracking it. So I asked a group of 500 self-trackers: Is anyone tracking their fingernail growth? That seemed pretty absurd. One person raised their hand.
Shrinking chips, stronger batteries, and cloud connectivity has encouraged some self-trackers to attempt very long-term tracking. Particularly of one’s health. Most people are lucky to see a doctor once a year to get some aspect of their health measured. But instead of once a year, imagine that every day, all day, invisible sensors measured and recorded your heart rate, blood pressure, temperature, glucose, blood serum, sleep patterns, body fat, activity levels, mood, EKG brain functions, and so on. You would have hundreds of thousands of data points for each of these traits. You would have evidence while at both rest and at full stress, while sick and healthy, in all seasons, all conditions. Over the years you would gain a very accurate measurement of your normal—the narrow range your levels meander in. It turns out that, in medicine, normal is a fictional average. Your normal is not my normal and vice versa. The average normal is not very useful to you specifically. But with long-term self-tracking, you’d arrive at a very personal baseline—your normal—which becomes invaluable when you are not feeling well, or when you want to experiment.
The achievable dream in the near future is to use this very personal database of your body’s record (including your full sequence of genes) to construct personal treatments and personalized medicines. Science would use your life’s log to generate treatments specifically for you. For instance, a smart personalized pill-making machine in your home (described in Chapter 7) would compound medicines in the exact proportions for your current bodily need. If the treatment in the morning eased the symptoms, the dosage in the evening would be adjusted by the system.
The standard way of doing medical research today is to run experiments on as many subjects as one possibly can. The higher the number (N) of subjects, the better. An N of 100,000 random people would be the most accurate way to extrapolate results to the entire population of the country because the inevitable oddballs within the test population would average out and disappear from the results. In fact, the majority of medical trials are conducted with 500 or fewer participants for economic reasons. But a scientific study where N=500, if done with care, can be good enough for an FDA drug approval.
A quantified-self experiment, on the other hand, is just N=1. The subject is yourself. At first it may seem that an N=1 experiment is not scientifically valid, but it turns out that it is extremely valid to you. In many ways it is the ideal experiment because you are testing the variable X against the very particular subject that is your body and mind at one point in time. Who cares whether the treatment works on anyone else? What you want to know is, How does it affect me? An N=1 provides that laser-focused result.
The problem with an N=1 experiment (which was once standard procedure for all medicine before the age of science) is not that the results aren’t useful (they are), but that it is very easy to fool yourself. We all have hunches and expectations about our bodies, or about things we eat, or ideas of how the world works (such as the theory of vapors, or vibrations, or germs), that can seriously blind us to what is really happening. We suspect malaria is due to bad air, so we move to higher ground, and that helps, a little. We suspect gluten is giving us bloat, and so we tend to find evidence in our lives that it is the culprit and then we ignore contrary evidence that it doesn’t matter. We are particularly susceptible to bias when we are hurting or desperate. An N=1 experiment can work only if we can separate the ordinary expectations of the experimenter from those of the subject, but since one person plays both roles, this is extremely hard. This kind of inbred prejudice is exactly what large randomized double-blind trials were invented to overcome. The subject is unaware of the parameters of the test and therefore cannot be biased. What helps overcome some of our self-fooling in an N=1 experiment in the new era of self-tracking is automatic instrumentation (having a sensor make the measurement many times for long periods so it is “forgotten” by the subject) and being able to track many variables at once to distract the subject, and then using statistical means later to try to unravel any patterns.
We know from many classic large population studies that often the medicine we take works because we believe it will work. This is otherwise known as the placebo effect. These quantified-self tricks don’t fully counter the placebo effect; rather they work with it. If the intervention is producing a measurable improvement in you, then it works. Whether this measurable improvement is caused by the placebo effect doesn’t matter since we only care what effect it has on this N=1 subject. Thus a placebo effect can be positive.
In formal studies, you need a control group to offset your bias toward positive results. So in lieu of a control group in an N=1 study, a quantified-self experimenter uses his or her own baseline. If you track yourself long enough, with a wide variety of metrics, then you can establish your behavior outside (or before) the experiment, which effectively functions as the control for comparison.
✵ ✵ ✵
All this talk about numbers hides an important fact about humans: We have lousy mathematical intuitions. Our brains don’t do statistics well. Math is not our natural language. Even extremely visual plots and numerical graphs demand superconcentration. In the long term, the quantification in the quantified self will become invisible. Self-tracking will go far beyond numbers.
Let me give you an example. In 2004, Udo Wachter, an IT manager in Germany, took the guts of a small digital compass and soldered it into a leather belt. He added 13 miniature piezoelectric vibrators, like the ones that vibrate your smartphone, and buried them along the length of the belt. Finally he hacked the electronic compass so that instead of displaying north on a circular screen, it vibrated different parts of the belt when it was clasped into a circle. The section of the circle “facing” north would always vibrate. When Udo put the belt on, he could feel northness on his waist. Within a week of always wearing the north belt, Udo had an unerring sensation of “north.” It was unconscious. He could point in the direction without thinking. He just knew. After several weeks he acquired an additional heightened sense of location, of where he was in a city, as if he could feel a map. Here the quantification from digital tracking was subsumed into a wholly new bodily sensation. In the long term this is the destiny of many of the constant streams of data flowing from our bodily sensors. They won’t be numbers; they will be new senses.
These new synthetic senses are more than entertaining. Our natural senses evolved over millions of years to ensure that we survived in a world of scarcity. The threat of not having enough calories, salt, or fat was relentless. As Malthus and Darwin showed, every biological population expands right to the limit of its starvation. Today, in a world made abundant by technology, the threat to survival is due to an excess of good stuff. Too much goodness throws our metabolism and psychology out of kilter. But our bodies can’t register these new imbalances very well. We didn’t evolve to sense our blood pressure or glucose levels. But our technology can. For instance, a new self-tracking device, the Scout from Scanadu, is the size of an old-timey stopwatch. By touching it to your forehead, it will measure your blood pressure, variable heart rate, heart performance (ECG), oxygen level, temperature, and skin conductance all in a single instant. Someday it will also measure your glucose levels. More than one startup in Silicon Valley is developing a noninvasive, prickless blood monitor to analyze your blood factors daily. You’ll eventually wear these. By taking this information and feeding it back not in numbers but in a form we can feel, such as a vibration on our wrist or a squeeze on our hip, the device will equip us with a new sense about our bodies that we didn’t evolve but desperately need.
✵ ✵ ✵
Self-tracking is much broader than health. It is as big as our life itself. Tiny wearable digital eyes and ears can record every second of our entire day—who we saw and what we said—to aid our memories. Our stream of email and text, when saved, forms an ongoing diary of our mind. We can add the record of the music we listened to, the books and articles we read, the places we visited. The significant particulars of our routine movements and meetings, as well as nonroutine events and experiences, can also be funneled into bits and merged into a chronological flow.
This flow is called a lifestream. First described by the computer scientist David Gelernter in 1999, a lifestream is more than just a data archive. Gelernter conceived of lifestreams as a new organizing interface for computers. Instead of an old desktop, a new chronological stream. Instead of a web browser, a stream browser. Gelernter and his graduate student Eric Freeman define the lifestream architecture like this:
A lifestream is a time-ordered stream of documents that functions as a diary of your electronic life; every document you create and every document other people send you is stored in your lifestream. The tail of your stream contains documents from the past (starting with your electronic birth certificate). Moving away from the tail and toward the present, your stream contains more recent documents—pictures, correspondence, bills, movies, voice mail, software. Moving beyond the present and into the future, the stream contains documents you will need: reminders, calendar items, to-do lists.
You can sit back and watch new documents arrive: they’re plunked down at the head of the stream. You browse the stream by running your cursor down it—touch a document in the display and a page pops out far enough for you to glance at its contents. You can go back in time or go to the future and see what you’re supposed to be doing next week or next decade. Your entire cyberlife is right there in front of you.
Every person generates their own lifestream. When I meet with you, your lifestream and mine intersect in time. If we are going to meet next week, they intersect in the future; if we met, or even shared a photo last year, then our lifestreams intersected in the past. Our streams become richly braided with incredible complexity, but the strict chronological nature of each one means that they are easy to navigate. We naturally slide along a timeline to home in on an event. “It happened after the Christmas trip but before my birthday.”
The advantage of a lifestream as an organizational metaphor, Gelernter says, is that “the question ‘Where did I put that piece of information?’ always has exactly one answer: It’s in my stream. The idea of a timeline, a chronology, a diary, a daily journal, or a scrapbook is so much older and so much more organic and ingrained in human culture and history than the idea of a file hierarchy.” As Gelernter told a Sun computer representative, “When I acquire a new memory of (let’s say) talking to Melissa on a sunny afternoon outside the Red Parrot—I don’t have to give this memory a name, or stuff it in a directory. I can use anything in the memory as a retrieval key. I shouldn’t have to name electronic documents either, or put them in directories. I can shuffle other streams into mine—to the extent I have permission to use other people’s streams. My own personal stream, my electronic life story, can have other streams shuffled into it—streams belonging to groups or organizations I’m part of. And eventually I’ll have, for example, newspaper and magazine streams shuffled into my stream also.”
Gelernter tried many times since 1999 to produce a commercial version of his software, but it never took off. A company that bought his patents sued Apple for stealing his Lifestream idea and using it in its Time Machine backup system. (To restore a file in Apple’s Time Machine, you slide along a timeline to the date you want and there is “snapshot” of your computer’s content on that date.)
But in social media today we have several working examples of lifestreams: Facebook (and in China, WeChat). Your Facebook stream is an ongoing flow of pictures, updates, links, pointers, and other documentation from your life. New pieces are continually added to the front of the stream. If you care to, you can add widgets to Facebook that capture the music you are listening to or the movies you are streaming. Facebook even provides a timeline interface to review the past. Over a billion other people’s streams can intersect with yours. When a friend (or stranger) likes a post or tags a person in a picture, those two streams mingle. And each day Facebook is adding more current events and news streams and company updates into the worldstream.
But even all this is still only part of the picture. Lifestreaming can be thought of as an active, conscious tracking. People actively curate their stream when they snap a photo on their phones, or tag friends, or deliberately check-in to a place with Foursquare. Even their exercise Fitbit data, counting steps, is active, in that it is meant to be paid attention to. You can’t change your behavior unless you pay attention in some capacity.
There is an equally important domain of tracking that is not conscious or active. This passive type of tracking is sometimes called lifelogging. The idea is to simply, mechanically, automatically, mindlessly, completely track everything all the time. Record everything that is recordable without prejudice, and for all your life. You only pay attention to it in the future when you may need it. Lifelogging is a hugely wasteful and inefficient process since most of what you lifelog is never used. But like many inefficient processes (such as evolution), it also contains genius. Lifelogging is possible now only because computation and storage and sensors have become so cheap that we can waste them with little cost. But creative “wasting” of computation has been the recipe for many of the most successful digital products and companies, and the benefits of lifelogging also lie in its extravagant use of computation.
Among the very first to lifelog was Ted Nelson in the mid-1980s (although he didn’t call it that). Nelson, who invented hypertext, recorded every conversation he had with anyone on audio or videotape, no matter where or of what importance. He met and spoke to thousands of people, so he had a large rental storage container full of tapes. The second person was Steve Mann in the 1990s. Mann, then at MIT (now at the University of Toronto), outfitted himself with a head-mounted camera and recorded his daily life on videotape. Everything, all day, all year. For 25 years, if he was awake, he kept the camera on. His gear had a tiny screen over one eye and the camera recorded his first-person viewpoint, foreshadowing Google Glass by two decades. When we first met in July 1996, Mann sometimes called what he did “Quantimetric Self Sensing.” Because there was a camera half obscuring his face, I found it was hard to be natural around Mann, but he is still routinely recording his whole life all the time.
But Gordon Bell at Microsoft Research may be the paragon of lifeloggers. For six years beginning in 2000, Bell documented every aspect of his work life in a grand experiment he called MyLifeBits. Bell wore a special custom-made camera around his neck that noticed a person’s body heat if they were near and photographed them every 60 seconds. Bell’s bodycam also snapped a picture if it detected a change in light of a new place. Bell recorded and archived every keystroke on his computer, every email, every website he visited, every search he made, every window on his computer and how long it remained opened. He also recorded many of his conversations, which enabled him to “scroll back” whenever there was disagreement on what had been said. He also scanned all his incoming pieces of paper into digital files and transcribed every phone conversation (with permission). Part of the intent of this experiment was to find out what kind of lifelogging tools Microsoft might want to invent to help workers manage the ocean of data this lifelogging generates—because making sense of all this data is a far bigger challenge than merely recording it.
The point of lifelogging is to create total recall. If a lifelog records everything in your life, then it could recover anything you experienced even if your meaty mind may have forgotten it. It would be like being able to google your life, if in fact your life were being indexed and fully saved. Our biological memories are so spotty that any compensation would be a huge win. Bell’s experimental version of total recall helped increase his productivity. He could verify facts from previous conversations or recover insights he had forgotten. His system had little problem recording his life into bits, but he learned retrieving the meaningful bits needed better tools.
I’ve been wearing a tiny camera that I clip to my shirt, inspired by the one Gordon Bell wore. The Narrative is about an inch square. It takes a still photo every minute all day long, or whenever I wear it. I can also force a shot by tapping on the square twice. The photos go to the cloud, where they are processed and then sent back to my phone or the web. Narrative’s software smartly groups the images into scenes during my day and then selects the most representative three images for each scene. This reduces the flood of images. Using this visual summary, I can flick through the 2,000 images per day very quickly, and then expand the stream of a particular scene for more images to find the exact moment I want to recall. I can easily browse the lifestream of an entire day in less than a minute. I find it mildly useful as a very detailed visual diary, a lifelogging asset that needs to be invaluable only a couple of times a month to make it worthwhile.
Typical users, Narrative has found, employ this photo diary while they attend conferences, or go on vacation, or want to record an experience. Recalling a conference is ideal. The continuous camera captures the many new people you meet. Better than a business card, you can much more easily recall them years later, and what they talked about, by browsing your lifestream. The photo lifestream is a strong prompt for vacations and family events. For instance, I recently used the Narrative during my nephew’s wedding. It includes not only the iconic moments shared by everyone, but captured the conversations I had with people I had not talked to before. This version of Narrative does not record audio, but the next version will. In his research Bell discovered that the most informative media to capture is audio, prompted and indexed by photos. Bell told me that if he could have only one, he’d rather have an audio log of his day than a visual log.
An embrace of an expanded version of lifelogging would offer these four categories of benefits:
· A constant 24/7/365 monitoring of vital body measurements. Imagine how public health would change if we continuously monitored blood glucose in real time. Imagine how your behavior would change if you could, in near real time, detect the presence or absence of biochemicals or toxins in your blood picked up from your environment. (You might conclude: “I’m not going back there!”) This data could serve both as a warning system and also as a personal base upon which to diagnose illness and prescribe medicines.
· An interactive, extended memory of people you met, conversations you had, places you visited, and events you participated in. This memory would be searchable, retrievable, and shareable.
· A complete passive archive of everything that you have ever produced, wrote, or said. Deep comparative analysis of your activities could assist your productivity and creativity.
· A way of organizing, shaping, and “reading” your own life.
To the degree this lifelog is shared, this archive of information could be leveraged to help others work and to amplify social interactions. In the health realm, shared medical logs could rapidly advance medical discoveries.
For many skeptics, there are two challenges that will doom lifelogging to a small minority. First, current social pressure casts self-tracking as the geekiest thing you could possibly do. Owners of Google Glass quickly put them away because they didn’t like how they looked and they felt uncomfortable recording among their friends—or even uncomfortable explaining why they were not recording. As Gary Wolf said, “Recording in a diary is considered admirable. Recording in a spreadsheet is considered creepy.” But I believe we’ll quickly invent social norms and technological innovations to navigate the times when lifelogging is appropriate or not. When cell phones first appeared among the early adopters in the 1990s, there was a terrible cacophony of ringers. Cell phones rang at high decibels on trains, in bathrooms, in movie theaters. While talking on an early cell phone, people raised their voices as loud as the ringers. If you imagined back then what the world would sound like in the near future when everyone had a cell phone, you could only envision a nonstop racket. That didn’t happen. Silent vibrators were invented, people learned to text, and social norms prevailed. I can go to a movie today in which every person in the theater has a cell phone, and not hear one ring or even see one lighted screen. It’s considered not cool. We’ll evolve the same kind of social conventions and technical fixes that will make lifelogging acceptable.
Second, how can lifelogging work when each person will generate petabytes, if not exabytes, of data each year? There is no way anyone can troll through that ocean of bits. You’ll drown without a single insight. That is roughly true with today’s software. Making sense of the data is an immense, time-consuming problem. You have to be highly numerate, technically agile, and supremely motivated to extract meaning from the river of data you generate. That is why self-tracking is still a minority sport. However, cheap artificial intelligence will overcome much of this. The AI in research labs is already powerful enough to sift through billions of records and surface important, meaningful patterns. As just one example, the same AI at Google that can already describe what is going on in a random photo could (when it is cheap enough) digest the images from my Narrative shirt cam so that I can simply ask Narrative in plain English to find me the guy who was wearing a pirate hat at a party I attended a couple of years ago. And there it is, and his stream would be linked to mine. Or I could ask it to determine the kind of rooms that tend to raise my heart rate. Was it the color, the temperature, the height of the ceilings? Although it seems like wizardry now, this will be considered a very mechanical request in a decade, not very different from asking Google to find something—which would have been magical 20 years ago.
Still, the picture is not big enough. We—the internet of people—will track ourselves, much of our lives. But the internet of things is much bigger, and billions of things will track themselves too. In the coming decades nearly every object that is manufactured will contain a small sliver of silicon that is connected to the internet. One consequence of this wide connection is that it will become feasible to track how each thing is used with great precision. For example, every car manufactured since 2006 contains a tiny OBD chip mounted under the dashboard. This chip records how your car is used. It tracks miles driven, at what speed, times of sudden braking, speed of turns, and gas mileage. This data was originally designed to help repair the car. Some insurance companies, such as Progressive, will lower your auto insurance rates if you give them access to your OBD driving log. Safer drivers pay less. The GPS location of cars can also be tracked very accurately, so it would be possible to tax drivers based on which roads they use and how often. These usage charges could be thought of as virtual tolls or automatic taxation.
✵ ✵ ✵
The design of the internet of everything, and the nature of the cloud that it floats in, is to track data. The 34 billion internet-enabled devices we expect to add to the cloud in the next five years are built to stream data. And the cloud is built to keep the data. Anything touching this cloud that is able to be tracked will be tracked.
Recently, with the help of researcher Camille Hartsell, I rounded up all the devices and systems in the U.S. that routinely track us. The key word is “routinely.” I am leaving off this list the nonroutine tracking performed illegally by hackers, criminals, and cyberarmies. I also skip over the capabilities of the governmental agencies to track specific targets when and how they want to. (Governments’ ability to track is proportional to their budgets.) This list, instead, tallies the kind of tracking an average person might encounter on an ordinary day in the United States. Each example has been sourced officially or from a major publication.
Car movements—Every car since 2006 contains a chip that records your speed, braking, turns, mileage, accidents whenever you start your car.
Highway traffic—Cameras on poles and sensors buried in highways record the location of cars by license plates and fast-track badges. Seventy million plates are recorded each month.
Ride-share taxis—Uber, Lyft, and other decentralized rides record your trips.
Long-distance travel—Your travel itinerary for air flights and trains is recorded.
Drone surveillance—Along U.S. borders, Predator drones monitor and record outdoor activities.
Postal mail—The exterior of every piece of paper mail you send or receive is scanned and digitized.
Utilities—Your power and water usage patterns are kept by utilities. (Garbage is not cataloged, yet.)
Cell phone location and call logs—Where, when, and who you call (metadata) is stored for months. Some phone carriers routinely store the contents of calls and messages for days to years.
Civic cameras—Cameras record your activities 24/7 in most city downtowns in the U.S.
Commercial and private spaces—Today 68 percent of public employers, 59 percent of private employers, 98 percent of banks, 64 percent of public schools, and 16 percent of homeowners live or work under cameras.
Smart home—Smart thermostats (like Nest) detect your presence and behavior patterns and transmit these to the cloud. Smart electrical outlets (like Belkin) monitor power consumption and usage times shared to the cloud.
Home surveillance—Installed video cameras document your activity inside and outside the home, stored on cloud servers.
Interactive devices—Your voice commands and messages from phones (Siri, Now, Cortana), consoles (Kinect), smart TVs, and ambient microphones (Amazon Echo) are recorded and processed on the cloud.
Grocery loyalty cards—Supermarkets track which items you purchase and when.
E-retailers—Retailers like Amazon track not only what you purchase, but what you look at and even think about buying.
IRS—Tracks your financial situation all your life.
Credit cards—Of course, every purchase is tracked. Also mined deeply with sophisticated AI for patterns that reveal your personality, ethnicity, idiosyncrasies, politics, and preferences.
E-wallets and e-banks—Aggregators like Mint track your entire financial situation from loans, mortgages, and investments. Wallets like Square and PayPal track all purchases.
Photo face recognition—Facebook and Google can identify (tag) you in pictures taken by others posted on the web. The location of pictures can identify your location history.
Web activities—Web advertising cookies track your movements across the web. More than 80 percent of the top thousand sites employ web cookies that follow you wherever you go on the web. Through agreements with ad networks, even sites you did not visit can get information about your viewing history.
Social media—Can identify family members, friends, and friends of friends. Can identify and track your former employers and your current work mates. And how you spend your free time.
Search browsers—By default Google saves every question you’ve ever asked forever.
Streaming services—What movies (Netflix), music (Spotify), video (YouTube) you consume and when, and what you rate them. This includes cable companies; your watching history is recorded.
Book reading—Public libraries record your borrowings for about a month. Amazon records book purchases forever. Kindle monitors your reading patterns on ebooks—where you are in the book, how long you take to read each page, where you stop.
Fitness trackers—Your physical activity, time of day, sometimes location, often tracked all 24 hours, including when you sleep and when you are awake each day.
It is shockingly easy to imagine what power would accrue to any agency that could integrate all these streams. The fear of Big Brother stems directly from how technically easy it would be to stitch these together. At the moment, however, most of these streams are independent. Their bits are not integrated and correlated. A few strands may be coupled (credit cards and media usage, say), but by and large there is not a massive Big Brother-ish aggregate stream. Because they are slow, governments lag far behind what they could do technically. (Their own security is irresponsibly lax and decades behind the times.) Also, the U.S. government has not unified these streams because a thin wall of hard-won privacy laws holds them back. Few laws hold corporations back from integrating as much data as they can; therefore companies have become the proxy data gatherers for governments. Data about customers is the new gold in business, so one thing is certain: Companies (and indirectly governments) will collect more of it.
The movie Minority Report, based on a short story by Philip K. Dick, featured a not too distant future society that uses surveillance to arrest criminals before they commit a crime. Dick called that intervention “pre-crime” detection. I once thought Dick’s idea of “pre-crime” to be utterly unrealistic. I don’t anymore.
If you look at the above list of routine tracking today, it is not difficult to extrapolate another 50 years. All that was previously unmeasurable is becoming quantified, digitized, and trackable. We’ll keep tracking ourselves, we’ll keep tracking our friends, and our friends will track us. Companies and governments will track us more. Fifty years from now ubiquitous tracking will be the norm.
As I argue in Chapter 5 (Accessing), the internet is the world’s largest, fastest copy machine, and anything that touches it will be copied. The internet wants to make copies. At first this fact is deeply troubling to creators, both individual and corporate, because their stuff will be copied indiscriminately, often for free, when it was once rare and precious. Some people fought, and still fight, very hard against the bias to copy (movie studios and music labels come to mind) and some people chose and choose to work with the bias. Those who embrace the internet’s tendency to copy and seek value that can’t be easily copied (through personalization, embodiment, authentication, etc.) tend to prosper, while those who deny, prohibit, and try to thwart the network’s eagerness to copy are left behind to catch up later. Consumers, of course, love the promiscuous copies and feed the machine to claim their benefits.
This bias to copy is technological rather than merely social or cultural. It would be true in a different nation, even in a command economy, even with a different origin story, even on another planet. It is inevitable. But while we can’t stop copying, it does matter greatly what legal and social regimes surround ubiquitous copying. How we handle rewards for innovation, intellectual property rights and responsibilities, ownership of and access to the copies makes a huge difference to society’s prosperity and happiness. Ubiquitous copying is inevitable, but we have significant choices about its character.
Tracking follows a similar inevitable dynamic. Indeed, we can swap the term “tracking” in the preceding paragraphs for “copying” in the following paragraphs to get a sense of its parallels:
The internet is the world’s largest, fastest tracking machine, and anything that touches it that can be tracked will be tracked. What the internet wants is to track everything. We will constantly self-track, track our friends, be tracked by friends, companies, and governments. This is deeply troubling to citizens, and to some extent to companies as well, because tracking was previously seen as rare and expensive. Some people fight hard against the bias to track and some will eventually work with the bias. Those who figure out how to domesticate tracking, to make it civil and productive, will prosper, while those who try only to prohibit and outlaw it will be left behind. Consumers say they don’t want to be tracked, but in fact they keep feeding the machine with their data, because they want to claim their benefits.
This bias to track is technological rather than merely social or cultural. It would be true in a different nation, even in a command economy, even with a different origin story, even on another planet. But while we can’t stop tracking, it does matter greatly what legal and social regimes surround it. Ubiquitous tracking is inevitable but we have significant choices about its character.
✵ ✵ ✵
The fastest-increasing quantity on this planet is the amount of information we are generating. It is (and has been) expanding faster than anything else we can measure over the scale of decades. Information is accumulating faster than the rate we pour concrete (which is booming at a 7 percent increase annually), faster than the increases in the output of smartphones or microchips, faster than any by-product we generate, such as pollution or carbon dioxide.
Two economists at UC Berkeley tallied up the total global production information and calculated that new information is growing at 66 percent per year. This rate hardly seems astronomical compared with the 600 percent increase in iPods shipped in 2005. But that kind of burst is short-lived and not sustainable over decades (iPod production tanked in 2009). The growth of information has been steadily increasing at an insane rate for at least a century. It is no coincidence that 66 percent per year is the same as doubling every 18 months, which is the rate of Moore’s Law. Five years ago humanity stored several hundred exabytes of information. That is the equivalent of each person on the planet having 80 Library of Alexandrias. Today we average 320 libraries each.
There’s another way to visualize this growth: as an information explosion. Every second of every day we globally manufacture 6,000 square meters of information storage material—disks, chips, DVDs, paper, film—which we promptly fill up with data. That rate—6,000 square meters per second—is the approximate velocity of the shock wave radiating from an atomic explosion. Information is expanding at the rate of a nuclear explosion, but unlike a real atomic explosion, which lasts only seconds, this information explosion is perpetual, a nuclear blast lasting many decades.
In our everyday lives we generate far more information that we don’t yet capture and record. Despite the explosion in tracking and storage, most of our day-to-day life is not digitized. This unaccounted-for information is “wild” or “dark” information. Taming this wild information will ensure that the total amount of information we collect will keep doubling for many decades ahead.
An increasing percentage of the information gathered each year is due to the information that we generate about that information. This is called meta-information. Every digital bit we capture encourages us to generate another bit concerning it. When the activity bracelet on my arm captures one step, it immediately adds time stamp data to it; it then creates yet more new data linking it to other step bits, and then generates tons of new data when it is plotted on a graph. Likewise, the musical data captured when a young girl plays her electric guitar on her live video stream becomes a foundation for generating indexing data about that clip, creating bits of data for “likes” or the many complex data packets needed to share that among her friends. The more data we capture, the more data we generate upon it. This metadata is growing even faster than the underlying information and is almost unlimited in its scale.
Metadata is the new wealth because the value of bits increases when they are linked to other bits. The least productive life for a bit is to remain naked and alone. A bit uncopied, unshared, unlinked with other bits will be a short-lived bit. The worst future for a bit is to be parked in some dark isolated data vault. What bits really want is to hang out with other related bits, be replicated widely, and maybe become a metabit, or an action bit in a piece of durable code. If we could personify bits, we’d say:
Bits want to move.
Bits want to be linked to other bits.
Bits want to be reckoned in real time.
Bits want to be duplicated, replicated, copied.
Bits want to be meta.
Of course, this is pure anthropomorphization. Bits don’t have wills. But they do have tendencies. Bits that are related to other bits will tend to be copied more often. Just as selfish genes tend to replicate, bits do too. And just as genes “want” to code for bodies that help them replicate, selfish bits also “want” systems that help them replicate and spread. Bits behave as if they want to reproduce, move, and be shared. If you rely on bits for anything, this is good to know.
Since bits want to duplicate, replicate, and be linked, there’s no stopping the explosion of information and the science fiction levels of tracking. Too many of the benefits we humans covet derive from streams of data. Our central choice now is: What kind of total tracking do we want? Do we want a one-way panopticon, where “they” know about us but we know nothing about them? Or could we construct a mutual, transparent kind of “coveillance” that involves watching the watchers? The first option is hell, the second tractable.
Not too long ago, small towns were the norm. The lady across the street from you tracked your every coming and going. She peeked out through her window and watched when you went to the doctor, and saw that you brought home a new TV, and knew who stayed with you over the weekend. But you also watched her through your window. You knew what she did on Thursday nights, and down at the corner drugstore you saw what she put in her basket. And there were mutual benefits from this mutual surveillance. If someone she did not recognize walked into your house when you were gone, she called the cops. And when she was gone, you picked up her mail from her mailbox. This small-town coveillance worked because it was symmetrical. You knew who was watching you. You knew what they did with the information. You could hold them accountable for its accuracy and use. And you got benefits for being watched. Finally, you watched your watchers under the same circumstances.
We tend to be uncomfortable being tracked today because we don’t know much about who is watching us. We don’t know what they know. We have no say in how the information is used. They are not accountable to correct it. They are filming us but we can’t film them. And the benefits for being watched are murky and concealed. The relationship is unbalanced and asymmetrical.
Ubiquitous surveillance is inevitable. Since we cannot stop the system from tracking, we can only make the relationships more symmetrical. It’s a way of civilizing coveillance. This will take both technological fixes and new social norms. Science fiction author David Brin calls this the “Transparent Society,” which is also the name of his 1999 book summing up the idea. For a hint of how this scenario may be possible, consider Bitcoin, the decentralized open source currency described in Chapter 6 (Sharing). Bitcoin transparently logs every transaction in its economy in a public ledger, thereby making all financial transactions public. The validity of a transaction is verified by a coveillance of other users rather than the surveillance of central bank. For another example, traditional encryption used secret proprietary codes guarded closely. But a clever improvement called public key encryption (such as PGP) relies on code that anyone can inspect, including a public key, and therefore anyone can trust and verify. Neither of these innovations remedy existing asymmetries of knowledge; rather they demonstrate how it is possible to engineer systems that are powered by mutual vigilance.
In a coveillant society a sense of entitlement can emerge: Every person has a human right to access, and a right to benefit from, the data about themselves. But every right requires a duty, so every person has a human duty to respect the integrity of information, to share it responsibly, and to be watched by the watched.
The alternatives to coveillance are not promising. Outlawing the expansion of easy tracking will probably be as ineffectual as outlawing easy copying. I am a supporter of the whistle-blower Edward Snowden, who leaked tens of thousands of classified NSA files, revealing their role in secretly tracking citizens, primarily because I think the big sin of many governments, including the U.S., is lying about their tracking. Big governments are tracking us, but with no chance for symmetry. I applaud Snowden’s whistle-blowing not because I believe it will reduce tracking, but because it can increase transparency. If symmetry can be restored so we can track who is tracking, if we can hold the trackers accountable by law (there should be regulation) and responsible for accuracy, and if we can make the benefits obvious and relevant, then I suspect the expansion of tracking will be accepted.
I want my friends to treat me as an individual. To enable that kind of relationship I have to be open and transparent and share my life with my friends so they know enough about me to treat me personally. I want companies to treat me as an individual too, so I have be open, transparent, and sharing with them as well to enable them to be personal. I want my government to treat me as an individual, so I have to reveal personal information to it to be treated personally. There is a one-to-one correspondence between personalization and transparency. Greater personalization requires greater transparency. Absolute personalization (vanity) requires absolute transparency (no privacy). If I prefer to remain private and opaque to potential friends and institutions, then I must accept I will be treated generically, without regard to my specific particulars. I’ll be an average number.
Now imagine these choices pinned on a slider bar. On the left side of the slot is the pair personal/transparent. On the right side is the pair private/generic. The slider can slide to either side or anywhere in between. The slider is an important choice we have. Much to everyone’s surprise, though, when technology gives us a choice (and it is vital that it remain a choice), people tend to push the slider all the way over to the personal/transparent side. They’ll take transparent personalized sharing. No psychologist would have predicted that 20 years ago. If today’s social media has taught us anything about ourselves as a species, it is that the human impulse to share overwhelms the human impulse for privacy. This has surprised the experts. So far, at every juncture that offers a choice, we’ve tilted, on average, toward more sharing, more disclosure, more transparency. I would sum it up like this: Vanity trumps privacy.
For eons and eons humans have lived in tribes and clans where every act was open and visible and there were no secrets. Our minds evolved with constant co-monitoring. Evolutionarily speaking, coveillance is our natural state. I believe that, contrary to our modern suspicions, there won’t be a backlash against a circular world in which we constantly track each other because humans have lived like this for a million years, and—if truly equitable and symmetrical—it can feel comfortable.
That’s a big if. Obviously, the relation between me and Google, or between me and the government, is inherently not equitable or symmetrical. The very fact they have access to everyone’s lifestream, while I have access only to mine, means they have access to a qualitatively greater thing. But if some symmetry can be restored so that I can be part of holding their greater status to a greater accountability, and I benefit from their greater view, it might work. Put it this way: For sure cops will videotape citizens. That’s okay as long as citizens can videotape cops, and can get access to the cops’ videos, and share them to keep the more powerful accountable. That’s not the end of the story, but it’s how a transparent society has to start.
What about that state we used to call privacy? In a mutually transparent society, is there room for anonymity?
The internet makes true anonymity more possible today than ever before. At the same time the internet makes true anonymity in physical life much harder. For every step that masks us, we move two steps toward totally transparent unmasking. We have caller ID, but also caller ID block, and then caller ID-only filters. Coming up: biometric monitoring (iris + fingerprint + voice + face + heat rhythm) and little place to hide. A world where everything about a person can be found and archived is a world with no privacy. That’s why many smart people are eager to maintain the option of easy anonymity—as a refuge for the private.
However, in every system that I have experienced where anonymity becomes common, the system fails. Communities saturated with anonymity will either self-destruct or shift from the purely anonymous to the pseudo-anonymous, as in eBay, where you have a traceable identity behind a persistent invented nickname. There is the famous outlaw gang Anonymous, an ad hoc rotating band of totally anonymous volunteers. They are online vigilantes with fickle targets. They will take down ISIS militant Twitter accounts, or a credit card company that gets in their way. But while they continue to persist and make trouble, it is not clear whether their net contribution to society is positive or negative.
For the civilized world, anonymity is like a rare earth metal. In larger doses these heavy metals are some of the most toxic substances known to a life. They kill. Yet these elements are also a necessary ingredient in keeping a cell alive. But the amount needed for health is a mere hard-to-measure trace. Anonymity is the same. As a trace element in vanishingly small doses, it’s good, even essential for the system. Anonymity enables the occasional whistle-blower and can protect the persecuted fringe and political outcasts. But if anonymity is present in any significant quantity, it will poison the system. While anonymity can be used to protect heroes, it is far more commonly used as a way to escape responsibility. That’s why most of the brutal harassment on Twitter, Yik Yak, Reddit, and other sites is delivered anonymously. A lack of responsibility unleashes the worst in us.
There’s a dangerous idea that massive use of anonymity is a noble antidote to the prying state. This is like pumping up the level of heavy metals in your body to make it stronger. Rather, privacy can be gained only by trust, and trust requires persistent identity. In the end, the more trust the better, and the more responsibility the better. Like all trace elements, anonymity should never be eliminated completely, but it should be kept as close to zero as possible.
✵ ✵ ✵
Everything else in the realm of data is headed to infinity. Or at least astronomical quantities. The average bit effectively becomes anonymous, almost undetectable, when measured against the scale of planetary data. In fact, we are running out of prefixes to indicate how big this new realm is. Gigabytes are on your phone. Terabytes were once unimaginably enormous, yet today I have three terabytes sitting on my desk. The next level up is peta. Petabytes are the new normal for companies. Exabytes are the current planetary scale. We’ll probably reach zetta in a few years. Yotta is the last scientific term for which we have an official measure of magnitude. Bigger than yotta is blank. Until now, any more than a yotta was a fantasy not deserving an official name. But we’ll be flinging around yottabytes in two decades or so. For anything beyond yotta, I propose we use the single term “zillion”—a flexible notation to cover any and all new magnitudes at this scale.
Large quantities of something can transform the nature of those somethings. More is different. Computer scientist J. Storrs Hall writes: “If there is enough of something, it is possible, indeed not unusual, for it to have properties not exhibited at all in small, isolated examples. There is no case in our experience where a difference of a factor of a trillion doesn’t make a qualitative, as opposed to merely a quantitative, difference. A trillion is essentially the difference in weight between a dust mite, too small to see and too light to feel, and an elephant. It’s the difference between $50 and a year’s economic output for the entire human race. It’s the difference between the thickness of a business card and the distance from here to the moon.”
Call this difference zillionics.
A zillion neurons give you a smartness a million won’t. A zillion data points will give you insight that a mere hundred thousand don’t. A zillion chips connected to the internet create a pulsating, vibrating unity that 10 million chips can’t. A zillion hyperlinks will give you information and behavior you could never expect from a hundred thousand links. The social web runs in the land of zillionics. Artificial intelligence, robotics, and virtual realities all require mastery of zillionics. But the skills needed to manage zillionics are daunting.
The usual tools for managing big data don’t work very well in this territory. A statistical prediction technique such as a maximum likelihood estimation (MLE) breaks down because in the realm of zillionics the maximum likely estimate becomes improbable. Navigating zillions of bits, in real time, will require entire new fields of mathematics, completely new categories of software algorithms, and radically innovative hardware. What wide-open opportunities!
The coming new arrangement of data at the magnitude of zillionics promises a new machine at the scale of the planet. The atoms of this vast machine are bits. Bits can be arranged into complicated structures just as atoms are arranged into molecules. By raising the level of complexity, we elevate bits from data to information to knowledge. The full power of data lies in the many ways it can be reordered, restructured, reused, reimagined, remixed. Bits want to be linked; the more relationships a bit of data can join, the more powerful it gets.
The challenge is that the bulk of usable information today has been arranged in forms that only humans understand. Inside a snapshot taken on your phone is a long string of 50 million bits that are arranged in a way that makes sense to a human eye. This book you are reading is about 700,000 bits ordered into the structure of English grammar. But we are at our limits. Humans can no longer touch, let along process, zillions of bits. To exploit the full potential of the zillionbytes of data that we are harvesting and creating, we need to be able to arrange bits in ways that machines and artificial intelligences can understand. When self-tracking data can be cognified by machines, it will yield new, novel, and improved ways of seeing ourselves. In a few years, when AIs can understand movies, we’ll be able to repurpose the zillionbytes of that visual information in entirely new ways. AI will parse images like we parse an article, and so it will be able to easily reorder image elements in the way we reorder words and phrases when we write.
Entirely new industries have sprung up in the last two decades based on the idea of unbundling. The music industry was overturned by technological startups that enabled melodies to be unbundled from songs and songs unbundled from albums. Revolutionary iTunes sold single songs, not albums. Once distilled and extracted from their former mixture, musical elements could be reordered into new compounds, such as shareable playlists. Big general-interest newspapers were unbundled into classifieds (Craigslist), stock quotes (Yahoo!), gossip (BuzzFeed), restaurant reviews (Yelp), and stories (the web) that stood and grew on their own. These new elements can be rearranged—remixed—into new text compounds, such as news updates tweeted by your friend. The next step is to unbundle classifieds, stories, and updates into even more elemental particles that can be rearranged in unexpected and unimaginable ways. Sort of like smashing information into ever smaller subparticles that can be recombined into a new chemistry. Over the next 30 years, the great work will be parsing all the information we track and create—all the information of business, education, entertainment, science, sport, and social relations—into their most primeval elements. The scale of this undertaking requires massive cycles of cognition. Data scientists call this stage “machine readable” information, because it is AIs and not humans who will do this work in the zillions. When you hear a term like “big data,” this is what it is about.
Out of this new chemistry of information will arise thousands of new compounds and informational building materials. Ceaseless tracking is inevitable, but it is only the start.
We are on our way to manufacturing 54 billion sensors every year by 2020. Spread around the globe, embedded in our cars, draped over our bodies, and watching us at home and on public streets, this web of sensors will generate another 300 zillionbytes of data in the next decade. Each of those bits will in turn generate twice as many metabits. Tracked, parsed, and cognified by utilitarian AIs, this vast ocean of informational atoms can be molded into hundreds of new forms, novel products, and innovative services. We will be astounded at what is possible by a new level of tracking ourselves.