Total Recall: How the E-Memory Revolution Will Change Everything - C. Gordon Bell, Jim Gemmell (2009)

Part III

Chapter 10. THE FUTURE

If the world follows my lead, Total Recall will be a very private matter. Encryption will be universal, e-memories will reside in Swiss data banks, and sharing will be careful and limited. I think the younger generation ought to eventually see their casual approach to privacy as a mistake and scale back their public disclosures. But maybe they won’t. Maybe my attitudes regarding privacy are headed toward extinction. There are those who say privacy is gone forever and good riddance to it.

If lifelogging becomes life-blogging, then the successors to Facebook and Twitter could have detailed records of every parameter of your life, with location, biometrics, sights, and sounds. Imagine for a moment that all memories are shared. One could then dream of data-mining all these memories, looking for collective good, much the way that my personal memories will be mined for my own good. There may even arise some kind of cyber communism that demands all of your information for the public good—for example your location, to help with city planning and emergency management. There could be an appeal to your own advantage: Just as Amazon.com and other Web sites track sales to predict items that you might want to buy, the collective cyber-mind might suggest many activities, places, and things that would be to your benefit or liking.

I don’t believe it. Embracing complete openness is like rescheduling Judgment Day for today. “What you did in secret will be shouted from the rooftops” might as well be the name of the next social networking Web site, echoing the words of Christ. But who can say for sure? I think that the future more than ten years from now is very hard, even impossible, to predict.

For one thing, it is simply hard to wrap one’s mind around the distant future. Passing on my e-memories to my grandchildren would be exciting enough—who can digest the idea of centuries ful of e-memories? You have thousands, if not millions, of ancestors from the past thousand years. What if you had all of their e-memories? Would one’s own family tree attract more attention than the History Channel? Would my family have a top-ten ancestor list, and a family highlight reel? Doubtless medical history would be pored over and different conditions identified in different branches. Genes would be compared to find ancestors similar to me, and lessons would be drawn from their lifestyle and health results. I can imagine drawing inspiration from an ancestor with similar interests to mine. I can also imagine angst over a tragic ancestor with some resemblance to me. I’d talk to his cyber twin: “But why did you want that?” “Did you realize … ?” If some great-great-grandsire had a gap in his e-memories, I might try to get access to the memories of his friends and relatives to try to piece together what he was keeping secrets about.

But with so many possible changes in society over a thousand years, my speculations may not be much better than a wild guess. And if culture is hard to predict very far ahead, I think technology is equally hard to predict in the long term. I don’t think anyone can predict technology more than a couple of decades ahead, because that implies knowledge of materials or phenomena that have yet to be discovered. Carver Mead, a Caltech computer scientist who coined the term Moore’s Law, posits an eleven-year rule: It takes eleven years to bring a high-tech product from the lab into existence. I feel comfortable predicting the progress of Total Recall about ten years out, based on technology that someone is already working on in some lab.

Next, I’m going to outline the technological context that will usher in Total Recall in the coming decade. There is a clear direction for computer hardware, sensors, and networking. Also, I see a trend in unified communications and storage. Total Recall will fuel demand for improved computer-user interfaces; it will also be key to enabling these interfaces to become more natural. Total Recall is far beyond speculation and, in many areas, is past the research phase and ready for straightforward product development. There is a clear path for hardware to build, software to write, and new services to develop. Product development for Total Recall has already begun in dozens of companies.

SHRINKING E-MEMORY MACHINES

The full spectrum of modern hardware is bringing us to the dawn of an era in which nearly every bit of information about your life can be captured and stored forever. This is not to diminish software. Nathan Myhrvold, the former chief technology officer at Microsoft who went on to found Intellectual Ventures, believes that only the software creator’s imagination limits what hardware can provide, and he is right. But if you can’t acquire it or store it, you won’t be computing it.

Throughout my adult life, hardware has been rapidly changing, and it continues to do so. Moore’s Law predicts that computers will be twice as powerful two years from now without changes in their size or price. Some time ago, I observed that there is another consequence of increased power and miniaturization: We get the same power at reduced size and cost. These cheaper, smaller versions of what we had before eventually get cheap enough and small enough to inspire the creation of an entirely new class of computers. A new class can be expected about every decade or so, with its own unique hardware-software environment, applications, user base, and vendors.

Eventually the power of your old PC finds its way into smaller devices, such as your digital camera, personal digital assistant, or cell phone. Looking ahead, it’s easy to see lots of multifunction pocket-size devices, with vast inexpensive storage for capturing everything you see and hear at higher and higher fidelity. The cell phones of the coming decade will have enough data storage and computing power to do some very powerful calculation and data mining on you and your environment. In fact, the smartphone I presently own is about a hundred times more powerful than the minicomputers I used to design—minicomputers that were shared among entire departments.

As noted earlier, the Total Recall revolution is being birthed on the strength of a few key devices: cell phones, digital cameras, and pocket-size GPS units. These devices have us snapping pictures, sharing media, and plugging into the networked world. Your PC is still extremely useful—it will not go away—but smaller, cheaper, more comfortable, and less obtrusive devices will provide the impetus for dramatic progress toward Total Recall.

In the next ten years our pocket-size devices will be accompanied by a host of even smaller cousins that will be able to compute, communicate, and, most significantly for our purposes, sense. There is no limit to the things it might be useful to sense in timely fashion. I have already discussed some of the wonderful prospects for health sensing, sensing your location, and automatically capturing the sounds and sights of your experiences.

Sensors on and in you will know not only about your body, but your environment: the location, temperature, humidity, sound levels, proximity to wireless devices, amount of light, and air quality.

Conference rooms and home offices are likely to end up with audio and video sensing, especially as teleconferencing continues to grow. This sensing enables the capture of individuals in organizational settings.

Every appliance will be sensing and logging. For instance, your dishwasher will measure the temperature of its hot water connection, while your washing machine will know the level of vibration during each load. These features will begin as diagnostic aids for repairing individual devices, but will find themselves being used in aggregate also. When you blow a fuse and wonder what caused the overload, you will check the appliance logs to find out everything turned on at the time. Eventually, your home’s e-memory will be part of your own Total Recall picture. Your time management software will be able to factor in how much laundry you do, and your health software will know you’ve been hauling around that heavy vacuum a lot.

You will literally sprinkle sensors in the dirt of your garden, and they will relay information through each other to a little powered hub that will forward information to your home network about soil conditions.

Your car will have its own lifebits, complete with location, health, and record of the environment it was in. It will know that on Tuesday it was driven up a 15 percent grade in the snow, carrying a load of 470 pounds and averaging 3,100 rpm. That, too, will start as a repair diagnostic and will eventually be used to supplement your own history with the story of all your driving.

NETWORKS OF UNIFIED COMMUNICATIONS

Network capacity and speed are ever growing, allowing us to move around bigger files and watch better quality video. I’m frustrated the television/telephone duopoly in the United States is so slow to get us high-speed fiber-optic networks to every home. Sometimes I wish that networking was considered part of the nation’s infrastructure, like highways, so that we could mandate fiber everywhere as in other countries. Still, the trend is in the right direction, and we already have a pretty good start.

Total Recall will come about within the context of networks within networks, interconnecting everything from in-body networks to home networks to global networks and finally to networks that include satellites and space vehicles. Dust-size sensors will automatically form wireless networks and connect to everything that can be sensed. In-body implants will communicate with each other to form a “body-area” network. The body network will connect to the car network while you’re driving. The garden network will connect to the home network. The car and home network will connect to the worldwide Internet.

This vast network of networks will host huge farms of servers with millions of processors and many petabytes of storage space. These farms will offer up computing and storage service to those who need it—and, amazingly, you will need it, even though your cell phone will boast more power and storage than your PC does today. From the microscopic to the heavens, all will be sensed, networked, and stored. This is not a forty-year-out wild guess. This is a decade-out sure bet. And I don’t lose many bets.

Microsoft has pretty good communications for its employees. If you phone my work number and leave a voice mail, I get it as an audio attachment in my e-mail. Actually, if you call when I am out of the office, I may well answer because the call is forwarded to my notebook PC wherever I am, to answer using my notebook’s microphone and speaker. From e-mail I can launch chat; from chat I can launch e-mail. All my chats are logged into a folder in my e-mail. All the RSS news feeds that I want to read appear in my e-mail client. I can set up my e-mail client to manage all my different e-mail accounts. I can also phone in to check my e-mail and hear it read to me.

This is called unified communications. Instead of telephone, chat, RSS, and my several e-mail accounts being completely independent, they are unified. I don’t have to go around checking in different places for messages, and it’s not a big hassle to switch to some other form of sending a message. Unfortunately, it seems that every new networking application out there wants to fragment my communications. My doctor makes me visit his Web site to check for messages from him. My bank sends me messages to tell me I have a message to read and reply to at their Web site. Facebook sends me an e-mail with an actual message from a friend, but makes me use their site to reply. I get a steady stream of messages from LinkedIn demanding I go over to their site to follow up. And there is a steady stream of new “must-have” communication channels. Frankly, no matter how wonderful they are, I am simply running out of time to keep up with all these virtual post office boxes.

If it is clear that communication ought to be unified rather than fragmented, it is equally clear that storage should be unified. I don’t want to have to jump around between online services or computers to get my stuff. I don’t want to be bothered with thinking about which hard drive on my computer contains what. And I sure don’t want to be launching different applications to get at different kinds of stuff and to face all kinds of hassles in grouping dissimilar items together. Unifying storage might mean actually bringing all the information together, or it may be sufficient to just have a catalog that virtually unifies everything; to look at it as one whole while retrieving particular items from many possible sources.

The consumer demand for unified communication and storage is clear. In the long run, those who don’t make it possible to unify their data or stream of communication will wither away. The number one requirement for unification is open systems that employ standards for information exchange. For example, I can get an invitation from Evite into my e-mail client’s calendar because it uses a standardized calendar event format.

Once you have open systems that use standard formats, the next step is the ability to translate. Remember the health data formats that use the code “DPT” to mean different things? Translation software is required to preserve the correct meaning between systems. As anyone who has translated between languages knows, a word-for-word translation is inadequate; it gives us translations that turn “The spirit is willing but the flesh is weak” to “The alcohol is good but the meat is bad.” Likewise, it can be difficult to translate between storage formats, and a lot of work is yet to go into this effort. The Semantic Web, which aims to standardize transmission and translation of information, is an important effort in this area.

There will also be a unification of networking in the sense that we will cease to have distinct networks for different types of data. Already we get telephone over our cable TV network and TV shows over our telephone’s DSL. Eventually, we will get a digital dial tone that carries anything and everything. In our homes, we will not have TV, telephone, and computer network wiring; we will have a digital home network for everything, and our home server will record TV shows and telephone calls, while it also serves up our e-memories. Our telephones and cell phones will just become small terminals into this universal digital network.

NATURAL USER INTERFACES

Your e-memories will be a vast ocean of data for you to navigate. Software will be your guide, summarizing, data mining, and anticipating what you may need. Still, there will be times that you will want to explore some particular area that has piqued your interest, or search after a very specific item. To take over the helm of this kind of navigation means handling many different controls, some of which may be complex. I recall bringing up one MyLifeBits interface and feeling as if I had sat down in the cockpit of a 747, with a host of knobs and switches to manipulate, and numerous gauges to examine. It was intimidating. But each gauge informs you and every knob empowers you, so getting rid of some of them in the name of simplicity could be dangerous.

I’ve mentioned already that software will be a personal assistant to take care of a lot of the Total Recall chores. It is also critical that software be able to communicate with you in a natural way, just as a personal assistant would. This is called a natural user interface (NUI). With a NUI, you can manipulate your vast e-memory collection in a complex way without the need for lengthy training .

The ideal natural user interface should be able to handle the way you already communicate. You should be able to type or speak in your normal language. “Show me pictures from my last trip to New York.” “When was my last doctor’s appointment?” “Sure,” “OK,” “Uhn-uh.” Gestures should be understood according to your culture, like a thumbs-up for approval or a throat-slashing motion for cutting something off.

You should be able to talk to your computer, but talking isn’t always best. Sometimes it’s quicker to point at something. It is often easier to type with corrections than to dictate (I couldn’t even compose this paragraph without a lot of changes in word choice—I could never have dictated it). There is an advantage to be gained by letting me interact with the computer in a natural way, whatever that may be, and not necessarily using speech.

A natural user interface would be very helpful for Total Recall. As it turns out, NUIs really need Total Recall even more than Total Recall needs them. A NUI would be severely hobbled if it had no memory or knowledge of you. For the interaction to be really natural, I must be able to use my own idioms and nicknames. I must be able to ask about “my sister” or the “Web page I saw last week.”

A really natural user interface, just like a real personal assistant, would ask questions to clarify: “Did you mean your uncle Bob, or Bob at the office?” It would know the context of your conversation to make sense of what you say, just as a real person would. By tapping all of your e-memories, it would have even more context, knowing your preferences and your usual schedule. A natural user interface would know what terms and acronyms you use regularly, and which require more explanation. “Know your audience” is the first rule of public speaking. “Know the user” is the key to a natural man-machine interface, and Total Recall will finally make it possible.

EXTREME LONG-TERM PRESERVATION

I was asked to give the keynote address at the British Library’s Digital Lives Conference while writing this book. There was a fascinating discussion about how the library of the future will preserve e-memories rather than papers. But looming behind all the technical details is the really big question: Who gets in the British Library’s digital lifeboat and who gets left behind? However incredible the growth of storage continues to be, the library’s storage will remain finite, and after all, they aren’t interested in keeping the e-memories of everyone. They will continue to save only those of the most eminent politicians, authors, philosophers, and so on. And it is not clear to what “depth” they will be kept—is it better to have ten full lives or twenty lives at half-resolution?

This raises a question closer to home: Will my progeny one thousand years hence really be able to have a copy of all their ancestors’ lives? As I pointed out, each person could have millions of ancestors in that time span, so each individual’s having a full family tree of e-memories is out of the question. The cost of storage would have to be shared among all members of the family. We might even think of the cost being shared among the entire human family; each generation could share the cost of trying to preserve all previous generations.

The capacity of hard drives and other storage devices is growing. So, too, are the number of them being sold. In 1995, 89 million hard drives were sold. In 2008 more than 480 million were sold. Still, we cannot presume that the amount of storage each person can afford will endlessly grow, even at a modest rate. If population growth were fast enough, one could imagine each successive generation being able to carry forward the past. However, we may see negative population growth, as in some Western countries, and additional population growth in conditions of poverty will not support the retention of e-memories.

How to keep our ancestral memories after the end of exponential storage growth is an open problem. It may even be impossible. So, though it goes against my grain to say so, it may turn out that most lives need to have their storage cost reduced over time. Video, which takes the most space by far, stands to be trimmed down the most. This could mean deleting repetitious or boring parts, or it could mean reducing the resolution, for example converting high-definition video to YouTube quality. However, I am in realms beyond my ten-year time frame. Thousand-year preservation is a matter shrouded in uncertainty.

THE WISDOM OF SPORTSCASTERS

The San Jose Sharks hockey team has just scored to tie the game with less than a minute remaining.

“And what a great pass by Thornton!” proclaims play-by-play announcer Randy Hahn. “Looks like we are going to overtime, folks!”

In the postgame show, Randy notes that Joe Thornton has assisted in more goals than any other player in the National Hockey League, and that this is the ninth time this season that he has helped tie a game up that seemed lost. He narrates over a video clip of tonight’s goal, and also another one from a similar game a few weeks past. He can recite the team’s record with and without Thornton. He has a “telestrator” that electronically diagrams the position of Thornton each time he made one of his legendary passes.

Sportscasters like Randy Hahn give us a real foretaste of Total Recall, with endless statistics at their fingertips, and the ability to replay game clips or interviews. Sportscasters for auto racing possess an added insight into a life filled with sensors, which record such values for each car as track position, rpm, and speed, while logging track conditions such as temperature, humidity, air pressure, rainfall, and wind speed. They use their Total Recall to entertain and inform us. Their hard data confirms or debunks our sporting theories. With Total Recall, they develop deep insight into their sport.

Diarists also have a foretaste of Total Recall. Whether done for posterity, for better memory, or simply for catharsis, journaling has many practitioners. Mark Stewart, a software engineer from Great Britain, was inspired by reading about MyLifeBits to create what he calls MyLifeDisk. It is a hyperlinked, two-volume DVD chronicle of his life, including words, spreadsheets, photos, videos, and songs. You can explore his family tree, where he has lived, his memorabilia, his education, his career, and a complete accounting of his girlfriends. Mark’s life-disk really illustrates where life stories are headed, and is so compelling that he was invited to present it to the British Library at the Digital Lives conference that I keynoted. A digital life is clearly a step forward in passing on one’s story to posterity.

It isn’t just about who was president or what wars were fought or even the troubles of your neighbors. It is about the substance of your autobiographical memories, from your environment to your myriad relationships. It is about your memories and how you remembered them. E-memories reveal the meaning of your life.

Of course, I’ve had my own foretaste of Total Recall with MyLifeBits. From the beginning, MyLifeBits was conceived as a project to understand the feasibility, cost, and value of storing everything in your life. It has largely served that purpose. Like Cathal Gurrin, who wouldn’t give up his SenseCam, I’m not ready to give up any of my lifelogging. I know just how much it is worth. I’m ready now to put my money where my mouth is and invest in start-ups that will take advantage of the e-memory revolution.

Total Recall will improve our lives and afterlives in many ways. It will shake our societies and change our cultures. We look back at the ages before the advent of writing as “prehistory.” The next generation will look back on our era as pre-Total Recall.