Total Recall: How the E-Memory Revolution Will Change Everything - C. Gordon Bell, Jim Gemmell (2009)

Part II

Chapter 7. EVERYDAY LIFE-AND AFTERLIFE

I have a hand-loomed blanket from the late 1800s, handed down from my grandmother Bell. Grandma had seven boys and three girls who, in turn, produced twenty-five grandchildren, including me. The only remaining mementos of her are half a dozen photos, a Bible, a dresser, and the blanket. As the older generation passed away, the Bible was passed on to the remaining eldest son, and eventually on to the grandsons. At each transition, there was inevitably a heated dispute over who ought to inherit the Bible, although none of the clan was especially religious. When it was my turn for the Bible, I declined, but I happily accepted the virtually unknown blanket, which was surreptitiously given to me by a favorite aunt who had been caring for Grandma.

This beautiful blanket, now used as a wall hanging, is one of the few physical artifacts that I care for, but I will soon pass it on to my son. For me, a high-resolution photo will suffice. If I’m feeling sentimental, I’ll print a life-sized copy. How can I be satisfied with a digital photo? Wouldn’t I take far more pleasure in the actual, tangible blanket? Nothing is quite like the blanket itself, but I have lots of mementos and memories, and taken as a whole, I’ve discovered that I derive more pleasure from them in digital form. While I am enjoying my e-memories, most people’s physical mementos gather dust in an attic—if they even have them.

A typical story of the distribution of physical items after a dear one’s passing goes like this: “I spent a day being careful to preserve and bundle my mother’s artifacts (mostly correspondence and scrapbooks) for my sister. After a day my filter became very narrow and all the stuff just went to the dump. And my sister never came to claim what I had saved.”

Physical family heirlooms and mementos pass down through random branches of the family, eventually arriving where they are unrecognized, unappreciated, and discarded. Other relatives, who might have been keenly interested in the mementos, are unlikely to even know of their existence. At best, only one person has custody of the valued heirloom, which requires physical space and careful preservation if it is to be passed on to the next generation.

How refreshing to contrast this to a digital legacy! All of the heirs can have a copy, it can be quite expansive, and in the event you don’t especially want to know anything about the departed, there is no cost to your space or attention.

Of course, a family heirloom may be some rare antique or a diamond bracelet. Objects with monetary value will be squabbled over until the end of time. But I am speaking here of emotional value. Not long ago, Jim Gemmell overheard a woman speaking to her friend:

I’m going to give you my cell phone number … you can’t leave messages on my home phone because I’ve saved too many messages from my grandson.

This lady wasn’t just keeping track of when her grandson had called or maintaining an accurate record of the words he said. For her, hearing his little voice was precious.

This is a very different aspect of Total Recall from what we have discussed in the last three chapters. This kind of memory gets to the heart of your emotional life, to the fabric and climax of the best stories you can tell about yourself. Sure, we can have perfect recall with regard to our work, our health, our pursuit of knowledge; we will improve our minds, bodies, and ventures. But this also touches our hearts, our emotional lives, with all that impractical stuff that makes up the rest of our days and nights. It is an awesome prospect that even these kinds of memories can become e-memories, totally searchable, even ready for scientific analysis. What would Proust have made of it?

These kinds of memories so often exist between you and someone you are close to. Between you and your grandmother, son, friend. They are often family memories. As such, the bio-memories overlap. Each individual’s enhances the others’. In the world of Total Recall, your e-memories will supercharge this enhancement. Your personal relationships inside and around your family will be transformed.

If we can have a complete record of the things about people that especially provoke meaning for us, what will we do with this complete record when they are gone? We will maintain the e-memory of that person as a treasured heirloom. And, someday, we will ask it questions. The e-memory will answer. You will have virtual immortality.

RECORDING AND STORYTELLING

Lifelogging implies collecting countless digital artifacts—and increasing the variety of artifacts is all to the good. We want all the strands of the fabric of our life. Vacation videos. Snowy dioramas from that skiing trip. Our first blanket (or our grandmother’s). Songs we wrote in high school. Birthday cards. Tickets to concerts. What your father said in that crucial moment of the third quarter in that crucial game. Maps of our travels. Recipes. Laundry lists. Lists of guests invited to a party. Toasts, eulogies, and your baby’s first attempts at speech. The fabric of your personal, intimate life.

These e-memories are so enjoyable when they come up on screensavers that I declared screensavers the “killer application” for e-memories (ahead of search, which so many people presume is the chief end of Total Recall). The MyLifeBits screensaver shows photos and also selects random ten-second video clips out of longer videos. Before Total Recall, years would have passed between those occasions when I bothered pulling out old videotapes to play selections from them; now I can enjoy clips from them daily.

Screensavers are pleasurable on my desktop PC and notebook, but they really shine on my big plasma screen. I am tracking the progress of organic light-emitting polymer technology and looking forward to the day that an entire wall can be a screen. I want to sit at my dining room table and have my wall transport me to some other place or cherished past event—perhaps I can feel like I am reliving a train ride through the Rockies or sailing on the San Francisco Bay. I also love little “picture frame” screens that can sit on an end table and, instead of being stuck on one photo as real picture frames are, root through my e-memories to show all kinds of things.

Having different types of artifacts contributes to the richness of your e-memories. Think of the lady I mentioned above with her phone messages from her grandson. Jim Gemmell has some treasured audio recordings of his grandfather’s Scottish brogue. Video is essential for “preserving people,” a view that I have only recently come to appreciate. Seeing someone move, speak, and make facial expressions provides a distinctive view of a personality that written records and photos can rarely touch. Now add location, temperature, heart rate, and other new values we will be sensing, and really fascinating perspectives are formed.

With location tracking, we can plot you in space as well as time—and that’s exactly what we did with the MyLifeBits Trip Replay program. Gemmell came back to work one Monday, after spending the weekend at a sports tournament with his son, traveling with a GPS and taking pictures with his digital camera. When he joined us in the office, we were able see on a map everywhere he had been, and where his photos had been taken. Trip Replay even animated his travels, showing him moving around on a map, with photos flipping up as they occurred.

Gemmell was excited. “Look at this!” he exclaimed. “And think about it: I have the information about each game my son played in my calendar. All the components are here to tell the story of my weekend without me doing any work at all!” We started brainstorming about how this could be automatically wrapped up in an attractive form and sent to his parents to fill them in on their grandson’s weekend. We later hired an intern to prototype a system that let you just select a time range, exclude a few duds or embarrassing moments, and then click “Blog it.”

Consequently, I’m bullish on automatic travelogues. I envision a service that takes your itinerary and produces a trip log with photos of where you stayed, sights that you passed by, meals eaten, et cetera. With the use of GPS, the entire trip can be created in great detail that might even exceed the actual trip experience. As I write, Telestial is preparing to roll out a service that will track where your cell phone is, and post stock photos of the locations you visit to a Web site you designate. You can add captions by sending SMS text messages from your phone. This is just an early (and very simple) entry in a coming wave of automatic travelogue offerings.

The landscape of our e-memories becomes lush as we share with one another. The value of pooling media is already evident in photo- and video-sharing Web sites, like Flickr and YouTube. Facebook shows us how much we enjoy having others’ comments on our photos. Think of an extended family gathering for Grandma’s birthday with a number of people taking photos and maybe shooting a little video. Once the media is shared, if there is even one keener in the family who adds comments, flags key moments, or groups the media in some way, then we all benefit.

Imagine the parents of players on a basketball team. Suppose the video of the game is posted to a Web site, keyed with the score clock, allowing everyone to quickly jump to a certain point in the game and skip breaks or dull parts. After the game, each family goes online and flags a few favorite plays made by their child. When all their annotations are added together, an automatic highlight reel is easily produced. Years later, a player can go back to the copy of the game in his e-memory and relive his big moments. I believe that many sports venues in the future will install cameras and post video automatically. They’ll make some extra money, and as we share our thoughts on the sport with one another, some splendid e-memories will be constructed.

Furthermore, sharing is necessary to fill in a key missing part of any e-memory. If I wear a camera capturing my own point of view, there is always one person left out of the footage—me! You and I must share our point-of-view footage with each other in order to appear in our own lifelogs.

One thing we will certainly share with each other is stories. Humans are storytellers, and no matter how much I value a recording, I’ll always love to hear someone else tell the story of the event.

In 1989, when my ninety-year-old mother visited me, I asked her to write some stories to pass on to her four grandchildren. In particular, I asked her to tell stories of the changes she had witnessed since her birth in 1899. She wrote about social life, clubs, church, and school. She told stories of Christmas, Thanksgiving, farming, gardening, and food from butchering to canning. Just recently, my sister thanked me for initiating these stories, as she was reading them to her own grandchildren.

Jim Gemmell was thinking about stories in 2005 when he asked me to spend a number of lunch breaks with the half-dozen others in the lab answering questions about my experience at Digital Equipment Corporation. We ended up with an hour of video stories and opinions never expressed elsewhere. I’m sure the viewer would get a different sense of me from seeing these videos than he would by reading what I have written about life at that time.

I believe oral histories are irreplaceable. While I love all the artifacts at the Computer History Museum, I’ve come to see the stories they collect in oral histories and public talks as the most important thing the museum does. Of course, memories fade over time, so for accuracy we want more than just oral histories. Still, inaccuracies and all, I want to capture people recounting stories.

For Total Recall to fulfill its full potential, people must be able to tell stories anytime, anyplace, any way they feel like it. This could be by sitting in front of video camera each night for a dear-diary entry, talking about some episode while driving, or typing in some thoughts about a recorded event.

AUTOMATIC SUMMARIZATION

With today’s technology, your e-memories would be a mixed blessing for your heirs. They would have the benefit of more knowledge about you, but it would come to them as an enormous, daunting mess. Your heirs may enjoy looking at random photos, or searching for e-mails containing the names of presidents in order to read some of your political perspectives, but they will likely miss the most important and interesting bits, and may be too intimidated to spend much time with your e-memories.

I felt this way about my own early scanned collection, and it was my frustrated eruption of “It’s just bits!” that galvanized Jim Gemmell into action and got us started on MyLifeBits. Thankfully, our prototyping work has assured me that things will get much better.

We will see the evolution of software that will reduce the chores involved in making one’s life bits worthwhile to others. It will help to develop tools to make storytelling and human arrangement easier. But fully automatic approaches are even more important. Some of this we get quite readily just by storing more information together. For example, your e-mails, calendar entries, Web-page visits, and digital photos all have time stamps that readily lend themselves to time line displays.

Time lines are a really compelling way to visualize your life, and software can help automatically produce digestible time lines. Our colleague Eric Horvitz and his research team have done some very promising work in predicting which events people will consider to be significant “memory landmarks.” This allows the best material to be put on a time line, and the less interesting material hidden away until it is asked for. Eric demonstrated the software to a reporter, starting with pictures of his wife and son:

“What’s cool—I love this feature—I can say, ‘Go to July Fourth,’ and it’s making guesses about the things I am likely to remember, to use memory landmarks, and it jumped right to this place,” he said. The screen showed several images—a small-town parade, and his wife and son among figures at a cookout, from July 4, 2005. Responding to his request, “the computer brought up its best guess.”

“It comes to understand your mind, how you organize your memories, by what you choose. It learns to become like you, to help you be a better you.”

Remember the Dublin City University project that finds novel SenseCam images out of thousands to highlight the interesting ones and spare you from the mundane? Automatic summarization has become an entire field of research. There are even sub-specialties, for example, summarizing just video. While we would like it best if a human creates a photo album, it is possible for the computer to do a pretty good job of automatic photo-album composition, choosing only interesting, high-quality photos. The nice thing about machine-composed albums or time lines is that, unlike physical albums, they keep the “outtakes”; with just a few mouse clicks you can retrieve the other shots that weren’t included in the album, allowing you to be as absorbed as you like with one particular event or topic.

With automatic summarization, posterity will be able to browse your e-memories starting from a manageable “birds-eye” view of a life, rather than just confronting an intimidating jungle of material.

IMMORTALIZING JIM GRAY

When Jim Gray went missing in 2007, I was not alone in wishing to immortalize him in the most rich and resilient way possible.

I am certain that Jim Gray’s name will be immortal at least in some ways. His name cannot be neglected in any history of computing as a winner of the Turing Award (often called the Nobel Prize of computer science). He is best known for his role in developing transaction processing, which we all use every time we withdraw cash from banking ATMs. In an effort to have his name be even better remembered, I helped establish the Jim Gray Endowed Chair in Computer Systems at UC Berkeley. I’d also like to see a building named after him. Jim’s astronomer friends have already identified an asteroid that will bear his name.

For a computer scientist like Jim, the most common way to gain an immortal name is to pass on ideas that are used by future generations. If you are lucky, some concept will be named after you. Moore’s Law is undoubtedly the best-known, predicting that transistor density in computer chips would double every two years, and explaining the meteoric rise of computing power. I hope that someday we will refer to the Gray Data Cube, the Gray Transaction Processing Benchmarks, the Gray Five-Minute Rule, and the Gray Paradigm of Scientific Discovery.

I’ll certainly be lobbying for such an immortal name for Jim. However, while it would be fitting for him to join the ranks of other such esteemed names, having an immortal name is a pretty superficial immortality. We may say Pythagoras’ name until the end of time when discussing geometry, but we will never know much about him, or even how he did his work. I want something better for Jim.

Someone’s work can be immortalized, as in the paintings of the great masters, buildings by a brilliant architect, or some notable equation. Going deeper, the way they worked may be immortalized: their techniques, their approaches, their professional relationships, and the stories of them at work. For instance, we know a fair bit about the work of Isaac Newton, including the story of him in his early twenties, going to the countryside to avoid an outbreak of the plague and, like any typical young man with too much time on his hands, whiling away his time—inventing calculus and discovering the law of gravitation.

Jim Gray’s Web site reveals a lot about him as a computer scientist. The extensive publications on the site reveal his drive for understanding through experiments and measurements. However, the site is missing those additional, critical stories that help us understand the man at work. Other sites, like the National Library of Medicine’s “leaders in biomedical research and public health Profiles in Science” Web site (http://profiles.nlm.nih.gov), provide a slightly more personal look. At this site, luminaries such as Francis Crick (who discovered the structure of DNA) have archives that include articles, bibliographies, books, brochures, certificates, drawings, exams, interviews, lectures, letters, notebooks, photos, and schedules. Still, we end up with only a fragmentary view of their lives.

Jim Gray’s family, friends, and colleagues are the sole repository of what he was really like: how Jim would slap your back with bubbling enthusiasm when he congratulated you, or how, when he thought your ideas were nuts, he would politely pronounce himself “puzzled” and furrow his brow. His Web site doesn’t tell you of his countless lunches on the least expensive sandwiches in San Francisco, despite his wealth. Then there are all the stories from his family, his sailing buddies, friends from college days, and others.

The story of Jim Gray is spread out on the computers he used, in his personal effects, and in the e-memories and bio-memories of those who knew him. Doing justice to his story means bringing them all together and presenting them in a comprehensible way.

MY MEMORIES OF JIM

After losing Jim, I naturally reminisced about my own relationship with him. A quick search in MyLifeBits turned up the following items that reference Jim in some fashion: 13,000 e-mails, 1,600 Web pages, 100 presentations, 289 photos, 600 documents, several videos, and a phone call. I don’t recall what first prompted Jim and me to get together, but MyLifeBits has a copy of the penciled calendar entry from 1994 for our first meeting. Other calendar entries include Jim taking me out on his sailboat—the same one he would later vanish in.

In 1994, Jim had just finished four years heading DEC’s San Francisco Lab on Market Street and had turned consultant. Since 1989, I had been a Silicon Valley angel investor and a consultant to Microsoft Research and others (since I didn’t spend much time consulting, some friends kidded me that consultant was a code word for “unemployed”). Our first meeting at my Los Altos home revealed our shared views on the importance of industry standards and an approach to increasing computing power via many cheap PCs working in concert together. We found that we both preferred small teams and esteemed building influential prototypes. It was the beginning of a stimulating collaboration and a heartfelt friendship.

After being an independent consultant for a while, Jim felt that he needed the confines of an organization, and he convinced me that I needed more structure too. He had been talking to Microsoft. We believed Microsoft was the place to be because of how we felt about standards and leverage, and moreover the respect and enjoyment of the community we would be part of. I jumped the gun and e-mailed the Redmond folks to hurry up and start a Microsoft Lab in San Francisco for Jim:

Sun Jan 08 15: 41: 55 1995

To: Rick Rashid ; Nathan Myhrvold ; … Dave Cutler

From: Gordon Bell

Subject: Approaches to Servers and Scalability … and an AD Lab here!

>Folks, Here’s how Jim Gray and I see the next decade or two: A Scalable Network and Platforms (SNAP) architecture is predicated on one set of standards: an ubiquitous ATM network and PC-sized platforms. SNAP allows upsizing i.e. building world-scale computers from a single platform architecture in a scalable fashion. SNAP will encourage further industry de-stratification. It eliminates the traditional computer price class distinctions (mainframes, minis, PCs) and goes a long way to eliminate the stratified business models of traditional computer suppliers. SNAP will cause a computer industry upheaval greater than the early 1990s client-server downsizing wave. That wave created a large UNIX market displacing IBM mainframes and proprietary minis. But the UNIX market is fragmented and small when compared to Compaq and NT. UNIX would have to consolidate around one or two dialects in order to get the volumes required to compete with NT. This seems improbable, so Microsoft’s NT is likely to become the dominant server standard for all hardware platforms, just as Windows garnered the desktop or client side.

Jim sent his own e-mail, pointing out that he had not “put me up” to writing mine, and enumerating the difficulties of operating a remote lab. However, he strongly validated the outlined vision. Microsoft liked the idea, and Jim’s Bay Area Research Center (BARC) opened in the summer of 1995 in San Francisco. I was honored and delighted to join the lab in August of that year. We hired Jim Gemmell to work with me that fall.

Though the BARC lab peaked at only around ten members, it had an impact beyond its numbers. Aerial imagery of the world was brought to the Internet by the BARC Terra Server, which led to the Microsoft Live Maps site and predated Google maps by five years. Later, Jim would turn the view up to the heavens, and work on the Sky Server project. His broad agenda got him involved in such far-flung projects as the “land speed record” for network transmission and fail-safe databases. Meanwhile, Gemmell and I were working on telepresence: putting a conference on the Web, playing with altering someone’s gaze direction in video, and shipping new network protocols in Microsoft operating systems. Later, of course, we got into MyLifeBits.

A memorable event was in May 1997, when Jim gave an on-stage demo with Bill Gates, using more than a hundred PCs to achieve one billion transactions per day. I also recall Jim’s glee on April Fool’s Day 2005, when he had just finished measuring a half-billion transactions per day using his relatively old laptop. He wrote a report observing that a common PC could execute eighty times “more than one of the largest U.S. bank ’s 1970s traffic—it approximates the total U.S. 1970s financial transaction volume. Very modest modern computers can easily solve yesterday’s problems.” The data and report illustrate Gray’s fondness for understanding through constant building and experimentation.

Through various paths, Jim infected me with the importance of data. It’s “all about the data,” he would say. In one of our more playful times, while discussing how to get the concern for data into the national computing resource allocation agenda, we bumped into John Markoff, a friend and columnist at The New York Times who also had an office in our building. We made the case that the national computing agenda was missing the point by just thinking about computation speed. John took our picture in the lab on Friday and wrote an article that appeared in the Times on Sunday. Our compute-centric friends in Washington were not especially happy, but they slowly came around to our view, and after many years the situation is gradually changing from models and simulation to real world, data-based, data-intensive science.

DIGITAL IMMORTALITY

I’ve just told some stories about Jim Gray, and shared an e-mail artifact. But if my publisher would let me, I’d probably include a photo; and in the e-book, some audio and video too. If Jim had lifelogged, there would be a wealth of e-memories to peruse.

A complete lifelog is an awesome prospect. Passing on your e-memories might be seen as a sort of one-way immortality, allowing you to communicate to the future. But Jim Gray and I used to speculate about the possibility of something really worth calling digital immortality, where a digital version of yourself lives on and interacts with posterity. What if, a hundred years from now, your heirs could ask you questions and you could answer?

You might be familiar with the word avatar, meaning an image or computer-generated character that represents someone in cyberspace. An avatar might be a picture of you that appears while you instant message with someone. Or, it may be a three-dimensional character in a virtual-reality program like Second Life. Someday, those who lifelog will be able to create avatars that do an amazing job of impersonating them.

Imagine if you could have a conversation with an avatar of your great-grandfather, seeing his face, hearing his voice, and having your questions answered in your great-grandfather’s unique turn of phrase. It seems clear that at least some aspects of your person can be expressed as a program that actually interacts with future generations. Lifelogging is the key.

Your lifelog will have all the details of how you sound, the phrases you employ, questions you have answered, and facts about your life. It will contain recordings of you under stress and relaxed, pleased and annoyed, in triumph and in defeat. It will know your favorite quips and mottos. It will have pictures of you that can be overlaid on a three-dimensional model. If someone should want to impersonate you, your lifelog will be the ultimate gold mine.

There has already been some progress making avatars that answer as the individual would have. Carnegie Mellon University researchers have created a program that lets you interview Albert Einstein. I’ve invested in a company called MyCyberTwin, whose software trains an instant messenger (IM) avatar how to answer as you would. You just give it content from your own IM conversations, e-mail, blog, et cetera, and it learns how to imitate you—it becomes your “cyber twin.”

MyCyberTwin analyzed transcripts of the television show The Simpsons to create a cyber twin for Bart Simpson, who had the following chat with me:

You: What’s up?

Bart: Well, I was interested in getting the fire hydrants from the school to power my downhill racer—wanna help me?

You: No. How is Lisa?

Bart: She can be so annoying, but she is still pretty cool. Do you like my Mom’s hair?

You: Blue hair is pretty cool too. What do you think of your dad?

Bart: My dad, Homer … he got a chess set once, then packed it away. When I asked him why, he said he is saving it for a brainy day.

Have you been married to my aunt Selma?

You: No. How old are you?

Bart: I’m ten, man. Best age in the world: You can do what you want without getting in too much trouble.

You: What do you want to do when you grow up?

Bart: It’s always important to consider the consequences of anything you do.

Are you a friend of Principal Skinner, by any chance?

You: Do you have a pet?

Bart: Who needs pets when I have Homer? Actually, I do have Santa’s Little Helper too. He’s a dog. What pets do you have?

MyCyberTwin specializes in creating chat avatars to perform customer service, for example, answering questions for bank customers. Their avatars have gotten better scores than humans in accuracy, sales performance, and customer satisfaction. Now the MyCyberTwin folks are intrigued by the idea of taking my own e-memories as input—there is enough of what I have said in e-mail, letters, chat, papers, and so forth, that one ought to be able to construct a pretty realistic Gordon Bell cyber twin.

Alan Turing, a founding father of computer science, proposed the Turing test for determining a machine’s capability to demonstrate intelligence: A human judge has a conversation with a human and a machine, each of which tries to appear human. If the judge can’t tell which one is human, then the machine has passed the test. Turing proposed typewritten exchanges; we can update that to computer chat without changing the essence of the test. Thus, we can have a cyber-twin test: You chat with someone and his cyber twin. If you can’t tell the difference, then the machine passes the test. Note that the cyber twin could have a much better memory than the human did; it should be taught to forget in a similar way to the human for real simulation. But it should remember when we really want the answer! As I write, there is a fair bit of work remaining before any cyber twin could pass the test, but substantial progress seems likely.

I see four steps in the progression of digital immortality. First is digitizing the legacy media one has. Second is supplementing one’s e-memories with new digital sources. The third is two-way immortality—the ability to actually interact with an avatar that responds just like you would. The fourth is an avatar that learns and changes over time just as you would have.

Having an avatar that actually learns and grows over time is a much more speculative idea. Who’s to say that it is working correctly? If we could predict how someone would behave after death, then we could predict people during life—an idea that sounds far-fetched. Perhaps a more realistic goal would be to shoot not for growth—as in change—but just accumulated knowledge, so the machine would recall when it last communicated with you and what was said: “Hi Gordon. I talked to you yesterday. You told me about your vacation.”

While I believe the fourth step will remain science fiction, an avatar passing the cyber-twin test is not. A lot of sci-fi and artificial intelligence discussion is about true machine intelligence, where programs actually learn and grow. Personally, the more I learn about machine intelligence, the more I am impressed with the learning ability of the average two-year-old. True machine intelligence remains elusive. Computers can beat human chess masters now, but the computer’s greatest advantage is its ability to enumerate each and every possible move and outcome; most of us would call that tediously mindless, not intelligent. Similarly, in an attempt to answer factual questions like “What year was Abra ham Lincoln assassinated?” brute-force approaches that just scan many encyclopedias and newspaper sources looking for common words often do as well as or better than programs that attempt to parse and comprehend the same texts. I am confident about avatars passing the cyber-twin test because I see that lifelogs will contain enough information to support such brute-force approaches. No sci-fi, truly intelligent machines are required.

By carefully mining your lifelog, we should be able to ask your e-memories questions and hear your answers. We can change the game from a search to a discussion.

Humans have a natural propensity for recording life. Just look at all the people walking around with cameras and video cams. You’d be hard-pressed to find a home without photo albums, home movies, scrapbooks, and mementos. The one thing many people would be sure to rescue from the flames of a burning home would be their photo albums. We love to reminisce, and if you think of all the photos and home movies taken, it seems we enjoy enhanced reminiscence: not just remembering but also hearing and seeing recordings or artifacts from the past. A few of us go beyond just confining ourselves to recordings and objects, and actually edit movies, or create scrapbooks with captions and artistic layout. Some even take classes from companies like Creative Memories to learn to do it better. The rest of us envy them the time and talent to produce such compelling stories.

Your e-memories will prepare you for the digital afterlife. Already, for a fee, Web sites like www.legacy.com and www.forevernetwork.com offer to store letters, essays, photos, videos, and stories to pass on to future generations. Famento.com helps people share family stories, building e-memorials to loved ones. These sites are the digital equivalents of cemeteries and libraries.

Imagine opening up an old tomb and finding it full of historic artifacts. That would be interesting for a while, but just think of how much more interesting the artifacts would be if they came with their own curator, ready to help guide you through and explain them. That’s what I expect Total Recall software to do for our e-memories, with automatic travelogues, automatic summarization, and cyber twins. Today, as I struggle with setting up Jim Gray’s digital legacy, I know we’ve got a long way to go. But I also know that Jim would have insisted on seeing this as a challenge—another chance to do and apply science by creating understanding and something of value.