February 3, 2003
By Stephen Warley
I think most communication professionals love what they do because they are storytellers and can share those stories with others.
For the most part, storytelling is still a linear experience. Television has brought video into our homes, so millions more are witness to history. It has stirred emotions in ways that print never could. And some day in the not too distant future, we may all be able to jump into a story or experience a historical event for ourselves. New technologies are being developed to create a new kind of story telling experience called “augmented reality”.
The photo shows an "AR-Cam" (Augmented Reality Camera) -- a live, interactive video camera that is showing a simple Augmented Reality application which you can control over the Internet. -- Courtesy, The Computer Graphics and User Interfaces Labs, Columbia University, NY
John Pavlik, Director of the Journalism Resources Institute at Rutgers and Senior Research Associate at the Institute for Learning Technologies at Columbia, has been instrumental in the development of this exciting new area of media. We asked him to explain.
Q: What is augmented reality?
JP: Augmented reality is a sort of cousin of virtual reality, except instead of replacing a person’s experience with reality using a computer generated reality, the user continues to experience reality, but we overlay computer mediated information on top of that reality.
We are doing that primarily through a wearable computer, a see-through, head-worn display. We use a global positioning system or WiFi, high speed wireless technologies, to locate the person and we have a three-dimensional map of where they are. We can determine how they are oriented in that environment – kind of where they are looking – and with all of that we can imbed multimedia presentations into the real world.
They can look at a building and we can overlay some information onto what they are seeing. It could be a piece of text. It may say something about that building, from the past or something current. We can overlay graphics. We can show video about something that might have happened there. We have sound that would allow them to hear something that had happened before.
Q: How was this technology developed?
JP: It has been developed by computer scientists mostly working on a variety of applications. We are collaborating with the computer scientists at Columbia, Steve Finer in particular, to develop this for story telling in journalism, communications and even education.
Q: What is the potential for using this technology in journalism and story telling?
JP: I think the potential is very significant, not that it will replace traditional ways of telling stories, but I think that we live in a world where a couple of things are happening.
One, people are increasingly mobile and they still want to be able to get their news and information. They are also connected through wireless communications technologies, so there is a marriage of mobility and connectivity. It makes news and educational information available to people and does it in ways that are consistent with that kind of evolving lifestyle. Secondly, younger people are increasingly interested in interactive types of technologies. That’s not to say older people have no interest, it’s just that younger people have grown up with it, so they are very comfortable with it. It’s as natural to them as reading a newspaper would be for a person from a previous generation. They still want to have news and information, but I think they like to have it in a different way than previous generations have experienced it.
Q: How would you produce a situated documentary if you had access to this technology? Can you give a few examples of how this may be used as it gains acceptance?
JP: I think news organizations cover the communities that people live in as their bread and butter. They have a treasure trove of news and information about those communities that goes back many years, sometimes more than a century. A lot of those materials sit in archives which are increasingly digital.
This technology gives news organizations a way to let citizens have access to that treasure trove of information that they have created over many years as they are out and about in those communities.
When a person is visiting a place, they will be able to easily access information that is relevant to where they are and it has been produced by a quality, credible, trust worthy source, the local news organization.
Q: You’ve created some projects at Columbia University, using this technology. Can you describe the process you went through?
JP: What I am presenting is a long-term vision, but in the meantime we take it one little step at a time. So far we’ve created a series of prototypes which have all been based at Columbia where our project began. We did stories about things that happened at Columbia in the past.
For example, our very first project was on an event that happened in 1968 at Columbia where some students and local community members protested university plans to build a gymnasium in a park next to the campus called Morningside Park. The university was breaking ground there for a gymnasium and this was at the height of the civil rights movement, the height of the Vietnam War era, and some of the students and local community members didn’t like the fact that the gym was being built. It was going to raise a lot of issues with regards to race. The Morningside campus is right next to Harlem. There were a lot of race-based issues, a lot of tension. Some of the students took over part of the campus. It was a big event and led to a lot of huge changes at Columbia, like the founding of the Student Senate.
We decided to tell the story in a way it hadn’t been told before. It had been done as a documentary in traditional media. It had been reported in the newspaper. We decided to tell it using this mobile augmented reality to create our first situated documentary.
My students produced the content. They developed a story about what happened. If you wear the head-worn display my colleagues in computer science developed, you can walk through the campus and you are immersed in a three-dimensional space. You almost go through a virtual time machine back to 1968. You see what happened back then and it plays out around you. We created three-dimensional models of things that existed back then. We used video that was shot in film back then. We also had current reporting. My students interviewed some of the protestors who were still alive and living in the New York area. They put all this together in an interactive narrative that allows you to go back virtually to 1968.
Q: What was the reaction of the people who experienced some of those situated documentaries?
JP: I think outside of the students only about 100 people directly experienced wearing the system, since we only have one prototype of it. The overwhelming experience was that it was extremely powerful emotionally, very engaging and has huge potential. I think that has been the general response of people who have tried it, including journalists, media professionals and people from business.
Of course there are still a lot of little technical issues that have to be resolved. The resolution of the display and the resulting images aren’t really very good, so it is inadequate in that regard. There are a lot of things about the interface that still need to be refined. All of this is due to the limits of the technology.
Q: How far way do you think this technology is from wider use?
JP: It’s being used in a number of places. The University of Southern California is doing some related research. We are doing a prototype now in Italy. It will be telling the story of the ancient Imperial Forums of Rome. We are doing this in collaboration with the Museum of Imperial Forums, under the umbrella of the Mediterranean Arts Bureau, an organization I co-founded.
There is also a group doing some related work in Greece on some ancient archeological sites. They have a Web site you can check out called Archeoguide.com.
When we started this back in ’96 and ’97, virtually no one was doing anything related to this, but we were already working on technologies like high speed wireless communications, where we were using things like wearable computers. Today, things like high-speed wireless, wearable computers, Palm, Trio, etc. are widely used. At this point the whole package is probably still many years away, but in 5 years we’ve gone from every piece of it being totally arcane to several pieces of it being pretty commonplace. I think within 5 to 10 years we’ll see several more pieces of the technology that will be fairly widely used in a series of public venues.
* * *
This article is reprinted from TVSpy’s Next Generation TV, a column devoted to digital media. To read more columns, click here: www.tvspy.com.
* * *
Stephen Warley has made a career out of assessing the future direction of television. From producing for CBS News and CNBC to working as a project manager for interactive media agencies like ThirdAge and Osprey Communications, he has gained an insider’s view as to where tomorrow’s content and business opportunities lie in the video media industries. He can be contacted at swarley@tvspy.com
Recent Comments