Article written by Lisa M. Krieger

Digital 3-D renderings of Earth are the backdrop for everything from computer games like “Grand Theft Auto” and Hollywood films like “Enemy of the State” to mortal combat simulation in Iraq.

This new technology permits astonishingly precise measurements of buildings, roads, waterways, coastlines and even vegetation, right down to individual plants. Its attention to detail is so accurate that thousands of tiny windows on skyscrapers seem to show “reflections” and the bricks holding the Statute of Liberty show detailed textures.

Avideh Zakhor, a UC Berkeley professor of electrical engineering, is the brains behind these realistic visualizations.

“I call it ‘reality capture,’ ” she said. “You’re trying to capture what’s out there in the real world…and create a representation of that so others can feel they’re in that same space.”

Dr. Avideh Zakhor has been at the forefront of 3-D mapping interior spaces, which, until recently, as proven to be very difficult.
Dr. Avideh Zakhor has been at the forefront of 3-D mapping interior spaces, which, until recently, has proven to be very difficult.

To test her technology, Zakhor had a novel idea. “Why not put all these sensors on top of a truck, drive around, acquire all the signals, images and videos and laser scans and make 3-D models of the real world of everything that is outside?” she wondered. So her team’s sensor-equipped car drove the streets of Berkeley, creating a 3-D model of the city. Her technology was licensed by Google in 2007 to help produce its 3-D Google Earth.

Now anyone can navigate through lifelike “virtual” settings – prowling Parisian parks, exploring ruins in Cambodian jungles or fighting zombies in alien landscapes.

But there’s a big problem in this approach to imaging: it only works outdoors.

A constellation of over 24 satellites that orbit the earth provide location information for the Global Positioning System (GPS)
A constellation of over 24 satellites that orbit the earth provide location information for the Global Positioning System (GPS) (NASA)

It uses GPS, which relies on a series of satellites to identify where it’s physically located – and satellite signals are carried through waves at a frequency that doesn’t pass through solid objects like walls. That’s why, when you use a GPS inside a building, the device can’t pinpoint your location accurately.

There are other ways to image interiors, like using laser devices that measure distances across walls and then assembles them. But those take time – 30 to 40 minutes for a small room, said Zakhor. “A whole building could take a week to scan.”

Zakhor had a different idea. She created a portable, laser backpack for 3-D mapping that creates fast, automatic and realistic inside images. It collects thousands of data points, then stitches them together with the system’s custom-built software, using fancy mathematical algorithms, into a 3-D model.

Inside the 30-pound backpack are high-tech laser cameras and scanners, which collect all of the information.

“It’s a way of documenting a building in a very fast way,” she said.

The 3-D mapping backpack is lightweight and customizable. Depending on the sensors you attatch, Y-you can map a map a building, measure heat loss from a room, or even monitor oxygen levels
The 3-D mapping backpack is lightweight and customizable. Depending on the sensors you attach, you can map a building, measure heat loss from a room, or even monitor oxygen levels (Derek Lartaud)

She was stymied by how to identify the position of the backpack indoors – an essential part of image localization. They ended up relying on a technique used in robotics, called SLAM (for “simultaneous localization and mapping”.) A combination of good sensors and good algorithms, it can find the backpack, construct a map and then build the 3-D models.

“Localizing the backpack – that took awhile,” she said. “We had to try many different things. You tried one thing doesn’t work, what if you try that other thing.”

They also had to tweak the backpack so it wasn’t so big and bulky, removing some components.

Now she dreams of many different ways her stripped-down 3-D backpack could be used. It could help during construction, for instance, if a plumber sees that an electrician’s wires are in the way of his pipes. It could be used in commercial real estate sales – agents could explore a property without actually visiting it. If the backpack has an energy sensor, it could measure everything emitted by lights. It could create drawings of ancient buildings that no longer have blueprints.

In fact, companies like Google are putting simplified, consumer-grade versions of the backpack’s 3-D mapping algorithm into phones, allowing users to create 3-D maps of rooms and other interior environments. It doesn’t have the customizability or the accuracy of the backpack, but it’s still an impressive feat to have that 3-D mapping power in your mobile phone.

Zakhor is particularly excited by the backpack’s potential to save lives during fires or earthquakes.

“First responders can figure out, when they get to a site, their plan of attack in terms of rescuing people,” she said. “It allows you to do much more methodical planning of how you send your people inside a collapsed building to rescue people…which staircases are connected to floors, where all the sprinklers are, or where the emergency exits are.”

Someday it could test the health of our interior spaces, if loaded with air-quality sensors that measure things like atmospheric pressure, carbon dioxide, air temperature and particulate levels.

“The beauty of the backpack,” she said, “is that, as I’m walking through, I collect as many signals as I possibly can.”

3-D Mapping Your World with a Backpack 17 August,2016Derek Lartaud
  • Elizabeth LaLonde

    When it comes to the idea of using a 3D printer to be able to map buildings is a terrible idea. the reason why I have to say that this is a terrible idea would have to do with groups that are spying for their government.
    The fact that it uses GPS is the first reason as to why I don’t think that it’s a great idea. I’m not saying that the mapping idea is a complete waste. According to the KQED website the GPS can only work with signals bouncing off in areas that have satellite available. This is where my issue begins. The fact that satellite is the trigger to make it work is something that you should have thought about. The reason I say this is because of all the hackers out there in the world, but this is only a minor problem. the major problem, however, has to do with spy’s. Compared to the knowledge of the hackers spy’s have not only the knowledge for it, but they are also trained to use it with out even getting caught. Even though “the device may not be able to pinpoint your exact location”, according to the KQED article, there is however a way for spy groups to figure out how to change this.

  • Audrey Giammalva

    If students can use their smart phones to create virtual reality video games, how would this affect students in class? In the article attached to the “3-D Mapping Your World With a Backpack” video, it states that you can, “…create immersive 3-D games.” In “The Age of Distraction: Getting Students to Put Away Their Phones and Focus on Learning” by Maryellen Weimer, PhD says that “The evidence that classroom use of technology for personal reasons distracts students.” Wouldn’t this simply just be one other thing that distracts students? Also, lets say for some reason a teacher uses this technology for a school project. How many students would actually do the assignment? In the article “Schools seek balance for cellphones in class. Are they a teaching tool or a distraction?” it is said that, “At Melrose High School, for example, science students use a physics app to collect acceleration data and measure sound intensity, and English students discuss literature on their smartphones. In Burlington, high school Spanish students practice language skills on their iPads by recording their speech and playing it back.” Would this 3-D mapping tool be used like the physics app or just another distraction for students?

    Now let’s focus on this outside of the classroom. Firefighters and police officers could use this technology to safe people and safe lives. If a building has been mapped and is on fire, firefighters could look at the map of the inside of the building to locate how to get to certain rooms rather than taking directions from someone off of a walky-talky or from directions from someone else and have to remember where to go. If someone is kidnapped or taken hostage in a building, for example, a bank, police officers could map where to get to that specific location rather through the front or back door to safe those who need saving.

    This technology could also be used as a tool for retailers. Rather than someone going through the whole house and taking pictures at every angle possible, they could just walk through and map the place, providing a better visual for possible home buyers. Architects could also use this to take other building concepts and combine them with others.

    My point is that the possibilities for this new technology can be of benefit for many people with a large range of professions. The only thing that is controversial is if students should be able to use this for themselves or for class projects. If this technology could only be available to those who will use it properly or for the benefit of others or for an actual purpose, then I would say that it would be okay to put into the hands of the public. If this kind of thing gets into the hands of someone how means to do harm, then I say that a law should be in place to prevent this kind of thing form happening. For example, there is a law that states that it is illegal to use a computer to commit a crime. If this technology has the same thing, then maybe less people will use it for negative purposes. In the end, I say that as long as this technology is in good hands, then it should be available to the public.


Derek Lartaud

Derek Lartaud came to the Bay Area after nearly five years of researching schizophrenia and diabetes at Yale University. Determined to tell visual stories, he’s worked for the BBC, Al Jazeera America, TIME, PBS, and the Center for Investigative Reporting. He has a bachelor’s degree in neuroscience and a master’s degree in journalism. When not holding a camera or editing a story, he’s trying to rebuild his 1969 Honda CL350.

Sponsored by

Become a KQED sponsor