Introducing SkyMap

You’re so hopelessly lost that your smartphone is no help. You can’t reconcile its map of the landmarks around you with their real-world counterparts. You spend so much time looking down that you’re more disoriented than when you started. All you can do is look up in frustration! But what if looking to the sky was all the help you needed?

This year at ACM Spatial User Interaction (SUI) 2019 in New Orleans, we won the Best Poster Award for “SkyMap”: World-Scale Immersive Spatial Display, in which we unveiled a fundamentally new wayfinding and spatial situational awareness technique for augmented reality (AR). For thousands of years, people have navigated the world by translating their positions between exocentric 2D representations (i.e. a small paper or phone-based map) and their real-world environment. This focus-switching between frames of reference can require significant cognitive effort, especially in information-rich or high-risk situations; think about how tricky it can be to figure out where you are on your phone map while you are driving a car. SkyMap reduces the need for these cognitive transformations by rendering your position on a map that’s always in view above you, scaled to the same size as the real world, and aligned with it in every direction.

By projecting a map like a “mirror in the sky,” SkyMap essentially provides a “birds-eye view from below” of the surroundings. World-scaled and world-aligned map features appear above their actual counterparts at real-world distances. Users can simply orient themselves toward a destination on the map without needing to constantly look down or translate their position in space from a scaled and unaligned map. In formal testing, we’ve found that people have a very natural reaction to navigation via SkyMap and understand how to use the system right away. We expect this approach could prove ideal for many navigational tasks, including wayfinding in unfamiliar or low-visibility environments. And because it occupies the normally underutilized space above you, it minimizes interference with direct view of surroundings at ground level.

We first conceived of our patented SkyMap technique in 2003. To this day, a decent outdoor, on-the-move-capable AR system is still not available, but we have been evaluating SkyMap in a virtual reality (VR) testbed environment to simulate an AR display. The VR testbed places the user in 3D environment with a SkyMap prototype map above them. Active evaluations of the approach are underway or in peer review with leading researchers in academia and defense, and initial results are encouraging.

As AR hardware systems continue to mature, we continue to evaluate pure AR implementations of SkyMap. This research is conducted in the AR lab of our new office space, and is leading to more projects related to outdoor, mobile AR for situational awareness. If you’re interested in learning more, check out our research below or contact us to find out about our current opportunities!