Google Street View started as a project to collect data from the real world. The plan was to drive down every street, photograph storefronts and signs, and extract information like business names, hours, and addresses.

But the imagery was optimized for data extraction and lost any sense of place. Cars in the foreground were squished, buildings in the background were stretched, and trees overhead disappeared. The vibe was gone.

That was until Andy Szybalski flipped the idea around. If Google was already capturing the world at street level, could that imagery give people the feeling of a place instead? Could it showcase the physical world from the eye of a human rather than the eye of a machine?

The original project

When Google Maps launched in February 2005, Andy Szybalski was a student at Stanford. He stayed up all night dragging the map around — just because it was so novel and fun. A year later, Andy joined Google as a UX Designer and said “I knew from day one, I had to figure out a way to weasel my way onto the Maps team.” Andy jumped in to help a quirky 20% maps-related project that was trying to scan the real world.

Google’s mission has always been to organize all the world’s information. At that time, Larry Page was confident there was tons of valuable information in the physical world and funded a collaborative research project with Stanford to scan street-level imagery. The approach was simple. A camera-mounted car drove down streets, capturing snapshots that were stitched into one long strip panorama. A machine then scanned the panorama for useful information, like text and numbers on buildings.

“There wasn’t a ton of thought given to what we’d actually do with that information once we had it,” Andy said. “Which was kind of typical Google, in the early days.”

The first prototype

But Andy wondered what they would do with all this distorted imagery of the physical world — or rather what they could do. He acutely felt the absence of any sense of place and believed there was a massive opportunity to shift the imagery from the eye of a machine to the eye of a human.

He looked on the web and found some 360-degree QuickTime panoramas of spots around San Francisco. He dropped these panoramas into a Google Maps prototype to show the team what Street View could be like, if people felt like they were standing on the roadside themselves.

“It was complete smoke and mirrors. There was no real product there. The map wasn’t draggable. It was completely fake. I showed it to one of the engineers, who looked at it and said: ‘This is great, let’s ship it.’ I was like, no, no, no — you don’t understand. This doesn’t exist yet. This is total forgery.”

But the team was sold. The prototype connected the technology to an experience people immediately understood and wanted (who doesn’t want the superpower of teleportation?). So the team started rebuilding Street View from image capture to interface, all through the eye of a human.

Weaving it into Maps

After convincing the team to pursue a totally different direction, Andy had to figure out how to integrate this new imagery into Google Maps. For the sake of simplicity, he didn’t want to introduce a new concept into maps. So he tried to relate Street View to existing concepts: Was Street View really just the closest zoom level? Or an extension of satellite view?

But at the time, Google Maps had only map and satellite views and users moved around by panning or zooming in a 2D plane. There were no tools for tilting, spinning, or moving forward in a third dimension. Ultimately Andy conceded that Street View was a different concept.

But Google Maps was skyrocketing and millions of people were already using it to look up addresses, find businesses, and get directions. Street View was just a crazy experiment in a few cities. It made no sense to redesign the Maps UI around Street View. So instead, Andy had to thoughtfully weave it into the existing Maps UI.

“This was a valuable lesson for me, as an early career designer. The reality is that anything you do on an already-optimized product has the potential to do more damage than to help. So you have to have a light touch.”

Getting in and getting around

Initially, Andy introduced “Street Views” as a “layer” button, like traffic. Available streets lit up and clicking on a street opened Street View imagery inside of a window, overlaid on the map (like clicking on a pin).

Dropping into a Street View panorama felt just like the classic subway problem of emerging from a station and not knowing which way you’re facing. Users needed to understand their orientation and how to move around within the 3D imagery (all from inside a small overlay window in a small browser window in a small screen).

Eventually Andy landed on a solution he called the yellow brick road. The team projected the roads into the 3D scene and laid arrows along them. The embedded arrows served as both navigation and orientation. They included street names, which mattered because at intersections the imagery resolution was too low to read actual street signs. (The technology for depth maps and click to walk didn’t come until later.)

The whole interaction of dropping into a place depended on having something to grab. Andy started with the textbook symbol for a vantage point, a stylized eyeball.

But the eyeball failed on every count. It creeped people out, and it looked wrong from most angles (pointed North, it became “a weird bird’s nest or a hat”) and nobody wanted to pick up and drag an eyeball.

The interaction needed something anyone could happily drop anywhere on earth. After significant exploration, the eyeball evolved into an endearing character, Pegman, who we still drag around the map 20+ years later. (More on this in another post!)

The visible part of the iceberg

Over the last 20 years, Street View has evolved significantly. It went from quirky research project to a foundational part of Google Maps. But the product most of us know with the cute yellow pegman and 3D panoramas from around the world, is only the visible part of the iceberg.

Underneath all of that, is Larry’s original vision: a map of the real world. Street View could have stayed it’s original course and successfully scanned data from the physical world with no need for a human interface. There would still have been fleets of cars driving around, amassing the most valuable collection of map data.

But instead, Andy’s pivot to a product intended for people to use, also brought Street View, the product, to the world and gave all of us all a superpower — teleportation.

A note to readers: I struggled with how to structure this story because there are so many gems to share from Andy’s early design explorations of Google Street View — too many for a single, cohesive story that isn’t a tome. If you want to know more, you can listen to the full V1 conversation with Andy Szybalski on how he designed Google Street View: