This prototype turns your car’s windshield into a giant AR display

Car infotainment systems are getting ever bigger, but a startup’s new prototype takes things further than even something like Ford’s pillar-to-pillar touchscreen — by turning your windshield into a full-color 3D heads-up display.

Distance Technologies, established by the co-founders of enterprise headset maker Varjo, showed off its design last week at Augmented World Expo in Long Beach, California. The company’s goal is to push heads-up displays beyond flat and simple overlays. Its early efforts are impressive but also illuminate how tough (and risky) putting it into a real car could be.

The Distance Technologies prototype uses an LCD panel pointed up at a windshield with a reflective coating, projecting a transparent image onto it. It’s conceptually similar to some existing car HUDs, which can be found in a slew of aftermarket accessories and integrated options from companies like Volvo and Mercedes-Benz. But where those tend to involve flat, ghostly projections on a narrow portion of the windshield, this system is far more expansive and convincingly spatial.

Distance Technologies’ prototype works a bit like a projected version of a glasses-free 3D monitor. Its large LCD panel is covered with a parallax barrier that can display a slightly different image for each eye, while tracking cameras help determine where the driver’s looking and redraw the image accordingly. Co-founder and CMO Jussi Mäkinen says the parallax barrier hardware is just a stopgap while they’re iterating on the software tech — “our core is in software-defined optics,” Mäkinen says. CEO Urho Konttori was slightly cagey about the precise mechanics in an interview with my colleague Sean Hollister, but overall, the effect is a semi-transparent 3D display that covers the driver’s half of a normal-sized windshield.

I tested the prototype under controlled non-car conditions (a hotel room with some tilted glass simulating a windshield) and got a hint of its possibilities. The projection creates a big, fairly crisp screen that allows for anything from common HUD elements like speedometers to detailed 3D visualizations. Vehicle makers can integrate voice and gesture controls, and the prototype is hooked up to an Ultraleap hand tracker, so you can do things like hit a notification to accept a phone call without looking away from the road. The full-color projection can display videos, which means features that are currently relegated to cars’ infotainment screens — like the feed from a rear-facing camera — could sit on one side of the windshield instead.

The flashiest hypothetical features would leverage lidar or other vehicle sensors to add sophisticated augmented reality elements. In cars, Distance Technologies hopes the display could place virtual signs realistically in your surroundings or paint over patches of darkness with night vision scans. In planes, pilots could see a detailed 3D topographic map projected on the cockpit without the need to look through a headset or eyepiece.

But the prototype, developed over the past several months, has many of the rough edges you’d expect from new display tech. It apparently costs far more than the hundreds-of-dollars price point Distance is targeting for manufacturers. It’s also way too dim. The demo I saw was around 100 nits of brightness — enough to see in a hotel room, but practically invisible in a bright outdoor environment, where you’d need something more like 10,000 nits. And it requires a large LCD panel to project a commensurately large display.

The eye tracking system adds its own set of challenges. The screen’s image is constantly being redrawn to compensate for where the driver is looking, but right now, there’s incredibly high latency, so some demos waver when you move. The sensors seem to lose track of your eyes easily — turning your gaze away does it, but apparently, so does having long hair or wearing a hat. And if this happens, everything goes haywire. That responsive 3D projection turns into a set of distorted lines crisscrossing the windshield. It’s a downright scary driving scenario that means good fail-safes will be vital.

But my biggest problem by far was that using the display simply hurt. I started out looking at some of the more complex demos: a 3D rendering of a topographic map and a car navigating a city. (In real life, the former could be used by airplane pilots, the latter shrunk to a corner minimap for cars.) Within minutes, I felt like I was going slightly cross-eyed. During simpler experiences, like a flat projected speedometer, my eyes still felt noticeably tired. After I left the room, my head ached gently for hours.

I have a slightly unusual vision situation — one of my eyes is a little nearsighted, and I don’t wear glasses — and while this rarely causes problems in real-life driving or normal AR headsets, it could account for parts of my experience. That said, I spoke to one other person who remarked that the demo made their eyes tired, too. If even a sizable minority of drivers experience the same thing, it seriously undercuts Distance Technologies’ pitch.

The startup’s dream is that a driver can see useful information blended into the landscape, so they’re not constantly shifting focus between their environment and their dashboard. And the prototype can sometimes manage impressive results. In a demo that let me move a video around the scene, I found a position that had me barely flicking my eyes between it and my real-life surroundings. (Maintaining that position while driving at 60 miles an hour, granted, might pose more challenges.) But overall, my eyes were doing more work than usual, not less.

Konttori says Distance Technologies is still working out its target market. The team is apparently in touch with some consumer car manufacturers, looking to establish the kind of relationships they developed at Varjo, whose partners include Volvo and Kia. The company could also end up focusing on specialist markets like military vehicles or airplanes, where you’re already likely to find HUDs.

If Distance Technologies fixes the prototype’s big issues and gets commercial partners on board, we’ll have to hope car companies use the tech responsibly. Drivers are already irritated and sometimes distracted by a surfeit of screen-related features. AR adds a host of new possible failure points. For the near future, it’s hard to see truly important information permanently moving to your windshield through this prototype — at least without a backup system. Even so, it definitely beats driving in a Vision Pro.