Augmented Reality in the Cockpit

Author:
Anne Corning

Augmented reality (AR) may be hot in the marketplace right now, but it’s nothing new in military aircraft. “It’s been around for nearly 60 years,” says Chris Colston, director of strategic growth at BAE Systems, which built the first head-up display (HUD) for the Blackburn “Buccaneer” aircraft that launched in the late 1950s. “We’ve supplied AR solutions long before that meant anything to the mass market.”1

Today, HUD, AR, and virtual reality (VR) systems are commonplace in the aerospace and defense industries, with applications ranging from manufacturing quality control, to engineer and pilot training, to intelligence and information communication in combat operations. In particular, AR is being used inside the cockpit to integrate digital information with reality to enhance pilot awareness and safety.

Aircraft Augmented Display Types

In military aircraft, helmets often provide information via AR displays, integrated with the instrumentation, sensing, and camera systems of the aircraft for which they’re designed. For example, the new Striker® II Digital Helmet-Mounted Display from BAE systems provides night vision, 3D audio, and target tracking for fighter jets.

Another type of augmented display common in both military and commercial aircraft are HUDs, which project information onto transparent screens in the pilot’s line of sight, typically onto a windshield or visor.

Cockpit of a military aircraft with Thales head-up display shown mounted above the control panel. Image Source: Thales Group

The newest type of augmented visual systems for pilots resemble consumer AR smart glasses. For example, San Diego-based Aero Glass is selling an system that integrates smart glasses and overlay panel to provide 3D, 360-degree AR capabilities to any pilot.

Engineers at BAE are even working on the futuristic concept of a “wearable cockpit”, where a pilot’s helmet functions as a complete personal avionics suite. Can we really be that far away from a realization of the scene in Marvel’s Black Panther movie, where ex-Air-Force-fighter-pilot-turned-CIA-operative Agent Ross “flies” a mission to chase and shoot down enemy planes all from a virtual cockpit?

Cockpit AR Design Considerations

In today’s aviation industry, visual clarity is paramount for all types of augmented cockpit displays—regardless of the technology or form factor. If information displayed on a HUD screen, AR device, or helmet visor interferes with the pilot’s ability to see and quickly comprehend the real-life environment outside the aircraft, then the technology can be more harmful than helpful. 

Because an AR projection is viewed on a transparent screen with the user’s surroundings visible behind it, any letters, markings, and symbology must contrast extremely well with the background environment. This requires precise luminance and color settings that change dynamically as the environment shifts or ambient lighting conditions vary.

Another key consideration is the projection focal point. If a pilot must follow a distant horizon or scan features in the landscape, it can be tiring and distracting to have to shift focus to view information on a display screen located within the cockpit. Projecting augmented information, instead, in the line of sight of the pilot—at variable distances to more closely match the depth of the environment—can save that pilot the effort of constantly refocusing.

Ensuring AR Display Quality

Designers and manufacturers of AR displays must meet visual performance requirements for color, contrast, resolution, brightness, and focus. Information must be displayed clearly and consistently, under all ambient lighting situations and operating conditions. Ensuring that AR display screens, helmets, and smart glasses are functioning correctly requires careful measurement and testing during both the R&D and production phase. Radiant provides complete camera-and-software solutions to evaluate the quality of near-eye displays (NEDs) such as AR/VR devices and HUDs that are viewed close to the human eye.

To learn more about the benefits of NED testing with Radiant systems, including our AR/VR measurement solution, read the whitepaper, "Enabling Display Measurement within Augmented & Virtual Reality Headsets”.

 

CITATIONS

  1. As quoted in Pozniak, H., “Augmented reality: making sci-fi true for the modern military,” The Telegraph, June 27, 2019
  2. Savvides, L., “Hey Siri, take off! Get ready for more-advanced planes”, CNet, March 2, 2018
     
radiant vision system wechat