The Human Touch: Johns Hopkins APL Engineers Develop Tech Usability Standards

When the Army began developing the shoulder-mounted Stinger anti-aircraft missile, it designed the weapon to hit incoming aircraft 60% of the time. However, if it had been fielded as originally designed, the Stinger would have achieved hits only 30% of the time when operated by soldiers.

Why?

First, soldiers found it difficult to use; it took 18 steps to fire it. The Army uncovered other contributing issues, most notably incomplete task analyses, poor usability of key functionality, no requirements for man-portability and a lack of consideration for the cognitive and physical capabilities of the users.

In short, the technology was not ready for human use, explained Heather James, a human systems engineer at the Johns Hopkins Applied Physics Laboratory (APL) in Laurel, Maryland. “Engineers had not considered the impact of human performance to overall system performance,” she said.

James recently played a key role in developing a Human Readiness Level (HRL) scale that indicates a system’s readiness for use by humans. In September 2021, the American National Standards Institute approved a newly developed standard — ANSI/HFES 400-2021, Human Readiness Level Scale in the System Development Process — that defines nine levels of human readiness and provides guidance for their application in the context of systems engineering and human systems integration processes.

The scale is designed to complement and supplement the Technology Readiness Level (TRL) scale widely used throughout government, industry and academia. While TRL focuses on the technical maturity of a technology or system, the HRL scale evaluates, tracks and communicates whether that technology or system is ready for humans to use.

In the HRL scale, human readiness refers to the maturity of a technology with respect to use by intended users in the intended operational environment, noted Judi See, human factors engineer and systems analyst at Sandia National Laboratories in Albuquerque, New Mexico, and a key member of the team that developed the standards.

“The term does not signify the current state of physical or mental readiness of human operators who will use the technology to accomplish their mission work,” See explained. “In other words, the HRL scale supports evaluations of suitability and usability of a technology for human use, not an operator’s transitory fitness for duty at a given moment in time.”

“This is an international standard to be used widely across government, industry and academia to assess the human readiness of the solutions being developed or deployed, while in parallel the solution is technology-ready to be deployed,” James said. “The point is that sometimes technology is ready, but it is not ready to optimally ‘team’ with a human, hence a reduction in mission performance.”

There are many benefits to using an HRL scale, James said. For one, it balances the common technology-centric focus with a human-centric perspective throughout the development life cycle. It also promotes the use of objective evidence to determine a system’s readiness for human use and provides a consistent framework for measuring and addressing a system’s maturity.

The application of the HRL scale ensures proper attention to human systems design throughout design and development to minimize or prevent human error, James noted. “Failure to fully consider human factors issues at all stages of system development can increase costs and delay system deployment,” she said.

The Joint Human Systems Integration Steering Committee under the Office of the Under Secretary of Defense is working on a proposed path forward for Department of Defense (DoD) adoption of the standard. The HRL scale is also being applied to ongoing Army and Federal Aviation Administration work.

James worked closely with industry partners to formalize levels that directly map to TRL. They learned of the effort at the National Defense Industry Association’s (NDIA) 2020 Human Systems Conference. Along with See, Holly Handley, an associate professor in the Engineering Management and Systems Engineering Department at Old Dominion University, was key to the development of the scale.

Download a copy of the HFES 400-20​21 standard.