A recently approved American Society for Testing and Materials (ASTM) International standard (ASTM E3064-16) provides a standard test method for evaluating the performance of optical tracking systems that track dynamic as well as static objects. The test can be used to evaluate how well an optical tracking system can define an object's position and orientation, i.e., pose, with six degrees of freedom: up/down, right/left, forward/backward, pitch, yaw and roll.
Optical tracking systems work on a principle similar to the stereoscopic vision of a human. In an optical tracker, the "eyes" consist of two or more cameras that record the room and that are partnered with beam emitters that bounce an IR, laser or lidar signal off objects in the area. When both data sources are fed into a computer, the room and its contents can be virtually recreated.
Previous performance tests for optical tracking systems relied solely on static measurements. For systems whose safe use depends on precise control of objects in motion, such as those used to pilot automated guided vehicle (AGV) forklifts, static measurement standards are not sufficient. These systems require 20/20 “vision” for both moving and stationary objects to ensure they work efficiently and safely.
The tools and procedure used in the new standard were largely developed by engineers from the National Institute of Standards and Technology (NIST).
“The tools are two barbell-like artifacts for the optical tracking systems to locate during the test,” said electronics engineer Roger Bostelman. “Both artifacts have a 300-mm bar at the center, but one has six reflective markers attached to each end while the other has two 3D shapes called cuboctahedrons (a solid with eight triangular faces and six square faces).”
Artifact with six reflective spheres at each end used in NIST's new standard test method for evaluating the performance of optical tracking systems. These systems, similar to home gaming devices that track a player's movements, have many uses, including flight simulators, moving robots around factories, and image-guided surgery. Courtesy of Roger Bostelman/NIST.
To conduct a test, the evaluator walks two defined paths — one up and down the test area and the other from left and right — with each artifact. Moving an artifact along the course orients it for the x-, y- and z-axis measurements while turning it three ways relative to the path provides the pitch, yaw and roll aspects.
“Our test bed at NIST's Gaithersburg, Md., headquarters has 12 cameras with infrared emitters stationed around the room, so we can track the artifact throughout the run and determine its pose at multiple points,” said computer scientist Tsai Hong. “And since we know that the reflective markers or the irregular shapes on the artifacts are fixed at 300 millimeters apart, we can calculate and compare with extreme precision the measured distance between those poses.”
Bostelman said that the new standard can evaluate the ability of an optical tracking system to locate things in 3D space with unprecedented accuracy.
“We found that the margin of error is 0.02 millimeters for assessing static performance and 0.2 millimeters for dynamic performance,” he said.
Optical tracking systems are at the heart of a variety of applications including virtual reality in flight, medical and industrial training; the motion capture process in film production; and image-guided surgical tools.
“The new standard provides a common set of metrics and a reliable, easily implemented procedure that assesses how well optical trackers work in any situation,” Hong said.
The E3064-16 standard test method was developed by the ASTM Subcommittee E57.02 on Test Methods, a group with representatives from various stakeholders, including manufacturers of optical tracking systems, research laboratories and industrial companies.
The E3064-16 document detailing construction of the artifacts, setup of the test course, formulas for deriving pose measurement error and the procedure for conducting the evaluation may be found on the ASTM website.
In this segment of the "NIST in 90" series, host Chad Boutin and NIST engineer Roger Bostelman demonstrate why it's important to evaluate how well optical tracking systems can place an object, such as a mobile robot in a factory, in 3-D space. Courtesy of NIST.