At the US Open in 2004, Serena Williams lost a tennis match in which the chair umpire ruled that her return had landed out of bounds. Television replays showed that the umpire’s call was incorrect.Some understandably declared the bad call a travesty, but officiating errors are to be expected. After all, with tennis balls moving at up to 150 mph, they can cross the court in a fraction of a second and travel a distance comparable to their own diameter faster than the human visual system can perceive.Up to eight high-speed CMOS cameras, each with a dedicated frame grabber, are integrated into an automated tennis officiating system to provide timely and accurate information on ball location during a match. The data from the system also can be formatted and reconstructed to provide a detailed replay of an event. Courtesy of Auto-Ref Inc. Auto-Ref Inc. of Waterloo, Ontario, Canada, is preparing to offer chair umpires some help, with an advanced technical solution. The company’s Auto-Ref system uses up to eight 1280 X 1024-pixel CMOS cameras from Mikrotron GmbH of Unterschleissheim, Germany, to image the court. Each features a dedicated frame grabber from Dalsa Coreco of St. Laurent, Quebec, Canada, to format the images for efficient tracking. The cameras, originally developed for monitoring crash tests, are compact, have low power consumption and can acquire images at 120 frames per second. Images are formatted by the frame grabber, then sent to a computer to track and reconstruct the flight of the ball. Gordon McKay, Auto-Ref’s president, explained that the primary product of the system is a display interface that enables immediate and unambiguous “in” or “out” calls.Immediate and unambiguous determinations of the location of a rapidly moving tennis ball are not easy to come by. In 2000, when Auto-Ref was founded to provide technological assistance to tennis officials, it had several technologies from which to choose -- anything from magnetic induction to electronic contact strips. The company elected to mimic the human situation by visually evaluating the progress of the game. Besides being noninvasive, McKay said, an optical approach offers information in a form that is natural to the user.The simple concept rapidly became complex in implementation. Using one or two cameras was unsuitable for live-play situations, where occlusions are unavoidable. The baseline professional system demanded up to eight cameras, each located 30 to 40 m from the court. This resulted in challenging data-crunching requirements. The images had to be reconstructed, formatted and evaluated for intensity and brightness. With data pouring in at 120 Hz, even heavy-duty processors could not keep up.McKay continued to collaborate with Dalsa Coreco, whose director of systems architecture, Yvon Bouchard, suggested offloading the image-formatting function to the frame grabber and executing interpolation and reconstruction with a high-performance field-programmable gate array chip. Although the Auto-Ref system initially was not economically feasible, customizing and adapting the hardware for the specific application brought it into the realm of the practical. The technical details of implementing automated line-calling systems are not important to certifying bodies such as the US Tennis Association or the International Tennis Federation. Accuracy and consistency are. McKay said that Auto-Ref offers an accuracy of 4 to 5 mm -- comparable to the depth of the “fuzz” on a tennis ball -- which is the best of any available system.He said that chair umpires have been pleased by Auto-Ref’s performance in tests for the International Tennis Federation. The federation, which is continuing field evaluations of the system, has requested some tweaking of Auto-Ref, but its position paper on electronic line-calling systems cites encouraging progress and a belief that electronic line-calling technology has the potential to be introduced as an officiating aid.