Case study: Automatically imaging the ocean and sending real-time pictures back to a base station enables faster panoramic shots for disaster prevention, aerial mapping and more. For the purposes of maritime surveillance, including coastal water inspection and intervention in dangerous situations, images must be available quickly – ideally, in real time. Ground personnel and decision makers have to know immediately when water contamination is an issue or when objects such as lost loads are drifting in the water; the information is hardly useful when the detected objects have already drifted somewhere else. A new camera from the German Aerospace Center (DLR) allows better image analysis and comparison with existing maps by revealing on the ground what the aircraft is recording from above – at the very same time. The DLR’s MACS (modular airborne camera system) family features a modular multisensor system for the manufacture of individual cameras based on different requirements. The new camera can photograph the Earth’s surface from up to 16 different perspectives, and it can take five pictures per second with full resolution. One of the first applications resulting from this concept was the MACS-Jet camera, which records 3-D images at such a fast exposure rate that it can even be used in high-speed jets. The camera also is so lightweight that it can fit in the external mount on a glider, and it is fully automatic, eliminating the need for an operator on the flight. It was developed especially for digital 3-D mapping of extensive regions from high altitudes and at high speeds. The only drawback of the system is that generation of the 3-D models takes about four to six weeks. But, as it turns out, 3-D representation is not absolutely necessary for maritime applications; optimal 3-D imaging of large bodies of water is not a practical endeavor, given the constant movement of the waves. The MACS system was used as a basis to develop a camera for 2-D digital mapping in real time. MACS-RT is a demonstrator that was designed to confirm the capabilities of such cameras; “RT” stands for “real time,” which is the camera’s strength. Its purpose is live monitoring of special incidents – natural disasters like floods or large events with lots of people or traffic. In combination with wireless data transfer, MACS-RT can send images directly to the ground station, where real-time aerial maps can be generated. Later, the 2-D images can be converted to 3-D models. The system is based on fast modules, high processing power and innovative software. Terrain data stored inside the camera is combined with real-time position and orientation data. The data are added to the images, which then are projected onto the existent map of the terrain. This digital surface model was created from data from the MACS camera system. Pictured is the Wista technology park in Berlin-Adlershof, Germany. Unlike MACS-Jet, flight altitude and speed are not the main factors with MACS-RT. Instead, the DLR researchers focused on reducing the size and weight of the system, with the goal of making it suitable for use in the external mount on the bottom of a glider. Thanks to smaller components, especially computing hardware, lighter materials and a more defined design of the mounting structure, the MACS-RT weighs only half as much as its forerunner. Because a pilot has to focus on flying, and because adding a passenger tasked with operating the camera would increase costs and reduce the maximum duration of a flight, the camera also has to be fully automatic. MACS-Jet was designed to need minimal controls, but it still needed to be switched on and off. In contrast, MACS-RT automatically starts up as soon as a specified altitude or target area is reached. The camera is being used in power gliders manufactured by Stemme of Strausberg that can remain in the air for 10 hours at a stretch and are therefore highly suitable for surveillance purposes. The images taken by the MACS-Jet camera are processed and geo-referenced by sophisticated software in real time, allowing exact measurement of distances and comparison of data with existing maps to detect any changes. A new camera developed by the German Aerospace Center (DLR), featuring built-in camera modules by PCO AG of Kelheim, Germany, has already been used successfully on jet planes traveling at speeds of up to 750 km per hour. Camera modules by PCO AG of Kelheim, Germany, are used in both MACS-Jet and MACS-RT. The pco.4000 module has 4008 x 2672-pixel resolution and features high exposure speed on the order of microseconds, low readout noise down to 11 electrons rms and a good dynamic range of 75 dB. The module also is mechanically sturdy for use in robust cameras that can be calibrated and remain geometrically stable, meaning insensitive to heat, cold and atmospheric pressure. The availability of extensive support for the instruments was one reason why the DLR chose PCO modules, according to Sebastian Pless, manager of the sensor technology section in the Optical Information Systems department of the DLR. “We can approach the company directly, and our contacts there provide us with documents containing information on the construction and electronic characteristics of the modules, which is not something every manufacturer is willing to disclose,” he said. “In this way, PCO supports our understanding of the camera modules. Not least of all, we have joint partners, which is important when making modifications to the hardware, for example.” A more advanced version of MACS-RT that is still in the planning phase is MACS-MAR, a camera system that will be optimized for maritime applications; it will use another module from PCO called pco.edge, which is highly photosensitive and therefore produces an interpretable image even under poor light conditions, for instance when used under clouds or in the early morning or late evening. A long operating time is especially important for monitoring purposes. Like its forerunner, MACS-MAR will also have real-time capability. While MACS-RT was developed as a demonstrator in preparation for maritime security research, MACS-MAR will be developed for a civil security program of the Federal Ministry for Education and Research (BMBF), Maritime Security section. The camera will go into the test phase in early 2013. It will be tested with realistic scenarios that are still to be defined; the planned project duration is three years. Some future applications will require the combination of 3-D and real-time capabilities. Mercedes-Benz already offers driver-assistance systems that can generate 3-D images at high speeds. The processor in this system can predict exactly when an object will move in front of the car. Establishing similar systems for 3-D aerial photography cameras would enable pilots and ground personnel to make better decisions while the craft is in operation; however, these systems will require much more computing power than the camera systems described above – at least for now, such technology can still be called “pie in the sky.” Meet the authors Dr. Mona Clerico is account manager at Storymaker Agency for Public Relations; email: m.clerico@storymaker.de. Dr. Gerhard Holst is head of the Science & Research Department at PCO; email: gerhard.holst@pco.de. Dipl.-Ing. Sebastian Pless is head of the Sensor Concepts Working Group at the German Aerospace Center (DLR).