A binocular vision-based platform has simulated autonomous aerial refueling (AAR) between two unmanned aerial vehicles (UAVs) in real time, using a boom approach. The platform may be the first step toward a vision-based sensor and navigation system that could enable in-air refueling of UAVs without pilot assistance. In order to verify that a vision system could be designed for boom approach AAR, researchers at Beihang University built and tested an integrated vision-based platform consisting of a tanker UAV, a receiver UAV and a ground station. The octocoptor serving as the tanker was equipped with cameras and a visual measurement system. When the UAV serving as the receiver came within the visual field of the cameras on the tanker (approximately five meters), the binocular vision system was able to capture pictures of the markers on the receiver UAV and extract and generate the pixel coordinates of the markers. An on-board next unit computing (NUC) device processed the pixel coordinates and estimated the position of the receiver. The visual information obtained from the binocular vision system was transferred to the flight controller and boom controller, which used the pictures to control the flight of both UAVs and the movement of the boom toward the fuel receptacle. Micro air vehicle (MAV) link protocol was used to control communication between the ground station and the UAVs and between the UAV subcomponents. The researchers based their binocular vision system on two primary procedures: feature extraction and pose estimation. The pixel coordinates of the markers on the receiver UAV, which were painted red, were generated after feature extraction. The generated pixel coordinates were utilized in pose estimation, and the pose estimation was used to calculate the relation between the binocular vision system and the marker coordination system. Transformation from the red-green-blue (RGB) to the hue-saturation-value (HSV) color model was implemented to process each captured image. In order to segment the regions of image containing red markers, the upper and lower bound of hue and saturation channels of HSV image were selected to conduct the threshold segmentation. To confirm the effectiveness of the binocular vision algorithm, the researchers simulated a refueling boom system. After the frame sequences of the markers were obtained and pose and position were estimated, an arm microcontroller used this information to determine the control level needed to control steering engines. The refueling boom pointed at the receptacle in the resolved pose and position, and successfully implemented in-air connection. A series of flight tests were conducted outdoors under various conditions, including in conditions of strong light and weak light. To ensure safety and ease of observation, the flight height of the UAVs was approximately 10 meters, or 33 feet. The results of the experiment showed that a binocular vision system for AAR of UAVs using a boom approach is feasible. The research team plans to focus its future efforts on improving the stability and accuracy of their binocular vision measurements in more complicated environments. UAVs are used extensively in military and civilian initiatives. Since most unmanned systems are designed to execute long-running missions, it has been necessary for UAVs to return to base periodically for refueling. As use of long-running UAVs grows, autonomous aerial refueling will be of increasing importance. The work of the researchers was partially supported by National Natural Science Foundation of China (Grant Nos. 61425008, 61333004, 61273054) and Aeronautical Foundation of China (Grant No. 2015ZA51013). The research was published in Science China (doi: 10.1007/s11432-016-5553-5). A binocular vision-based UAVs autonomous aerial refueling platform. Courtesy of Science China Press.