With new technologies being used in production lines, new requirements for inspection systems are arising. One example of an upcoming 3D application in the semiconductor industry is the inspection of small connectors, such as solder balls or pins used to connect wafers and dies. These components have to be inspected with 3D methods to measure precisely the critical height of the conducting elements (Figure 1). Figure 1. Color image with pseudo-color overlay showing small pillars, each with a diameter of 70 µm (a). A segmented, pseudo 3D view (b). The pseudo-colors denote the height: Red is closer to the camera. The images are made with the Chromasens 3DPIXA stereo camera (prototype model) with 2.5-µm optical resolution. Courtesy of Chromasens GmbH. In general, the objects to be inspected are getting smaller with new technologies. The typical dimensions of such components currently are around 50 µm, requiring a high optical resolution of the inspection systems in the range of at least 5 µm. Such small metallic objects often have partially reflective surfaces and there are demands for high processing speed. Well-known 3D approaches with resolutions in that range are triangulation methods and interferometry. There is a trade-off between resolution and accuracy on one hand, and processing speed on the other. Interferometry methods provide high resolution and can handle partly specular reflecting surfaces, but are relatively slow. Triangulation methods are quite fast, but they slow down with high resolutions as well, and work best if the surface reflectivity is almost diffuse. Stereo inspection Stereo inspection is a common approach and widely used in combination with area cameras for applications such as high-precision object measurements based on special markers. Stereo inspection, in combination with line scan cameras, is a novel approach that provides high speed and very high optical resolution as well as 3D inspection capabilities. The basic principle is the same as with area cameras: Two line sensors in a stereo configuration are combined in one camera. The stereo concept is designed to capture two images in a stereo configuration from the same object. Consequently, the two stereo images show the same scene from slightly different perspectives and are the basis for triangulation, which involves an object point and two image points (Figure 2). The preconditions for stereo inspection are: 1. Only points of the object surface that are visible in both stereo images can be measured; and 2. The stereo algorithm has to find the right match of the two image points that correspond to the same point on the object surface. Figure 2. Stereo principle: The object point P is projected in both stereo images, denoted as Pr and as Pl. A triangle can be spanned including those three points, which is used to calculate the distance of point P. The stereo basis is the distance of the two optical centers of the stereo cameras. Courtesy of Chromasens GmbH. Active illuminations with fringe patterns or random patterns help to identify corresponding image points by introducing a known texture on even untextured surfaces and enhancing the stability of the algorithm. With passive stereo inspection, the structure of the object surface itself is used to find the matching points. The latter approach has the advantage that there are no limitations in resolution or in the type of illumination involved by the active illumination. The precondition for this approach is that structures on the surface of the object can be resolved in the image. There are two important parameters that influence this: the type of illumination and the optical resolution. The illumination can be chosen in a way that fine structures show up with high contrast. The right optical resolution ensures that those structures are resolved in the image. Figure 3. Color image with pseudo-color overlay showing solder joints on a printed circuit board (PCB). The width of the solder joints is about 50 µm. The pseudo-colors denote the height: Red is closer to the camera. The images are made with the Chromasens 3DPIXA stereo camera (prototype model) with 2.5-µm optical resolution. Courtesy of Chromasens GmbH. For stereo inspection, a standard approach to find the corresponding image points is based on pattern matching. In doing so, a small area around each point in one image is used as a pattern to be matched with the corresponding stereo image. The best match of the pattern in the corresponding image is defined as the corresponding image point. Doing this for all points, or pixels, in the image results in dense 3D data. Applications Typical applications for high-resolution stereo cameras are small objects with metallic surfaces that have to be measured precisely in height. One application example is to measure the height and shape of small pillars used as connecting elements in semiconductor components (Figure 1). By using diffuse illumination, specular reflexes are avoided and the underlying structure on the surface shows up. Another example is small solder joints on printed circuit board (PCB) having partly specular reflecting surfaces (Figure 3). Inspection of wire bonds requires high resolution as well. Even with the reflective surface on the thin wires, the passive stereo approach shows the relevant details in 3D (Figure 4). The wiring’s curvature, for example, can be important for the functionality and reliability of the product and can be verified with this approach. Figure 4. Color image with pseudo-color overlay showing wire bonds of an imaging sensor, with the width of the wires being about 30 µm (a). A pseudo 3D view of a part of (a) with a 5-µm resolution (b). Courtesy of Chromasens GmbH. Challenges for the future In most applications, the stereo algorithms are running on GPUs in the PC. This requires a high-speed data interface to transfer the image data from the camera into the PC, making the whole system quite complex. Quickly running intelligent stereo algorithms on embedded hardware will allow calculations in the camera, which will make the stereo systems smaller, easier to use and less costly. Such developments are crucial for the wider use of stereo inspection in industrial machine vision. On the other hand, still higher optical resolutions are necessary for the objects getting smaller. With higher resolutions, the height range gets limited since the depth of field is reduced with higher optical magnification. The extension of the height range of line scan stereo with new optical and camera designs will be another challenge for the future. Meet the author Klaus Riemer is a product manager for Chromasens GmbH in Konstanz, Germany; email: Klaus.riemer@chromasens.de. Unique Features of Stereo Inspection with Line Scan Cameras Early stereo line scan cameras, such as those developed by Chromasens GmbH, had optical resolutions of about 50 µm, similar to other prominent 3D cameras. The typical dark-field line scan illumination had been used to provide high intensity light for high speed. The advantages of those stereo cameras were speed and the ability to achieve both color and 3D imaging. A range of applications with partly reflective surfaces quickly drove demand for higher optical resolutions. For example, Chromasens developed high-resolution stereo cameras and bright diffuse line illuminations. The type of illumination is important to reduce shadows and to emphasize small surface structures. A 2 euro coin is shown with dark-field (a) and diffuse illumination (b). Courtesy of Chromasens GmbH. Features unique to today’s stereo cameras based on line scan include: • High resolution: Line sensors provide a large number of pixels — sensors with up to 16,000 pixels are available. This provides a large field of view in combination with a high optical resolution. As an example, with an optical resolution of 5 µm, the field of view with 7000 pixels is 35 mm. • Flexible illumination: With passive stereo all kinds of illuminations can be used — dark-field, bright-field, diffuse and co-axial light. The light conditions can be evaluated and optimized for each application. This opens up stereo for partially specular surfaces — something that is often not feasible with other triangulation methods. • High speed: Depending on the sensor and the interface line, frequencies up to 60 kHz are available. In the future even faster line sensors with higher speeds will be available. Line sensors with a large field of view and a high line frequency allow for inspecting a large area in a short time. As an example, with 20-kHz line frequency and 7000 pixels, an area of 3000 mm2 can be inspected in one second with a resolution of 5 µm. Much processing power is needed for high-speed image processing. Mostly GPUs are used right now to achieve the required computational power for stereo processing. • Less occlusions: With the stereo line scan camera oriented perpendicular to the object surface, there are no occlusions in transport direction. As a result, the occlusions are reduced in comparison to other stereo or triangulation methods with typical viewing angles of 20° or 30° with respect to the object surface. • 2D images in combination with 3D: Stereo provides two gray or color images from the object that are available for 2D image analysis. This is a benefit if, in addition to 3D, relevant features have to be inspected in 2D or in color.