. . . and Everything in Its Place

Hank Hogan, Contributing Editor, hank.hogan@photonics.com

Machine vision advances help with sorting tasks in manufacturing.

In an automated manufacturing environment, things must be placed accurately and very quickly: Industrial sorters frequently must decide in milliseconds where a blueberry should go. Glass in a recycling plant and packages being shipped have similar speed and accuracy requirements.

Getting those decisions right demands vision technology that is fast and of high enough resolution; sometimes color is needed, too. Also, the camera-computer interface must be beefy enough to handle what can be a mountain of data.

Inspecting and sorting bottles using machine vision boosts throughput by about 75 percent over manual systems.

Vision advances have made it possible to solve some tough sorting problems. A case in point comes from Rixan Associates Inc., a Dayton, Ohio-based automation integrator that had a customer producing airplane assemblies. The manufacturing process required feeding just one out of more than 400 types of fasteners into a press, with that one particular widget varying according to the part being assembled.

The required sorting used to be done manually, but that changed a few years ago, said Stephen Harris, president and CEO of Rixan. “We looked at vision technology and realized that we could differentiate all these different parts with a machine vision system.”

The solution involves a vibratory bowl feeder, a contraption in which oscillating motion pushes parts up an incline. In the variation devised by Rixan, a vision system inspects the parts at the edge of the bowl. The correct type, which has been selected by the operator using a touch screen, goes into the press. All others end up back in the bowl.

A vision system combined with a vibratory bowl feeder allows aviation parts, represented here by candy bars of various sizes, to be sorted so that only the right type out of hundreds is selected.

The system is error-free and operates continuously, Harris said, adding that the vision technology can distinguish between parts that differ by 0.10 mm. The vision system runs at 32 fps, fast enough so that any bottleneck is a result of the mechanical movement of the robot and press. Compared with the previous manual method, the result is a 17 percent throughput improvement.

One important element, as with other machine vision applications, is lighting. Candidate parts travel over a semitransparent section, allowing them to be illuminated from below.

Lighting often is the most difficult aspect of industrial sorting to control, Harris said. There may be changing shadows due to sun, intermittent intense light from welding or, occasionally, flashing lights mounted on passing vehicles. The use of panels to block out most ambient light plus LED illumination, which is bright and constant, can help overcome such issues.

Bottles in a carton may be difficult to count because of shrink wrap and poor lighting. Good lighting often is the most important requirement for machine vision sorting.

Cognex Corp. of Natick, Mass., provided the vision systems used by Rixan in its bowl feeder. The company also is a supplier of software and sensors as well as a manufacturer of industrial bar-code readers. The two areas overlap. Indeed, vision technology is finding a home in logistics and the associated sorting, said John Lewis, Cognex’s market development manager.

An identifying bar code often is attached to packages of various sizes, which may be located anywhere on a conveyor that is several feet wide. Solving this problem with a laser-based approach – the traditional bar-code solution – requires multiple scanners angled around the belt, since the illuminating beam must traverse all lines in the bar code.

In contrast, an image-based approach can be more forgiving because software can be used to locate the bar code and read it, even if it is in an unexpected location, scratched in the middle or damaged in some other way, Lewis said. “Because it images the whole bar code, intrinsically it achieves higher read rates.”

An image-based approach also can be used to improve the production process. For instance, saving those images for which no bar code was read may reveal that packages are upside down. It may then be possible to find and fix the cause.

Hardware and software

NPI of Fort Worth, Texas, uses this technology to sort up to 30,000 parcels and flats an hour in mail applications. Such problems can now be tackled because of advances in hardware and software. Faster processors make it possible to handle more data and hit the required sorting speeds. Better algorithms also help, as these allow bar codes to be found very quickly. They also reduce the required resolution, which increases the imager’s depth of field. That makes the entire setup more forgiving.

Increasing ease of use and connectivity are strong trends, Lewis said. An example of the first involves autofocus cameras, useful when an imager is embedded deep within a machine and a process changeover is needed. As for the second, customers want access to data created by vision systems, which means that Ethernet connectivity has become a standard.

CoaXPress is another connectivity standard that could have a big impact on machine vision sorting applications. The technology runs over standard coax cables and offers a number of advantages, said Donal Waide, director of sales for frame grabber maker BitFlow Inc. of Woburn, Mass. There is a high-speed downlink of up to 6.25 Gb/s, as well as an uplink channel of 20 Mb/s per second for communications and control.

The downstream data rate is orders of magnitude faster than the 88-Mb/s-rate norm for analog cameras, which Waide said are found today in the majority of sorting systems worldwide. Switching to CoaXPress requires swapping out the camera and frame grabber, but does not demand running an extra wire for power.

“Because CoaXPress offers power-over-coax, there would be no need for additional cabling work,” Waide said.

The new interface standard is a successor to Camera Link and supports cable lengths >130 m; this should make it easier to do a better job at such tasks as sorting fruit into different classifications by skin tightness, width and height.

A fruit-sorting and -inspection system uses software from Matrox to process images of apples. Binarization and blob analysis are used to separate the fruit from its background and to obtain dimensions.

The development of CoaXPress was driven by the arrival of new sensors, processors and algorithms, noted Pierantonio Boriero, product line manager at vision component and solution supplier Matrox Imaging of Dorval, Québec, Canada. These advances, in turn, were made in response to customer demands for more capable machine vision. For sorting, this new technology allows tougher problems to be tackled.

Identify, analyze, categorize

One example is the sorting of recyclables. A particularly challenging task is to identify, analyze and categorize glass as it falls freely past a point. A decision may have to be made within a millisecond or so as a piece of glass moves past a battery of air nozzles. The nozzles will turn on as needed to deflect the glass to the proper destination.

The advent of multicore processors has changed how this is done. Before, dedicated hardware might be used; now, one or more processor cores can be assigned the task of sorting, while the rest are responsible for other activities. This approach ensures that sorting can be done in a set period of time, and the processor will not be distracted by other tasks such as interfacing with people or machines. Multicore technology does require that algorithms take advantage of the power offered, Boriero said.

Historically, he noted, machine vision has been either monochrome or gray scale. But that is changing. “Now we want to make use of color information to do the sorting. There have been a lot of advancements in color processing and analysis tools,” he said.

The use of color enables sorting to be done on more than just shape or size. That positive must be balanced against a loss in resolution that arises because pixels in a sensor often respond only to a given color. The resolution is therefore cut by half or two-thirds, depending upon the color filter scheme. It also is computationally more challenging to deal with the additional color data. Finally, the color analysis must mimic people’s color perception, as such systems often are replacing humans.

People are good at certain sorting tasks, said Benjamin M. Dawson, director of strategic development at Teledyne Dalsa Inc. in Billerica, Mass. The company, whose headquarters are in Waterloo, Ontario, Canada, supplies vision technology components to OEMs and helps end users who face difficult or challenging problems.

Counting breaded chicken fingers using a vision system: On the left is a color image, while on the right the fingers are tracked and counted.

Sorting examples of the latter involve picking beryllium nuggets out of a stream of material, determining whether a bottle is missing from a shrink-wrapped box or counting how many breaded chicken fingers have moved past a certain point. In general, the challenge here is that the objects vary in size or may touch and overlap. In addition, lighting may be poor or inconsistent.

People can easily count and sort such objects, but can only do so for about 10 minutes before tiring, Dawson said. That allows errors to creep in, one reason why a machine vision solution is sought.

That solution may not be easy to devise or implement, which can be irritating because people solve such tasks easily. However, advances now make it possible for a machine to tackle chores that before would have been very difficult or impossible to handle.

“The improvements in lighting, faster hardware and better algorithms help solve these problems,” Dawson said.