Researchers at Ohio State University are pioneering the field of “imageomics.” Founded on advancements in machine learning and computer vision, the researchers are using imageomics to explore fundamental questions about biological processes by combining images of living organisms with computer-enabled analysis. The field was the subject of a presentation by Wei-Lun Chao, an investigator at Ohio State University’s Imageomics Institute and a distinguished assistant professor, during the annual meeting of the American Association for the Advancement of Science (AAAS). The presentation focused on the field’s application for micro- to macro-level problems by turning research questions into computable problems. “Nowadays we have many rapid advances in machine learning and computer vision techniques,” said Chao. “If we use them appropriately, they could really help scientists solve critical but laborious problems.” The field of imageomics promises a marriage between machine learning and computer vision that has the potential to help researchers better understand and identify ecological phenomena, such as the differences between species of butterflies. Courtesy of Ohio State University. Imageomics researchers suggest that with the aid of machine and computer vision techniques, including pattern recognition and multi-modal alignment, the rate and efficiency of next-generation scientific discoveries could be expanded exponentially. This includes creating foundation models that will leverage data from multiple sources to enable various tasks and the development of machine learning models that are able to identify and discover traits to make it easier for computers to recognize and classify objects in images. “Traditional methods for image classification with trait detection require a huge amount of human annotation, but our method doesn’t,” said Chao. “We were inspired to develop our algorithm through how biologists and ecologists look for traits to differentiate various species of biological organisms.” Conventional machine learning-based image classifiers have achieved higher accuracy by analyzing an image as a whole, and then labeling it a certain object category. However, Chao’s team takes a more proactive approach, using a method that teaches the algorithm to actively look for traits like colors and patterns in any image that are specific to an object’s class – such as its animal species – while it’s being analyzed. In this way, imageomics can offer biologists a more detailed account of what is and is not revealed in the image, paving the way to quicker and more accurate visual analysis. Assistant professor of engineering inclusive excellence in computer science and engineering and investigator at Ohio State University’s Imageomics Institute Wei-Lun Chao. Courtesy of Ohio State University. According to Chao, the technique and approach have been tested and shown to handle challenging recognition tasks, such as butterfly mimicries, in which species are differentiated by fine details and variety in their wing patterns and coloring. The ease with which the algorithm can be used could also allow imageomics to be integrated into a variety of other diverse purposes, ranging from climate to material science research, he said. Chao said that one of the most challenging parts of fostering imageomics research is integrating different parts of scientific culture to collect enough data and form novel scientific hypotheses from them. That being said, he is enthusiastic about its potential to allow for the natural world to be seen within multiple fields. “What we really want is for AI to have strong integration with scientific knowledge, and I would say imageomics is a great starting point towards that,” he said. Chao’s AAAS presentation, “An Imageomics Perspective of Machine Learning and Computer Vision: Micro to Global,” was part of the session “Imageomics: Powering Machine Learning for Understanding Biological Traits.”