The Imaging Source blog

Which neural circuits drive adaptive motor behavior? How are these behaviors represented in the neural code? Researchers at the Mathis Lab (The Rowland Institute at Harvard University) are unlocking the answers to these questions by studying brain/behavior interaction. The team, led by Mackenzie Mathis, "[aims] to understand how neural circuits contribute to adaptive motor behaviors." The challenge is to relate specific brain events to particular behaviors. Using mice as a model, the scientists are tracking behavioral events and corresponding brain activity using high-speed videography provided by The Imaging Source DMK 37BUX287 cameras and machine learning algorithms from their own open-source toolbox, DeepLabCut.

Fundamentally, the researchers must be able to accurately and vigorously track mouse behavior and deliver quantitative data to describe animal movement. "We care how animals adapt to their environment, so watching their motor actions is a great way to start to interpret how the brain does this. Therefore, the first step in our research is to observe the animals during learning new tasks," says Dr. Mathis. Her research team relies on a multi-camera system using DMK 37BUX287 cameras. Their test subjects are fast: "[...] mice can reach out and grab an object in about 200 ms, so we wanted high frame rates and good resolution" says Dr. Mathis.

Videography provides an efficient method of recording animal behavior, but pose extraction (i.e. the geometric configuration of multiple body parts) has been a problem for researchers for years. In human studies, state-of-the-art motion capture is achieved by using markers to track joints and limb movement. With animals, however, such methods are impractical for a variety of reasons. Which meant, up until now, animal behavior was tracked using manually-digitized videography (i.e. humans coding videos frame by frame) - a labor-intensive process which was often imprecise and could add hundreds of hours to research projects.

In order to automate pose extraction, Dr. Mathis's team developed DeepLabCut: an open-source software for markerless pose estimation of user-defined body parts. Based on the (human) pose estimation algorithm, DeeperCut, the researchers use deep-convolutional-network-based algorithms which they have specifically trained for the task. In a paper published in Nature Neuroscience, the authors write that the team was able to dramatically reduce the amount of training data necessary by "adapting pretrained models to new tasks [...through] a phenomenon known as transfer learning." DeepLabCut has become so robust and efficient that even with a relatively small number of images (~200), "the algorithm achieves excellent tracking performance". Many scientists are hailing the development of the software as a "game changer". Mathis Lab also uses The Imaging Source's IC Capture and has added a camera control API for The Imaging Source cameras to GitHub.

DeepLabCut automatically tracks and labels (red, white and blue dots) a mouse's movements. Image credit: Mackenzie Mathis

October 8 marks the start of STEMMER IMAGING's fourth Machine Vision Technology Forum. Approximately 40 leading machine vision manufacturers will present their latest developments and state-of-the-art technology, for both newcomers and pros, in a series of presentations and exhibitions. Specifically, attendees can level-up their machine vision expertise and speak with experts from seven areas: IIOT, Embedded Vision, 3D-Technology, Machine Learning, Spectral Imaging, Future Trends and Fundamentals. During the five-city European tour, The Imaging Source will present its latest embedded vision solutions and give a talk on the advantages of FPD Link III - a technology which allows for cable lengths of up to 15 m.

• October 8 - 9, 2019: Munich, Germany.
• October 15, 2019: 's-Hertogenbosch, Netherlands.
• October 17, 2019: Paris, France.
• October 22, 2019: Stockholm, Sweden.
• November 13 - 14, 2019: Birmingham, England.

Registration has already begun, so please click on the links above for additional information regarding each of the five events.

Shenzehn, China's electronics manufacturing hub (with an economic output exceeded only by Shanghai and Beijing), is home to South China's premier automation show, Integrated Automation Motion & Drives SHENZHEN 2019 (IAMD). The Imaging Source and reseller Sunvision attended the show from June 26 - 28, 2019.

Continuing Chinese governmental support for increased automation and efficiency provides the backdrop for the show's focus on automation and its key enabling technology: machine vision. In addition to a static camera display featuring the latest USB 3.1 (gen. 1) and GigE cameras, The Imaging Source booth featured a toy "factory", offering a dynamic miniaturized factory floor using various camera models to complete typical inspection and automation tasks. Completing the line-up was a stereo camera system featuring DMK 38GX267 cameras which replicated a high-accuracy positioning system for screen application/lamination.

AI Labs, a subsidiary of The Imaging Source dedicated to automation intelligence and artificial intelligence, has just released its first product, Pick & Load, to the public.

Pick & Load is an affordable and efficient solution for the automation of machine loading and unloading tasks. Even small and medium sized enterprises can now automate their CNC loading and unloading tasks. A compact 3D sensor with active illumination is integrated with an embedded system and smart software to deliver a cost-effective solution with low power dissipation and a compact form factor. With a minimum amount of time and effort, Pick & Load can be configured to load and unload unpalletized parts from tables, palettes or drawers.

Those people looking for the latest trends in embedded vision at this year's Embedded World in Nuremberg could not overlook the presence of MIPI/CSI-2. Behind the abbreviation MIPI/CSI-2 is the Camera Serial Interface 2 (CSI-2), specified by the Mobile Industry Processor Interface Alliance. This alliance, which consists of over 250 companies worldwide, specifies interfaces for mobile devices which includes not only camera interfaces such as CSI-2 but also, for example, interfaces for displays (Display Serial Interface 2, DSI-2) or audio devices (SoundWire, SLIMbus). MIPI Alliance's focus is the interface standardization of mobile end devices, enabling various interfaces to operate in the same physical layer. In the meantime, MIPI/CSI-2 has firmly established itself in industry and the embedded systems used there. The reasons for this are diverse: For one, SoCs (Sytem on a Chip) originating from the smartphone segment, became available as industrial variants, offering the CSI-2 interface by default; and for another, MIPI interface components are very widespread, well-tested, inexpensive and energy-efficient.

MIPI for Vision

Today's SoCs with MIPI/CSI-2 inputs generally offer hardware-accelerated image pre-processing operations via an Image Signal Processor (ISP). The ISP takes over operations such as de-mosaicing or color correction and, on some platforms, even demanding tasks such as H.264/H.265 coding or distortion correction. The ISPs usually only process data that is delivered via the MIPI/CSI-2 inputs. This excludes, therefore, the processing of data from GigE-Vision or USB3-Vision devices via ISP. Optimal use of the SoC's hardware resources (including ISP), however, requires the MIPI/CSI-2 interface. The SoC performs almost all image pre-processing tasks (i.e. operations that were often calculated directly in the camera in the case of industrial cameras) allowing for the use of compact and cost-effective camera designs. Another driver for MIPI/CSI-2 is currently the automotive industry's use of intelligent driver assistance systems. Today, hardly a vehicle rolls off the assembly line without camera modules or displays. In addition to digital rear-view mirrors, surround view, distance control or collision avoidance, MIPI Alliance protocols are also used for such components as infotainment systems.

15 m Cable Thanks to FPD-Link III

Especially in the automotive sector, however, one is quickly confronted with the problem that standard ribbon cables, such as those used in smartphones between SoC and camera module, rarely allow cable lengths beyond 30 cm. Camera modules in an automotive surround-view application, for example, require cable lengths of several meters. The same often applies to industrial applications where camera modules are being installed into systems. The Flat Panel Display Link III (FPD-Link III) interface from Texas Instruments provides a solution. Designed for the transmission of high-resolution video data for automotive applications (in addition to pure data transmission), the interface offers bidirectional channels for control commands (e.g. for configuring a camera module via I2C or feedback from a touch display), as well as the option of power supply via a single coaxial cable. Such cables are thin, flexible and inexpensive - features that play a decisive role in price-sensitive market segments like the automotive industry. Two additional components are used to transmit the MIPI/CSI-2 signals via FPD-Link III: a serializer that translates from MIPI/CSI-2 to FPD-Link III and a deserializer that translates from FPD-Link III back to MIPI/CSI-2 (Ser-Des). While the serializer is placed directly on the camera module, the deserializer is located near the MIPI/CSI-2 input of the processing SoC. The FPD-Link III path is completely transparent for the user. The Imaging Source recognizes the need for longer transmission systems and now offers, together with its MIPI/CSI-2 modules, FPD-Link III bridges for common embedded systems such as NVIDIA Jetson.

The above article, written by Dr. Oliver Fleischmann (Project Manager at The Imaging Source), was published in the April 2019 edition (02 2019) of the German-language industry journal inVISION under the title, "Einfach länger: MIPI/CSI-2 Module bis 15m Kabellänge mit FPD-Link III". Translated into the English by Amy Groth.