All-in-One Multimodal Computer Vision System
BINA Pro (2019 - present) is a low-cost portable multi-sensor, multimodal imaging system with the Raspberry Pi single-board computer at its heart. The BINA Pro is designed mainly for agricultural research in high throughput phenotyping and precision agriculture. The unit combines crop models and algorithms, Internet of Things (IoT), and onboard computer vision. It combines thermal and multispectral bands (thermal-VIS-NIR) and pulls data from a microclimate unit to calculate a variety of useful plant parameters including crop canopy cover and crop coefficient. The unit has the ability to carry out automatic distance-based image registration, target detection (e.g. plant leaf) and background removal, and displays up to three image layers such as plant surface temperature, NDVI, CWSI, actual transpiration, etc. in real-time. It stores raw and processed geo-tagged, timestamped images, and also exports meta data and model outputs in ‘csv’ format with the click of a button. The target can be as small as part of a leaf, a number of leaves or entire canopy.
Key Features & Specifications
System specifically designed and optimized for agricultural applications
Relies on Raspberry Pi single-board computer
Combines thermal and spectral bands (red and NIR)
- 100% on-board, real-time and automatic image processing (max 5 f/s)
On-board Graphical User Interface
Pulls data from a microclimate unit
Multiple image layers: surface temperature, multispectral, NDVI, ΔT, CWSI, stomatal conductance, transpiration, etc
Automatic and real-time alignment of images based on distance from the target
Calculates canopy cover percentage, canopy height & reference ET
Calculates crop coefficient (Kc) & crop ET (ETc)
Automatic white balance (adjustment for change in light source)
Calculates surface temperature of the target by automatically detecting & removing background from thermal image
Stores raw & processed images
Exports meta data and a wide range of calculated plant parameters to a ‘csv’ file
Thermal pixel resolution of 160 x 120 (19,200 pixels)
Thermal measurement accuracy of ±0.5 °C (after calibration)
Multispectral image resolution of up to 8 MP
Designed for Agricultural Applications
Multimodal: Thermal and Multispectral Imaging
In many agricultural applications, we need to identify specific targets (e.g. leaves) in the thermal image. In order to do that we need to remove the unwanted background (e.g. soil, water, artificial objects). The solution is to combine thermal imaging with another imaging technique (e.g. RGB). Instead of conventional visible RGB images, the BINA Pro unit uses NIR multispectral images to detect the target/background. The added NIR band improves the ability of the onboard computer vision algorithm in distinguishing dead plants and non-organic objects from healthy leaves. The user can also benefit from saved raw multispectral or NDVI images in nutrient or disease management.
Three Image Layers & Automatic Image Fusion
The GUI supports automatic overlay and display of image layers in real-time. The image layers include surface temperature, canopy and air temperature difference (ΔT), raw visible bands, NIR band, multispectral image, NDVI (different indexes), crop water stress index (CWSI), stomatal conductance (SC), and actual transpiration (Ea). Additional image layers (e.g. photosynthesis) can be easily added to the software.
Automatic & Manual Image Alignment
A calibrated mathematical equation and distance (from the target) data measured by the embedded ultrasonic range finder are used to automatically align thermal and multispectral images. The GUI also allows for a very simple-to-do manual image alignment.
Automatic Background Removal
There are different ways to detect a target of interest in an image ranging from simple color-based to more accurate machine and deep learning algorithms (e.g. convolutional neural network - CNN). The BINA Pro software relies on an automatic image segmentation algorithm based on NDVI values calculated for image pixels to separate plant leaves and canopies from their background (e.g. soil, water). The user can set upper and lower NDVI thresholds that are used to remove the background from multispectral images. We are currently working on improving the process by developing and trying different machine/deep learning-based models and algorithms for semantic and instance image segmentation. This will allow the system to cover a wider range of applications with a much higher accuracy.
Communication with Microclimate Unit
The BINA Pro is programmed to pull microclimate data (air temperature, relative humidity, solar radiation and wind speed) from a commercially available unit (Campbell Scientific ClimaVUE50). Currently, only wired communications (SDI-12) with the unit is supported. Depending on the availability, an API can potentially replace the in-field microclimate unit and allow for pulling data from any other commercial weather station.
Calculation of Crop Coefficient
The GUI automatically calculates the canopy cover percentage (Cc, %) in the multispectral image. To calculate crop coefficient (Kc) and crop ET (ETc), it uses polynomial equations that define Kc as a function of Cc (Kc = f(Cc)). These equations are available for a variety of crops, and can be easily developed for others. Please read this article for more information on using the imaging to determine crop coefficients.
Sensor Output & Data Retrieval
The GUI stores raw and processed images. It also exports the results of on-board processing in ‘csv’ format. The unit is configured as FTP server allowing the user to connect to it using its FTP address. Raw images and output data can be accessed and downloaded easily.
As long as there is WiFi access, the user can also access the desktop and GUI remotely from any computer using a "Remote Desktop" app.
Compatibility with Third-Party Hardware
The BINA Pro thermal imaging module puts a pixel resolution of 160x120 at user's disposal. The multispectral sensor also supports images up to 8 MP. However, some applications might require higher resolutions. With some modifications, the BINA Pro software can work with other commercially available camera hardware setups.
BINA Pro Demo
Most features of the BINA Pro are explained in the following videos. The first four videos are parts of the same demo session, in which an actual ''plant' 🌿 is used to demonstrate how the system works. To simplify things, in the last video we used a 'hot' cup of tea ☕ instead of a plant, put it in front of the camera and ran the software. Let's watch videos below to learn how the system 1) uses multispectral images to separate the target (plant or cup of tea) from its background, and 2) uses thermal images to calculate the hottest and coldest points on the surface of the target in real-time.
Monitoring of crop transpiration, and growth
High throughput phenotyping
Monitoring temperature in fruit storage facilities
Precision livestock management
Detection and prediction of pests
Fruit and tuber vegetables size measuremet
Fruit loss management (e.g. apple sunburn prevention)
Bud, blossom and fruit count
Handheld imaging-based leaf porometer
Non-invasive detection of disease before visual symptoms are apparent
Monitoring of greenhouse environment (root zone, heating system, irrigation system, fans, …)
Ground-truthing data collected by drone or satellite
Detection of crop water stress
Produce quality assessment and sorting
Automation and robotics
Detection of apple bitter bit disorder
Detection of disorders (e.g. leaf tip burn due to calcium deficiency at high light)
Detection of nutrient deficiency in controlled environment agriculture