Smart Solutions for Sustainable Environment
All-in-One Multimodal Computer Vision System
​BINA Pro
​
BINA Pro is a low-cost portable multi-sensor, multimodal imaging system with the Raspberry Pi single-board computer at its heart. The BINA Pro is designed mainly for agricultural research in high throughput phenotyping and precision agriculture. The unit combines crop models and algorithms, Internet of Things (IoT), and onboard computer vision. It combines thermal and multispectral bands (thermal-VIS-NIR) and pulls data from a microclimate unit to calculate a variety of useful plant parameters including crop canopy cover and crop coefficient. The unit has the ability to carry out automatic distance-based image registration, target detection (e.g. plant leaf) and background removal, and displays up to three image layers such as plant surface temperature, NDVI, CWSI, actual transpiration, etc. in real-time. It stores raw and processed geo-tagged, timestamped images, and also exports meta data and model outputs in ‘csv’ format with the click of a button. The target can be as small as part of a leaf, a number of leaves or entire canopy.
​
Key Features & Specifications
​​
-
System specifically designed and optimized for agricultural applications
-
Relies on Raspberry Pi single-board computer
-
Combines thermal and spectral bands (red and NIR)
- 100% on-board, real-time and automatic image processing (max 5 f/s)
-
On-board Graphical User Interface
-
Pulls data from a microclimate unit
-
Multiple image layers: surface temperature, multispectral, NDVI, ΔT, CWSI, stomatal conductance, transpiration, etc
-
Automatic and real-time alignment of images based on distance from the target
-
Calculates canopy cover percentage, canopy height & reference ET
-
Calculates crop coefficient (Kc) & crop ET (ETc)
-
Automatic white balance (adjustment for change in light source)
-
Calculates surface temperature of the target by automatically detecting & removing background from thermal image
-
Stores raw & processed images
-
Exports meta data and a wide range of calculated plant parameters to a ‘csv’ file
-
Thermal pixel resolution of 160 x 120 (19,200 pixels)
-
Thermal measurement accuracy of ±0.5 °C (after calibration)
-
Multispectral image resolution of up to 8 MP
Multimodal: Thermal and Multispectral Imaging
​
In many agricultural applications, we need to identify specific targets (e.g. leaves) in the thermal image. In order to do that we need to remove the unwanted background (e.g. soil, water, artificial objects). The solution is to combine thermal imaging with another imaging technique (e.g. RGB). Instead of conventional visible RGB images, the BINA Pro unit uses NIR multispectral images to detect the target/background. The added NIR band improves the ability of the onboard computer vision algorithm in distinguishing dead plants and non-organic objects from healthy leaves. The user can also benefit from saved raw multispectral or NDVI images in nutrient or disease management.
​
Three Image Layers & Automatic Image Fusion
​
The GUI supports automatic overlay and display of image layers in real-time. The image layers include surface temperature, canopy and air temperature difference (ΔT), raw visible bands, NIR band, multispectral image, NDVI (different indexes), crop water stress index (CWSI), stomatal conductance (SC), and actual transpiration (Ea). Additional image layers (e.g. photosynthesis) can be easily added to the software.
Automatic & Manual Image Alignment
​
A calibrated mathematical equation and distance (from the target) data measured by the embedded ultrasonic range finder are used to automatically align thermal and multispectral images. The GUI also allows for a very simple-to-do manual image alignment.
​
Automatic Background Removal
​
There are different ways to detect a target of interest in an image ranging from simple color-based to more accurate machine and deep learning algorithms (e.g. convolutional neural network - CNN). The BINA Pro software relies on an automatic image segmentation algorithm based on NDVI values calculated for image pixels to separate plant leaves and canopies from their background (e.g. soil, water). The user can set upper and lower NDVI thresholds that are used to remove the background from multispectral images. We are currently working on improving the process by developing and trying different machine/deep learning-based models and algorithms for semantic and instance image segmentation. This will allow the system to cover a wider range of applications with a much higher accuracy.
Communication with Microclimate Unit
​
The BINA Pro is programmed to pull microclimate data (air temperature, relative humidity, solar radiation and wind speed) from a commercially available unit (Campbell Scientific ClimaVUE50). Currently, only wired communications (SDI-12) with the unit is supported. Depending on the availability, an API can potentially replace the in-field microclimate unit and allow for pulling data from any other commercial weather station.
Calculation of Crop Coefficient
The GUI automatically calculates the canopy cover percentage (Cc, %) in the multispectral image. To calculate crop coefficient (Kc) and crop ET (ETc), it uses polynomial equations that define Kc as a function of Cc (Kc = f(Cc)). These equations are available for a variety of crops, and can be easily developed for others. Please read this article for more information on using the imaging to determine crop coefficients.
Sensor Output & Data Retrieval
​
The GUI stores raw and processed images. It also exports the results of on-board processing in ‘csv’ format. The unit is configured as FTP server allowing the user to connect to it using its FTP address. Raw images and output data can be accessed and downloaded easily.
As long as there is WiFi access, the user can also access the desktop and GUI remotely from any computer using a "Remote Desktop" app.
Compatibility with Third-Party Hardware
The BINA Pro thermal imaging module puts a pixel resolution of 160x120 at user's disposal. The multispectral sensor also supports images up to 8 MP. However, some applications might require higher resolutions. With some modifications, the BINA Pro software can work with other commercially available camera hardware setups.
Publications and Presentations
The BINA Pro project has a long history and was built on the top of years of research efforts by our team (started in 2012) and others in the area of crop thermal sensing. Please feel free read our blog articles, and watch this ("Thermal Imaging-Based Sensors for Agricultural Applications") and this ("Development of A Low-Cost Multimodal Imaging System for Real-Time Plant Health Analysis") presentations to learn more about the history of the BINA Pro.
Applications
​
Monitoring of crop transpiration, and growth
High throughput phenotyping
Monitoring temperature in fruit storage facilities
Precision livestock management
Detection and prediction of pests
Fruit and tuber vegetables size measuremet
Fruit loss management (e.g. apple sunburn prevention)
Bud, blossom and fruit count
Irrigation scheduling
Handheld imaging-based leaf porometer
Non-invasive detection of disease before visual symptoms are apparent
Monitoring of greenhouse environment (root zone, heating system, irrigation system, fans, …)
Ground-truthing data collected by drone or satellite
Detection of crop water stress
Produce quality assessment and sorting
Automation and robotics
Detection of apple bitter bit disorder
Detection of disorders (e.g. leaf tip burn due to calcium deficiency at high light)
Detection of nutrient deficiency in controlled environment agriculture