RGB cameras built on Raspberry Pi have become a very popular agricultural research tool because of their lower cost and higher flexibility compared with commercial cameras. The best example is perhaps using Pi cameras for indoor plant phenotyping in the PlantCV project (DDPS Center). Using Pi cameras outdoors, however, is a bit challenging.
I was introduced to the world of credit card-sized Linux single board computers (SBCs) in 2015 by a friend of mine who is an electronics engineer. At the time, he was developing an IoT-enabled device by combining a BeagleBone Black SBC and a USB microscope to monitor bees and beehives. So I started with the BeagleBone first and then switched to the Raspberry Pi. Why I did that is another story for another day, but I've had great experience with both. If you've only played with the Raspberry Pi, you may want to give BeagleBone boards a try as well!
Figure 1. Assembly and test setup for the low-cost imager for continuous unattended field monitoring in agricultural fields.
A few years ago, I had to build several multimodal (thermal and RGB) Raspberry Pi cameras (Fig. 1) and mount them on a center pivot irrigation machine in a mint field. They had to stay in the field for at least an entire growing season (~4 months) and capture images automatically. Even though it seemed straightforward, I was facing several challenges. I had to find answers to several important questions, the two most important of which were:
How to waterproof the cameras?
How to keep them powered?
I built the imager and tried the housing in several projects. It proved to be very robust, and shock resistant, yet very inexpensive to print and assemble. I did this project in 2016-2017 and published a few papers about it. Nevertheless, the news hasn't quite reached the intended audience and I still hear in conferences and seminars people asking how they can deploy Raspberry Pi cameras in the field.
In this article I'm going to go over the first challenge and share with you the details of the [almost] weatherproof field enclosure I developed for the imager including its 'stl' 3D design files (at the end of this article). I will discuss the second challenge in another article. If you're not patient enough to wait, please feel free to read papers listed in the references section and my white papers and research articles. They provide a lot of information on the camera and its field applications.
Thermal-RGB Imager Design
Most Raspberry Pi cameras rely on only one imaging sensor/module (RGB or thermal), but for this project I needed a multimodal imager. My plan was to use RGB images to create masks (image processing) for thermal images of the same target. Fig. 1 depicts the electronic components of the thermal-RGB imager I developed I built. The electronic hardware is comprised of a Raspberry Pi SBC (Raspberry Pi Foundation) as the core, radiometric thermal module with shutter (FLIR Lepton® 2.5, FLIR Systems, Inc., Wilsonville, OR), RGB Raspberry Pi camera module (V2, Raspberry Pi Foundation), GPS module (Ultimate GPS Breakout, Adafruit Industries, New York City, NY), 2-channel relay board (SunFounder, Shenzhen City, Guangdong Province, China), and a precise DC-DC step-up/down voltage convertor (S18V20ALV, Pololu Robotics and Electronics, Las Vegas, NV).
The Lepton v.2.5 thermal module has a resolution of 80 (horizontal) × 60 (vertical) pixels, , frame rate of 9 Hz, and a spectral response wavelength range of 8-14 µm. The horizontal field of view (HFOV) of the module is 51°. I used a breakout board (FLIR Systems, Inc., Wilsonville, OR) with the SPI communication method to acquire data from the module. The resolution of the Pi camera is 3280 × 2464 pixels, and has HFOV of 62.2° and vertical field of view (VFOV) of 48.8°.
Figure 2. Electronic components of the thermal-RGB imager. The electronic hardware is comprised of single-board computer, thermal module (radiometric with shutter), RGB camera module, GPS module, and relay board.
During my preliminary experiments with the thermal module, I noticed that the module would freeze after working for a number of hours. In order to resolve this issue, I added a relay board to reset the module automatically or manually. The GPS module has an accuracy of ±1.8 m under ideal conditions, and only works with the traditional latitude/longitude/altitude system. The GPS provides a real-time clock (RTC) to the Raspberry Pi.
To ease configuring the imager in the field and to allow for real-time monitoring, I developed a graphical user interface (GUI) in the Qt IDE (The Qt Company, Santa Clara, CA) using various C/C++ libraries including OpenCV for computer vision. The main features of the GUI include automatic real-time overlaying of RGB and thermal feeds, manual capture mode (snapshot), programmable capturing time window, and automatic interval shooting. I programmed the imagers to capture images automatically at specified time window (10:00AM - 2:00PM) and time intervals (1 min) and automatically shutdown to save power.
Captured images were processed and stored on a 16-GB SD card. Four images were recorded at each shooting: 1) thermal image in binary format, 2) thermal false color image in JPG format,3) RGB image in JPG format, and 4) automatically aligned RGB and thermal images in JPG format. GPS coordinates were also stored in TXT format separately for geotagging images. A the aforementioned logging interval (1 min), the available space on the SD card was enough to continuously record images for about 45 days.
Weatherproof Housing
The Raspberry Pi unit can produce a considerable amount of heat depending on CPU usage, which needs to be dissipated. At the same time, dust, humidity and water need to be kept out of the enclosure. To design an appropriate housing, I conducted a series of experiments with various custom-design enclosures. The final enclosure design can be seen in Fig. 3. It is weatherproof for the most part, with the most sensitive points being the thermal and RGB module lenses. I added the conical frustum-shaped head to keep drops of rain away from the camera modules. I embedded a hole in the housing for the cables and wires to go in or come out, and sealed the hole using duct sealant. Duct sealant was very cheap and worked really well. An alternative, however, is 100% silicon sealant. The imager housing measures 20 cm in length. The diameter of the widest section is 16 cm and the diameter of the narrowest section is 9 cm. The distance between the camera module lenses is 25 mm.
Figure 3. Thermal-RGB imager enclosure.
I designed the housing in Tinkercad (Autodesk, Inc., San Rafael, CA) and printed using an Ultimaker 3D printer (Ultimaker, Geldermalsen, Netherlands) in our lab, even though I do not recommend this model of 3D printer to anybody! As it can be seen in Fig. 3 (top view), I attached an external GPS antenna and bullseye level to the imager enclosure. Provided the imager is installed at nadir view, the lenses will be protected from precipitation but not dust or tiny droplets of water traveling in the wind during an irrigation event. After leaving the imagers in the field for several months and many irrigation events, that proved to not be an issue.
Figure 4. All the electronics were attached to a compartment (printed in gray filament) except for the camera modules, which were screwed to the cap.
The heat produced by the Pi CPU will accumulate at the top of the housing away from the electronics and dissipate gradually. I designed and printed an internal compartment (Fig. 4) to secure electronics including the Pi inside the enclosure. I attached all the electronics to the compartment except for the camera modules, which were screwed to the cap. The cap was fastened to the body using two small screws and sealed using 100% silicon sealant.
3D Design files (stl format)
I have put the zipped "stl" files for the 3D design on our download page (3D Designs > DurUntashLab_Thermal_RGB_Imager_Housing_WSU_2017.zip). You can use the design as is or modify if you like.
I strongly recommend that you print the design using ABS filament. PLA works just fine, but you need to be careful not to leave your reader under the sun or in your car during hot summer days, or it might deform or even melt! If you have a 3D printer that can print ABS filament, please use the following settings or follow the manufacture's instructions to achieve the best results:
Material: ABS
Layer height: 0.25 mm or finer
Infill: 80%
Brim width: 8 mm
Print speed 40 mm/s
Printing temperature: depends on filament brand (220 °C, default)
Build plate temperature: 100 - 110 °C
Radiation Protection
Fig. 5 shows how I installed the imagers in the field. Under normal use, prolonged exposure to sunlight can result in some degradation and yellowing of the enclosure. This, however, does not affect the imager electronics in anyway. I strongly recommend that you shield the imager with UV protectant (example) before deploying in the field. This layer of protection can significantly increase the life of your imager especially in situations when exposure to sunlight for extended period of time is unavoidable.
Figure 5. Thermal-RGB imager mounted on a center pivot irrigation machine.
Figure 6. Sample thermal and RGB images of mint plants automatically captured by the thermal-RGB imager.
References
Osroosh, Y. et al., 2019. Detecting fruit surface wetness using a custom-built low-resolution thermal-RGB imager. Computers and Electronics in Agriculture, 157: 509-517.
Osroosh, Y. et al., 2018. Economical thermal-RGB imaging system for monitoring agricultural crops. Computers and Electronics in Agriculture, 147: 34–43.