How do you analyze soil data?

That was actually a question from one of my customers. To be honest, I’m a physicist and engineer. I wish I knew more about soil science and agriculture but I’m open to anyone who is willing to share some knowledge. My basic understanding is if you can measure dielectric permittivity with the sensor. Then you can calculate volumetric water content with formulas the sensor manufacturers provide, for a few typical types of soil. Or you make your own calibration with your own soil, such as using a different method to measure VWC (baking the soil dry?) and correlate with dielectric permittivity so you develop your own formula. More specifically the question was about METER Group Teros 12 sensor. If you use this sensor or something similar, could you share a few lines of how you get VWC and/or other data from it? Any reference materials to point to? Thank you!

New videos posted

It’s been a while since my last video introductions so I took some time to record an overview of all the adapters and extension boards.

I also added detailed introduction to each one and their features. Here is the playlist containing all of my recent videos:

Sensing tipping spoon rain gauge

If you have a tipping spoon rain gauge to sense besides other SDI-12 sensors, you can use a basic SDI-12 USB adapter equipped with an analog/digital input terminal block to sense it. You can sense up to 4 pulse sensors such as a tipping spoon rain gauge, a flow meter etc.

In the following picture you can see the 12-terminal long green terminal block between my thumb and index finger with markings “3 + – 2 + -” etc. You can either purchase this option on tindie.com when you place your order or purchase a 0.1 inch (2.54mm) pitch 12-pole terminal block and solder to the basic adapter yourself but you need to check the connectivity between the processor and the terminal block yourself because I only check that if you order this option. Connect 0,1,2,3 to + and perform analog read, you should always get near 4.995V and connect 0,1,2,3, to – you should always get 0.

With this terminal block, you get up to 4 digital counters although you will unlikely deploy 4 rain gauges within close proximity unless I imagine you want to see how an agriculture pivot irrigation system distributes water in your field. It’s still nice to have if you need one rain gauge.

Here is an adapter with the terminal block unpopulated, with the space between my index finger and ring finger. You can see more clearly the markings.

Each group of 3 pins on the terminal block is one counter so there is a total of 4 counters. There are many pulse sensors such as tipping spoon rain gauges and flow meters. In theory each tip of the spoon creates a pulse of voltage from low to high then back to low. Rotation of a flow meter creates these pulses continuously. The following shows some low-to-high and high-to-low level transitions that form pulses:

But this is only the the theory. The actual situation of mechanical contacts bouncing off and on until they settle looks like this:

So there are multiple bouncing from the time marking 1 when the signal has been low but just start to become high until time marking 2 when the signal finally settles at high. The duration of the multiple bouncing between high and low is about half a millisecond. If you feed this signal directly to the counter, the counter will register all of these pulses instead of just one.

To count a rain gauge, you usually need to condition the input because it tends to be noisy. The cause of the noise is the electrical contact bouncing of the reed switch inside the gauge. When the spoon tips, it waves a magnet across a reed switch, which short circuits the two connections on the wire. This reed switch mechanically bounces back and forth before making the contact when the magnet is near and then breaking the contact when the magnet moves far enough. The bouncing causes the two wires to make repeated connections and disconnections so each tipping of the spoon causes multiple (inconsistent) number of contact making and breaking. To condition the input, you can use a resistor and capacitor. This causes the electrical signal from multiple contact making and breaking to smooth out into a single pulse.

The rain gauge you have most likely has two contacts in the cable. Either contact can act as the signal wire and the other as the ground. The diagram below assumes there is a power wire but if you don’t have one you can ignore it. You can start with a 10k ohm resistor and 1uF capacitor and follow this diagram to solder the resistor inline with the signal wire (usually either wire), and place the capacitor between the signal and ground (the other wire). The following diagram assumes you are connecting to input 0.

Now you are almost home free. There is one more detail. The counter registers both high-to-low and low-to-high transitions instead of just counting pulses because some sensors use both transitions and others like the rain gauge only use one to indicate a pulse. If you count tipping spoon rain gauges for instance, your tip count is exactly half of the reported transition count because each tipping corresponds to a pulse that has both H-to-L and L-to-H transitions that both get counted.

Here is a photo of a setup provided by one of the users. You can see a pair of twisted green/white wires coming off one side of the 12-pole terminal block. The wires are connected to the 2 by 3 clear plastic wire terminal blocks on the top and center of the right side terminals. This type of terminal blocks connect the left side to the corresponding right side to save you from having to solder wire ends together. So the green/white wire pair connect to the pair of orange/white wire that are connected to a 1uF capacitor seen on the second photo wrapped in translucent protective tubing. Then the rain gauge with thick black wire on the right side of the 2 by 3 is connected to the white wire going to the counter terminal. The red one on the other hand connects to a short red heat-shrink tube in the shape of a side-way U with a resistor of 10k ohm inside connected to the left side of the 2 by 3. This makes the resistor inline between the red wire and the green wire. I think I might want to design a small breakout board to make this connection easier.

This shows how the resistor is inline connected.
This shows the capacitor in translucent protective tubing that is connected to the pair of orange and white wires.

SDI-12 USB adapter mechanical drawings

I’ve added the mechanical drawings to the adapter’s home page. You can print out a copy and use it as a guide to drill mounting holes and plan wiring of your data loggers. Make sure you print with “actual size” and download the file corresponding to your paper size, Letter or A4.

Mechanical dimension in mm units of the latest SDI-12 USB adapter (Letter)

Mechanical dimension in mm units of the latest SDI-12 USB adapter (A4)

Update to SDI-12 USB adapter adds more protection

It’s been a while since my last post. So here is an update. Based on inputs of a number of users, I have made some updates to the basic SDI-12 USB adapter. Here are the new features:

Zener diode between SDI-12 bus line and ground.

This protects the microcontroller from voltage spikes due to power spikes or other interference, as well as accidentally short circuiting power with SDI-12 bus. Some SDI-12 sensors come with stereo plugs, they are notorious for short circuiting all three contacts when plugging in. This causes shocks to the SDI-12 bus if you are using external 12V power, which eventually causes it to die or at least its pin to degrade.

Inline 510 ohm resistor

This also helps with transient protection. I’ve made a few prototypes and plan to test them. Here is a photo:

This update shouldn’t affect the logging script or your existing loggers. I’ll consider adding these components to the SDI-12 USB + Analog and GPS adapters later after I identify space for them since there’s not a lot of space on those variants. I recommend the 1N4737 Zerner diode if you wish to add some transient protection yourself. Just make sure to place it correctly, with the black stripe side connected to the SDI-12 and the other side to ground, on a spare SDI-12 terminal or solder between the S and – pins on the middle 8-pin add-on header:

FT231X USB IC

This IC replaces the FT232RL that is a bit outdated. The new IC will not affect any existing logging scripts, which only identifies the adapter by its vendor ID of 0x0403. If you have modified my scripts to identify the adapter by its USB vendor ID:device ID, the old device ID for FT232RL is 0x6001 while the new device ID for FT231X is 0x6015.

Volumetric water content and dielectric constant from METER group/Decagon 10HS sensor

If you are measuring volumetric water content of soil with an analog sensor such as the METER group (formally Decagon) 10HS sensor, you need to convert the voltage you read from it into either VWC or dielectric constant. The manual of the sensor gives you the conversion formula. In the following video, I demonstrated how to apply the conversion formula to an analog reading:

Even if you are using an SDI-12 sensor, its output may be dielectric constant instead of VWC, which needs conversion using a similar formula. I would do the conversion post data logging since I could potentially find a more precise calibration curve of my soil instead of using their standard mineral soil curve or potting soil curve. Without applying the curve, I can easily apply a different curve.

New compatible sensor added and data logger demo upload resumed

With the help of SDI-12 USB adapter users, I constantly add more compatible sensors to my list. This time is a soil sensor:

EnviroPro EP100G

https://www.enviroprosoilprobes.com/

Credit EnviroPro

I will update my list of sensors at the end of the month.

If you’ve been wondering why my demo data logger stopped showing data, good question. I started a project and set my router SSID and password for that project, forgetting my data logger needs it too. So I recently added an ethernet connection to the logger. Data are stored on the sd card, not lost, just not uploaded to thingspeak channel. Now that the logger has resumed logger, expect to see nice sine temperature data and occasion spikes on dielectric measurement due to raining again.

This is what I see on my side with 30 days:

Spikes are rains and no more shutting down due to outdoor outlet short circuiting and triggering GFCI.

 

SDI-12 USB adapter with larger terminals

After some design and prototyping, now I have a newer version of the SDI-12 USB adapter that features larger 3.5mm terminals. The original 2.54mm terminals can accommodate 18AWG wires and the new 3.5mm terminals can accommodate 16AWG wires. I’ve never seen sensor wires that thick although you could get some really thick wires from AC adapters. The bigger terminals makes installing wires easier with the additional spacing between pins, besides they accept slightly thicker wires. The screws are also slightly bigger, making the terminals sturdier. I have to make the terminals overhang a millimeter or so to fit them on the same board edges. I’ve kept the 2.54mm terminals for the optional analog and digital input terminals on the top edge. I moved the power selector to fit the larger terminals on the left. Otherwise, the height of the terminals is the same as before and you won’t notice a difference unless you look carefully. For example, I no longer have a 3-pole terminal for the external power. Instead, I have a 2-pole terminal to save space. Here are some photos:

The new prototype held diagonally in my hand. I had the board printed in black matte instead of regular black glossy. This has made part placement and inspection easier since there is no longer a glare from the glossy surface. Besides the terminals, I also added a footprint for the transceiver and a solder jumper. Most people will not need a transceiver. I’ve tested my adapters without this added transceiver with cable up to 100ft (30m) without visible signal degradation. You only need it if you have a total data cable length significantly greater than 100ft(30m).

Here is a side view photo of the prototype:

Here are some comparisons between the current version (top) and the new prototype (bottom):

Since the spacing is no longer 2.54mm, like the pin headers, I will have to make a header for testing with SDI-12 sensors after assembly. I also need to find proper source to buy the larger terminals in quantity, and print out more than a handful of boards. It will take a while before I am ready to sell these in my stores.

I have also designed a 3.5mm version of the SDI-12 + Analog USB adapter but haven’t built a prototype yet. It should look the same as the basic adapter though.

2019 summer run update

The 2019 summer SDI-12 USB adapter data logging run has been relatively smoothly but there were a few issues that are worth mentioning.

1. Battery

The battery that I recommended as a backup battery for raspberry pi (see photo below) needs to be used with some consideration:

I have not tested how long the charge lasts when it is powering a raspberry pi 3B (I used a raspberry pi zero last year but had to use some USB OTG adapter and a USB hub with Ethernet dongle etc. so I decided to throw my 3B in since I have a 3B+ now). What I thought should happen is that the battery will eventually drain completely after hours of power outage but once power is back on, it should turn on the raspberry pi right away. That turned out to be not the case. The battery seems to need about half an hour of charge time after it has been drained before it can power the raspberry pi 3B with enough stability that it would run normally. Within the first half hours of initial charging after a power outage, the raspberry pi was not stable and I couldn’t log in to it. The logging script didn’t run either. I ended up unplugging the pi and let the battery charge for 30 minutes and plugged the pi back in.

In case you want to prevent this issue, which may or may not affect the integrity of the pi’s operating system, you need to buy a backup battery with proper capacity. Mine is only 3000mAh, since I have the logger in my garage, only expecting at most hours of power outage:

https://www.amazon.com/TalentCell-Rechargeable-Amplifier-Multi-led-Indicator/dp/B00MHNQIR2/

There are 6000mAh and 12000mAh versions that will definitely address longer periods of power outage. If I was only expecting up to a few hours of power outage, then how come my logger drained the battery, twice?

The issue was actually with the power outlet it was plugged in, which is protected by an upstream Ground Fault Circuit Interrupter (GFCI) outlet. When I bought the house, there was no GFCI outlet in the garage! I wonder how those previous owners never thought about installing one. So I replaced the outlet in the garage with a GFCI outlet, which protects all downstream outlets. The outlet my logger is plugged in is downstream of the GFCI outlet thus is protected against ground fault, which means shorting the hot with the ground. It was the right move but unfortunately not enough. Outside my garage there is an outdoor outlet with some “outdoorish” cover. We had some VERY heavy downpours in the past month.

As you can see on the plot below, the two high spikes on soil dielectric constant correspond to two heavy rain falls:

The vertical blue lines simply indicate how quickly the rain fall caused soil’s dielectric constant to first rise then later drop (exponentially). The two almost horizontal blue lines immediately following the spikes were periods of power loss. I didn’t check my online plots and was surprised to find that the plot stopped. I guess I could use the spikes to estimate my battery’s run time to be between 4 hours and 13 hours, probably 4 hours. The following soil temperature plot during the same period shows the power outages more clearly:

So what caused the GFCI outlet to trip? It turned out to be the outdoor outlet. I took it apart. Here is what it looks like:

The gasket on the left wasn’t doing enough to protect the outlet. You can see the outlet’s rusted top side (the outlet was mounted horizontally). This must have been caused by rain water seeping into the outlet, causing ground fault.

I do recall a couple of times after bad weather our garage lost power (circuit breaker). That must have been before I installed the GFCI outlet. So in order to address the real issue that the outdoor outlet is causing, I purchased a better cover and installed a separate GFCI outlet in the power box. To be honest, the brick veneer didn’t help. It was harder to seal against due to its rough surface but the new cover with a new gasket hopefully will work better. I’ll later apply some ready-to-use cement on the top of the cover to seal any possible leaks. With this new GFCI outlet, hopefully even when it is shorted, it would act before the upstream one acts and prevents the logger from losing power for extended time again. Here is how it looks:

Next time I will write about my experience running the logging script automatically with the latest raspbian distribution.

SDI-12 USB adapter manual updated

It’s been a year since I last released an update to the manual. There has been a lot of updates since last year. I finally finished it and here is the file:

Manual 2019-06-18

I included detailed descriptions of how the optional analog and digital inputs and addon boards work on the basic adapters. There are also two one-page descriptions to easily print out and refer to. Photos were added and updated. More information about deploying your logger has been added. I’ll make an effort to update my manuals twice a year to reflect changes more frequently.

%d bloggers like this: