Python scripts updated

I’ve just updated some of my data logging python scripts. The 1.5.0 version is still the most recommended version for beginners. In this version, I removed the dependence of cURL and replaced it with the builtin module urllib. This way you will be able to upload data without installing cURL. I also updated the API address of thingspeak.com. Although the old API address from a few years ago still works, having the new address helps. I also enabled fields 7 and 8. This means each channel can record up to 8 values instead of 6. If you have more than 8, you still need additional channels.

I also released version 1.5.7. This version is based on 1.5.0 but has better error handling and also has a config file. Only during the first execution, you will be greeted by interactive questions such as which serial port, SDI-12 sensor addresses etc. It then saves the config to a Liudrlogger.conf and will load the config from this file when it is executed next time, so no questions will be asked anymore. This helps auto run the script at boot. Also it remembers the exact SDI-12 adapter by serial number, in case you have multiple serial ports and are using Linux or RPI and these ports tend to change order on every reboot (unless you bind the hardware to a port number).

I have not updated 1.6.0 BP since not many people use multiple SDI-12 USB adapter on the same computer. If you wish, you can change max_upload_values = 6 to max_upload_values = 8 to take advantage of the up to 8 fields per channel capacity of thingspeak.com and update http_request_url_format = ‘https://api.thingspeak.com/update/?api_key=%s%s’ to http_request_url_format = ‘https://api.thingspeak.com/update.json?api_key=%s%s’

Documentation and scripts:

How I am programming and testing an SDI-12 USB adapter

In case you wonder how I managed to program and test all the adapters I sell, here is a video for that:

So I made a programming jig and put it in a box. I then have two terminal programs open, one for the programming jig’s control and the other for sending test commands to the SDI-12 adapter. I plan to write a Python script to further automate this process.

Please check USPS international service alters before shopping!

I have just noticed that USPS has suspended packages to Australia when I was searching for first-class package price to Australia at USPS.com and couldn’t find that option. There were no explanation so I searched around and found some news from last week:

Check the service alerts here. I can only ship USPS:

https://about.usps.com/newsroom/service-alerts/international/welcome.htm

Slightly lower shipping rates for France Germany and UK

I’ve recently started selling on my square online store. This store allows more freedom in setting up shipping rates. So I’ve set up US, Canada, and rest of world. Except that the rest of the world is subdivided into many shipping regions by the USPS. So starting today I’ve just set up a slightly lower rate for first-class shipping to France, Germany, and the UK. All have very similar rates that you can check here:

https://postcalc.usps.com/

Here is the link to up to 8oz of package (up to 6 adapters) to France as an example:

The USPS has the same rate for 1-8oz and then same rate for 9-16oz.

I needed a slight overhead to cover the transaction cost and currency exchange rate so I added a few dollars to the rate listed at the bottom of the list (First-class).

If you live in a country outside of US and find a cheaper rate on USPS than my “rest of the world” rate, please post a request here to have your country added to the rates. It’s a laborious work to anticipate where the shipments might go but if you make a request, I’ll make sure I do it for you.

Sentek SDI-12 soil probe troubleshooting

A number of years ago, I had an opportunity to use a Sentek SDI-12 soil probe with 16 sensor nodes in the probe. It was quite something! Lots of data to extract and lots of measurement and data commands to issue. I think that my experience with the probe helped me better understand SDI-12 protocol and ultimately helped me develop and test my SDI-12 USB adapter’s firmware. Recently one of my customers reached out to me regarding troubleshooting tips with this sensor. I felt quite interested to help and refresh my memory on how to properly use the sensor. Here are the troubleshooting tips in case you need them.

First of all, the sensor probe has multiple sensor nodes, making it require longer delays before data can be extracted from it. According to the manual, if you have 16 sensor nodes, the complete moisture measurement M! could take up to 13 seconds (first 9 sensors in the probe), then M1! could take up to 11 seconds (next 7 sensors in the probe). Salinity takes up to 23 seconds with M2! (first 9 sensors in the probe), then M3! could take up to 18 seconds (next 7 sensors in the probe). Temperature and humidity measurements are faster so they only take up to 5 seconds for all 16 sensors.

So if you have more than 9 sensor nodes, you need all measurement commands M! (AKA M0!) thru M7!. If you have 9 sensor nodes or less, you only need M!, M2!, M4!, and M6!.

What this means to you is, you must use my logging script version 1.6.x, which allows you to enter multiple measurement commands per sensor address. For instance, you have 16 sensor nodes, you want all measurements, you can enter 01234567 when asked what measurement commands to use. The ‘0’ means the M! command, which is also known as M0!. Then ‘1’ means M1! command. If you only have 9 or less sensors and you want all data, you need to enter 0246 for all measurements. But if you only want moisture and temperature nothing else, you would use 04.

Next, you want your delay between data points to be longer than these values. If you wish to save ALL data every minute, and you have 16 sensor nodes, this may be too little delay. You can try out delay between data yourself. If you see -999.999, then you need to increase your delay.

Finally, because more sensor nodes require more time before data become available, you may have to increase the serial port timeout value from 10 to a larger value, if you’re not able to obtain data from your sensor.

ser.append(serial.Serial(port=port_device, baudrate=9600, timeout=10))

This is the line you need to change the timeout. The script only waits this long after issuing a measurement command before it times out. Increasing this value will NOT slow down your data collection. The time your sensor requires to get data determines how much time is need to get your data.

Want USB AND UART/Serial both?

While it is impossible to have both USB and UART/Serial interfaces simultaneously, you CAN switch between them with the UART-version of the SDI-12 adapter and the help of an extra adapter (for USB interface).

SDI-12 USB adapters on a map

[UPDATED] I’ve been thinking about making a map of where the SDI-12 USB adapters have gone to. Now it’s finally taking shape, thanks to my son. It includes locations I shipped SDI-12 USB adapters to between 2019 and 2021 (so far). I’ll get more locations added to it for later this year but too bad PayPal doesn’t keep records older than 2019. I am only putting the city and country for each package, without postal code or any details of where. In case you wish something removed or corrected, please drop me a message. If you want to leave a message for others to see, you are very welcome to do so!

Map

My own online store is open!

So after some research and putting in some work, I now have my own online store! It’s powered by Square and has a fairly simple interface: A few items that I sell and shipping costs depending on your shipping address. I currently have 3 regions with different shipping costs: within USA, Canada, and rest of world. I will add more regions, I believe up to 6, each with different shipping costs and try my best to get the costs to be more accurate according to the quantity ordered. Anyway, here it is! Please let me know if there’s something I can improve on. If this goes well, I can retire the dated inmojo.com marketplace and the Paypal purchase links.

https://liudr.square.site/

Reading SDI-12 sensors with your Android smartphones

So if you want to get started with your SDI-12 sensors quickly, you can use your Android smartphone as a display and install a serial port app. Then within a minute, you are reading your SDI-12 sensors already! Watch my short tutorial for Android phones. I’ll investigate how to do this on an iphone and ipad as well and post my findings.

Reading SDI-12 sensors with an Android Smartphone

By the way, those 6 customizable buttons are really useful. You can long press it to edit both its name and value to a command. Then you can just tap the buttons instead of typing. You can even type in both M command that starts the data acquisition and the D command that reads back the data. You will need some delays between the commands though. The following is how I configured my serial port app for a better experience with SDI-12 sensors:

Here is the serial terminal program:

Impact of global semiconductor shortage

As many already have noticed, there is a global semiconductor shortage. Many common integrated circuits such as the ATMEGA328P-AU processor featured on most of my SDI-12 USB adapters are simply out of stock for the foreseeable future or until later 2022.

As you can see, this is going to be a problem for many businesses big and small, including my line of SDI-12 USB adapters. Luckily I grabbed enough parts last year so I still have over 200 of them that will last for a short while before I run out.

Processors (200), USB chips (200), and 100 blank boards (boards can be reordered quickly) are photographed here. Hope this lasts for a while. If you are doing your own projects, unfortunately you won’t be able to purchase ATMEGA328P-AU or ATMEGA32U4-MU etc. until they are back in stock. Try the DIP version of 328P. They are still in stock.

As a response, I will temporarily focus on only producing the basic SDI-12 USB adapter and stop producing the SDI-12 USB + Analog adapter (red) that requires a processor. You CAN still get the hi-res analog features with a basic SDI-12 USB adapter and an analog input extension board. The combined cost is the same as the red adapter.

I can still produce the SDI-12 USB + GPS adapter although it is on low demand. If “worse comes to worse” and I run out before I can get restocked, I have over 100 ATMEGA1284P-AU processors that can be used to produce the SDI-12 USB + GPS adapter, just that I won’t attach any GPS and will sell those as basic SDI-12 adapters without extension ports. Or maybe I will redesign it to have an extension port.

%d bloggers like this: