SDI-12 USB adapter with larger terminals

I have recently received a few comments regarding the SDI-12 USB adapter’s terminals being too small. As a matter of fact, they are not big. They are 2.54mm (0.1″) pitch terminals. On the other hand, they can comfortable accept wires as thick as 18 AWG. I’ve rarely seen sensor cables having wires thicker than 20 AWG. Larger gauges are thinner so 18>20>22>24, AWG-wise. Also, if you have wire leads that are not tinned, you should twist the wire strands and tin the leads before inserting them into terminal blocks.

Still, having wider pitch makes it easier to insert the wire leads, including power and ground for the external power if you need that. So I made an update with 3.5mm (0.1385″) pitch terminals. I really like the size of the board and the mounting hole positions have not been changed since I made these square boards. I would like to keep them unchanged for past customers who may rely on the size to make more loggers. So here is an updated version board view (top) vs. current version (bottom):

The new 3.5mm pitch terminals will hang over the edge of the board a bit but fit the same board, after I moved the external power jumper a bit. To save space, I used a 2-pole terminal for the external power connector instead of 3-pole. The 3.5mm terminals can accept up t o16 AWG wires. I will print this board out during my next board order, which is probably a month from now. If you care to give me your opinion, please use the poll below, leave me a message, or write me a private email to zliudr@gmail.com. I will print this board on paper and make a mock-up to compare side-by-side with the current version. There is no reason both versions can’t coexist.

I also considered 5.07mm (0.2″) pitch terminals but they are just way too big to secure sensors with thinner wires. Let me know if you are interested in having this 3.5mm terminal version and why. If there is enough interest, I’ll make a batch or two.

I have a photo of 2.54mm vs. 3.5mm vs 5.07mm terminals:

2.54mm terminals are narrower than 3.5mm terminals. 5.07mm terminals are both wider and taller than the smaller versions. These are the most common terminal sizes on circuit boards.

 

 

Conversion table

2019 demo logger started on 6/3/2019

Last year’s demo logger data stream was very successful. It demonstrated the stability of my SDI-12 adapters for long-term data logging. It ran between June and October (Minnesota is cold) for 124 days. Except for occasional power outage at my house, the logger was running without a problem using the 1.5.0 logging script. This year I updated the logging script to 1.6.0 and have got a CCTV power bank to use as a backup power supply in case of power outage.

Here is a battery similar to mine that is sold on amazon.com:

The nice thing about this battery is that it gets charged via the 12V power barrel and discharged via the USB connector while it is charged. Most power banks can only be charged or discharged but not both. This one does them simultaneously. It is always charged to full when there is power in the AC. The USB port always has 5V power either from the AC or from its internal battery. Essentially this is a cheap Uninterruptible Power Supply. There is no surge protection except if your power strip has it. It is also very compact. I’ll embed it in an enclosure for a more complete enclosed logger later this month.

I expect to see next to zero down time due to this battery. This is also great for the raspberry pi since every time it loses power it could corrupt the SD card a little.

Here is the first day of data. We had some rain in the afternoon.

You can visit the live stream by clicking here.

Summer consulting projects

The summer is finally coming! This year we had a lot of snow and I was busy during the semesters. I anticipate to do some travel this summer and further develop my data logger solution but still I have the bulk of May to August open for consulting projects. Let me know if you need my help!

Update to SDI-12 USB adapters

I recently got a request to add bidirectional transceivers to my SDI-12 USB adapters to handle very long SDI-12 bus wires (result of long wires for each sensor and a large number of sensors). Currently a couple of these adapters are being tested by one customer who requested this feature but I am pretty confident with its functions and will conduct my own testing with long wires. If this is what you have in mind, I have a handful of them I’ve built as prototypes. You can go ahead and purchase a regular SDI-12 USB adapter and request one with a transceiver. I don’t have a lot of them so I can only send you one or two. If you really need more of these, I’ll need to order boards and components.

The added transceiver will not affect any program code such as my Python data logging script. It is operated transparently. When the adapter receives a complete SDI-12 command, it will turn on the transceiver and transmit the command to the SDI-12 bus. Once done with transmission, it turns the transceiver off and returns to listening mode. The transceiver in the following photo is located just to the left of the top-right 3-pole terminal block (small black rectangle with 6 pins).

Additionally, I have received several requests to use my USB adapter as a TTL/serial adapter, such as connecting to arduino or MicroPython boards, either at 5V or 3.3V. I’ve updated my board design to make those requests easier to fulfill. This option is now added to inmojo marketplace as well as to Tindie marketplace (options used to cost $2.5 and now is free).

First, if you purchase a TTL/serial only adapter, you will not get USB connection anymore (notice the missing long black chip to the right of the empty USB connector pattern). You can’t really have both active simultaneously since there is only one TTL/serial port on the processor. It’s either connected to the breakout pins for TTL/serial use, or connected to the USB chip to communicate to PC/raspberry pi. The use cases of USB vs. TTL/serial also don’t overlap. One is for those who want to use PC or raspberry pi to log data, and another who want to use MicroPython boards or Arduino boards to log data. What you will get is a 6-pin connector on the bottom of the board, at 90 degrees so it’s not pointing straight down, rather sideways. See how the wires are under the board, running along the board and the next photo for the underside. This makes it possible to stack expansion boards or have optional analog/digital input headers (12-pole block on top edge). You still need a 5V supply even if you want a 3.3V TTL/serial interface. The following photo shows a 3.3V version. Note the solder blob on the top right to the immediate left of the text “TX3”. Then the TX3 on the serial port (marked JP9 on left and Serial Port on right) is outputting 3.3V logic. Remember that the adapter’s TX or TX3 should be connected to your other board’s RX pin since the adapter’s transmit (T) goes to your other board’s receive (R).

Simple Python SDI-12 logging scrip

In case you wish to integrate the SDI-12 USB adapter into your existing Python script, here I provide a simple script to demonstrate how to get data. You can also use this script as a spring board to establish serial port communication in Python for anything else, such as talking to Arduino.

The goal is to not complicate things with the full-feature data logger script. You’ll see that the actual data logging only needs a few lines of code:

Simple sensor detection and reading (2018-12-03)

This script demonstrates how to integrate SDI-12 sensors into your existing Python data logging system by providing the minimal necessary features. It MUST run with a single sensor on the bus. For full-featured logging script, download the Data Logger script.

I’ll keep this script together with all my other scripts and provide updates to it when necessary. I’m also considering writing simple scripts for other programming languages. Do you wish to use the SDI-12 USB adapter with a programming language other than Python? Leave me a message here.

2018 5-month SDI-12 USB adapter run

I conducted a long run (5 months) on an SDI-12 sensor with a raspberry pi zero in the months of June to October to test the long-term stability of my data logging solution. I was using a regular SDI-12 USB adapter, a Decagon/METER Group soil sensor model 5TM with dielectric constant and temperature sensors and I buried it in my back yard. Over 100K data points later, the ground was freezing and I unearthed my sensor and stopped the feed. Here is an overview of the data feed. Please click the plot to see a full-size screen grab.

The top plot is dielectric constant. Each spike represents a major rainfall in my area that made the electrical conductivity of the soil shoot up and then gradually die down. Each small step is when I turned on my sprinkler system to water my lawn (smaller than the spikes but still significant within a range of a few days).

The bottom plot is ground temperature, about 20 cm in the soil. The diurnal temperature variation is greater during the summer than during the fall, as represented by the larger amplitude oscillations of temperature at the beginning of the graph and the gradual reduction of oscillation amplitudes towards the end of October.

Despite of a number of power outages (can you even see them from the graphs?), the system worked flawlessly, restarting when power was restored. In retrospect, I should have invested in a backup battery such as this one:

https://www.amazon.com/gp/product/B01M7Z9Z1N/ref=oh_aui_detailpage_o02_s00?ie=UTF8&psc=1

It is charged by a 12V ac adapter and provides 5V power on its USB port. When the ac power is out, it uses its internal battery to continue to provide 5V power. For a small system such as raspberry pi zero, it could power it for hours.

If you wish to see the complete data set, here is the link to download:

https://www.dropbox.com/s/4sry1pqwp9hdxuo/2018%20summer%20fall%20lawn%20data%20feeds.xlsx?dl=0

For more information on the SDI-12 adapter and data logging scripts, go to their dedicated page here:

https://liudr.wordpress.com/gadget/sdi-12-usb-adapter/

More about the data logger

Enclosed logger

Now that the logger prototype is enclosed, it is a lot easier to use. I have a METER group 5TM soil sensor connected to it from the left side and 12V power to it from the bottom. Without a heavy enclosure, these cables will make it hard to set the logger flat and connect an adapter to it. Plus, I’ve started using Telnet to connect to it wirelessly so I don’t need an adapter to connect to it most of the time.

Here is the adapter I modified to use on the logger:

2018-07-26 22.33.25

I bought it from moderndevices.com (Not shown on this photo) Later I had to short a resistor and a capacitor in order to actually make it work, due to the fact that the adapter was designed for Arduino boards.

Besides Telnet, I can also upload Python sketches via FTP. I later installed a couple of jumpers to easily reset the logger since I am still developing the logging script and need to reset the logger when I make changes. I’ll install a push button instead of the jumpers when I have some time.

Everything is coming along nicely. Soon I’ll start testing 4G LTE-M connectivity of the logger. 4G LTE-M has a lower bandwidth than the regular 4G LTE on smartphones so the modem would consume less power and data plans would cost less (in Megabytes instead of in Gigabytes). The Digi XBee3 LTE-M modem is rather expensive, at $100, or $150 for a dev kit. Anyway, if you are deploying the logger in a research field close to your lab/office or really don’t mind regularly visiting remote areas to collect data logs, you may skip the modem.

SDI-12 USB adapter upgraded

2018-07-02 15.25.47

After some more firmware development and testing, I am happy to announce that the upgraded SDI-12 USB adapter is now available. The above is the first batch of these adapters and one hi-res analog input add-on board.

The upgraded board features the following:

  1. 4 analog inputs. 12-pole terminal block that features 4 analog voltage inputs. These are 10-bit or 5mV resolution inputs without differential reading. There are there to provide basic voltage inputs for projects that don’t require hi-res analog voltage inputs.
  2. Pulse counters. Alternatively, these 4 inputs can be used as pulse counters. Say you have a rain gauge or flow meter that outputs pulses, these pins can count the pulses. You may need additional filtering (one capacitor and two resistors) if the pulses are noisy. Each time you read the pulse counts, you get the counts since you last read and the adapter will start counting from zero again. This way, if you collect data every minute, the counts will be counts/min. Because each data point is accompanied with date-time information, you can always calculate the count rate with your data set.
  3. Extension port. There is now an 8-pin extension port for add-on boards. The first extension board I have designed and tested is a hi-res analog voltage input board. This board features the same four 16-bit auto-scaling inputs and differential inputs as the SDI-12 + Analog USB adapter, with an added benefit of address jumper. You can add as many as 4 such extension boards to the new SDI-12 USB adapter, with each extension board taking a different address. That is up to 16 hi-res voltage inputs.
  4. Serial port. There is also a serial port connector with RX, TX, etc. This port helps you connect the adapter to an arduino or a micropython board that don’t have USB connections but have serial ports.

With the added features, comes added costs of parts, quality checking, and development times. So I am currently offering two-tier pricing:

  1. The board with everything included and tested at $55
  2. The board without the 12-pole terminal block or these pins tested at $45
  3. If you need the serial port, I can solder the header and configure it free of charge.
  4. The hi-res analog add-on board is $35 each. If you want the 4 additional SDI-12 port soldered on it, I can do it for $5 extra.

The full-featured board has the additional terminal block and needs to be tested with all the analog pins to make sure they are all properly connected (reflowed). In the photo, the bread board and 4 blue potentiometer is the test rig I use to test hi-res analog inputs for the SDI-12 + Analog USB adapter. You CAN solder on your own 12-pole terminal block and test the pins yourself too. You will have to do a lot of screw/unscrew of a potentiometer or resistors though. The firmware is the same so once you solder the connector on, you can use the features.

2018-07-02 14.05.33

Although the analog add-on + new adapter costs the same as the original SDI-12 Analog USB adapter, the stack of two boards does add to its height so the board needs more space. My intention is to add flexibility to the adapter so I can later add more features to the ecosystem without scraping the existing devices. I did a custom board for someone that wants magnetometer, accelerometer, and gyroscope with SDI-12 sensors:

2018-06-17 12.15.47.jpg

I glued the sensor board on top of the adapter board and wired them together via the extension header. This way I didn’t have to spend time designing a new board, which will likely cost more time and money. Also, designing a new extension board is easier than designing a whole new adapter with the sensors on it. I can do more custom sensor boards even if there is only market for a few.

The purchase links to the full-featured adapter and the hi-res add-on board have not been established yet. I’ll get them up and running on my blog and at inmojo soon.

Calibrate a magnetic sensor

I recently worked on a project that required a magnetic sensor (MPU-9250) be calibrated to get best accuracy. I had some basic understanding on how to calibrate a magnetic sensor, although I’ve not done calibration before. The calibration turned out to be a bit complicated and took a while to understand. I got some help, including method to get best calibration, from the author of the MPU9250 Arduino library, Kris Winer. I thought that if I shared my experience here, others that are planning to calibrate their magnetic sensors may find it useful.

First of all, what is calibration? In a general sense, calibrating a sensor makes the sensor provide the most accurate readings allowed by the sensor’s own precision. As an example, let’s assume for a moment that the earth’s magnetic field and any other stray magnetic fields are shielded and you have a uniform magnetic field generated artificially for the sole purpose of calibration. Let’s say that the field strength is 400 mG (milliGauss), equivalent to 40,000 nT (nanoTesla). Now if you align one axis of your magnetic sensor parallel to the direction of the field, it should read 400mG. If you then carefully rotate your sensor so that the axis is anti-parallel with your field, it will read -400mG. If you didn’t do a good job in either alignments, you will read less values, say 390mG, if you’re off by about 13 degrees, because only a portion of the field, which is a vector, is projected along your magnetic sensor’s axis.

In the diagram above, the thick blue arrows represent the constant magnetic field of 400mG pointing to the right. The thin arrows represent various orientations your sensor could take. If your sensor + axis is also pointing to the right, you get the full 400mG. If your sensor + axis points to the left, you get -400mG. If your sensor + axis makes an angle, it reads a projection of the field, which is less. You can figure out the angle:

The above was assuming that there IS a constant magnetic field and the sensor’s reading IS symmetric along its positive and negative axis, meaning with zero magnetic field, the sensor reads zero. When not calibrated, the sensor reading will NOT be zero under zero field. It could read say 10mG. As a result, you might get say 510mG and -490mG with the field on. You know what that means. There is an offset (bias) of 10mG that should be subtracted from your reading to get the correct reading of +-400mG.

The above was the basis for calibration to remove the offset (bias) on each axis. In order to get the maximal and minimal value, you need to write your code to store max/min out of a stream of live data while you rotate your sensor in space, trying to maximize or minimize the readout. Then repeat two more times for a 3-D magnetic sensor. Since you don’t have a magnetic shield, you are relying on the earth’s magnetic field as the constant field. The earth’s magnetic field is not horizontal, or pointing from north to south. In most areas, the field either has a vertical up component, or a down component. And in most cases the field points from south to north as the rotational north pole (AKA the north pole) is near the magnetic south pole, where field lines go in, not coming out. In my area for example, the earth magnetic field points primarily downwards, only slightly towards north, making an angle over 70 degrees with the horizontal. The magnetic field has very little component in the east direction. The relative strength between East, North, and Downward is about 1:64:195. The angle the magnetic field vector makes with the horizontal is called magnetic inclination, with downward being positive. This is approximately atan(195/64)=72 degrees. The angle the magnetic field vector’s horizontal component makes with the true north is called magnetic declination, with east being positive. This is approximately atan(1/64)=0.9 degrees. The properties of magnetic field varies greatly from place to place and also changes from time to time. To find out the magnetic field in your area, visit noaa.gov:

https://www.ngdc.noaa.gov/geomag-web/#igrfwmm

The following is from my area:

The next calibration is for sensitivity. The sensor either returns an analog voltage or a digital value. How do we convert this return into actual magnetic field in mG? This means finding the relation between the sensor readout and actual physical values. Say the sensor is digital and returns values between 0 and 32767, which represents magnetic field between 0 and 49150mG. Then you can use the conversion vactor 49150/32767=1.5mG/LSB to convert your readout. Here LSB means one digit (least significant bit). For an analog sensor, you will need an x.xx mG/V.

All sensors provide this factor in their spec sheets so you can just use this factor to get the actual magnetic field. But since not all sensors were made identical, some sensors should use larger or smaller values than the spec’s factor. Some manufacturers test their sensors at factory and store a correction factor for each axis in the sensor for better accuracy. For example, the MPU-9250 sensor (the magnetic sensor is AK8963) has digital magnetic field sensor output. One of the sensors I got has the following factory trims:

X-Axis sensitivity 1.18

Y-Axis sensitivity 1.19

Z-Axis sensitivity 1.14

So instead of a straight 1.5mG/LSB, the x-axis has 1.5*1.18=1.77mG/LSB. We’ll multiply this factor to the x-axis readout to get x magnetic field in mG. Same for y and z axes.

And yet sometimes these adjustments are still not able to make all 3 axes read the “400mG” value. They are very close to be identical already so we can apply a small correction. We average the maximal readings from all three axes, then divide by the maximal reading of each individual axis to get three factors. If say the x-axis reads slightly higher maximum than y and z axes. Then the avg_max/x_max will be slightly less than 1. We apply this factor to the the final result:

x_field=(1.5mG/LSB)*(x_sensitivity)*(avg_max/x_max)*x_readout

For one of my sensors, this yields 1.5*1.180*0.975*x_readout=1.726*x_readout(mG)

If the sensor you’re using doesn’t have the factory trim, then you’re out of luck unless you have both a magnetic shield and a nice uniform magnetic field inside the shield so you can find out the trim.

The following are steps for the AK8963 magnetic sensor calibration I did, using Kris Winer’s MPU9250 Arduino library:

  1. Extract factory sensitivity factor by calling initAK8963().
  2. Call readMagData() to extract raw data repeatedly enough to get accurate max/min for all axes.
  3. Calculate bias in raw counts by (max+min)/2, such as (510+ (-490))/2=10
  4. Combine factory sensitivity factor and Kris’s scale factor described above.
  5. Keep these biases and factors in program.

As a testimony to the method that works, here are two graphs.

This graph has three separate plots from raw data that represent mx vs. my, mx vs. mz, and my vs. mz. I rotated my sensor as much as I could for a few minutes before I got bored and couldn’t think of any other ways of rotation. The fact that all three plots are near circular means that the sensor’s three axes have very similar sensitivities. I was simply rotating the sensor around, giving all three axes opportunities to read the whole magnetic field. This means when the x axis reads the whole field, y and z axes read nothing. Some trigonometry can show that the result is a series or circles making a disc. But the discs were not centered as I expected, because the z axis had a very large bias (this means the blue is mx vs. my). After my bias calibration, here is what I got:

Now all three plots are centered at zero pretty well visually, although I didn’t multiple the factory sensitivity or the final factor from Kris’s calculation. This shows that bias calibration is the most crucial. If your sensor doesn’t have stored factory sensitivity factors, getting it calibrated for bias alone will go a long way.

FYI, this is my sensor board inside an enclosure. I found it much easier to hold and rotate when it’s in a box. The cable became much less of an issue when I was rotating the box. The black square board is my SDI-12 USB adapter. The purple board is the sensor board. I customized the SDI-12 USB adapter by gluing the sensor board to it and connecting it to the I2C bus on the adapter.

Second update on the new SDI-12 USB adapter

I found sometime to assemble a batch of the new boards. I populated the 12-pole terminal block on top and a row of headers on this one board for firmware development. Since not everyone will need these new features, the 12-pole block and the header for extension board or UART serial port will be optional and you can specify with your order that you need them. Adding these components adds more cost due to parts cost, assembly, and testing time. You could solder these headers yourself if you have some basic soldering skills. The UART serial port header is soldered on the underside of the board with a right-angle header to avoid the extension board and keep wires tidy.

If you need to use these boards over UART serial port such as connecting them to an Arduino or MicroPython board, please let me know with your order. I will place a solder blob between two pins on the USB serial IC so that it is placed in RESET mode to not interfere with UART serial communication with your microcontroller off board.

Here is the high-precision analog input extension board:

I assembled two extension boards, stacked them on top of the adapter, and set them to address 0 and address 1. These extension boards with come with a stacking header soldered on and four M3 standoffs, washers, and nuts. This ensures the proper spacing between boards to prevent short circuiting. I also need to trimming 20 pins on the underside of the board so that the underside of one extension board won’t touch the top side of another extension board below it. If you want, you can buy a set of 4 3-pole terminal blocks and populate them on the extension board to connect to more SDI-12 sensors, although I don’t recommend more than about 8 SDI-12 sensors from any vendor on the same adapter and extension board. A basic test running the SDI-12 + Analog USB adapter firmware on this adapter and extension board was successful, which was how I tested the extension board’s assembling quality.

My next steps are:

Extension board:

  • Expend the firmware to talk to as many as 4 such extension boards for a total of 16 high-precision analog inputs
  • Test address-setting jumpers (don’t expect any issues)
  • Populate SDI-12 headers on one extension board and test it (don’t expect any issues)

With one extension board and its address set to 0, getting high-precision analog readings is the same as using the SDI-12 + Analog USB adapter, by sending zM! and zM1! (differential reading), then using zD0! to retrieve data. With more extension boards, reading the 4 channels on board with address 1 will be zM2! and zM3! (differential reading) then the same zD0! to retrieve data. Board address 2 will have zM4! and zM5!, while board address 3 will have zM6! and zM7!. Then zM8! is reserved for the on-board basic analog channel read, while zM9! retrieves number of pulses from these channels.

Main adapter:

  • Develop firmware to read analog channels on the adapter itself (for basic analog signals at around 5mV precision).
  • Develop firmware to read pulses from the analog channels on the adapter itself (for rain gauges, flow meters etc. that output pulses).

Then I’ll test everything with a test rig. Stay tuned!

%d bloggers like this: