More about the data logger

Enclosed logger

Now that the logger prototype is enclosed, it is a lot easier to use. I have a METER group 5TM soil sensor connected to it from the left side and 12V power to it from the bottom. Without a heavy enclosure, these cables will make it hard to set the logger flat and connect an adapter to it. Plus, I’ve started using Telnet to connect to it wirelessly so I don’t need an adapter to connect to it most of the time.

Here is the adapter I modified to use on the logger:

2018-07-26 22.33.25

I bought it from moderndevices.com (Not shown on this photo) Later I had to short a resistor and a capacitor in order to actually make it work, due to the fact that the adapter was designed for Arduino boards.

Besides Telnet, I can also upload Python sketches via FTP. I later installed a couple of jumpers to easily reset the logger since I am still developing the logging script and need to reset the logger when I make changes. I’ll install a push button instead of the jumpers when I have some time.

Everything is coming along nicely. Soon I’ll start testing 4G LTE-M connectivity of the logger. 4G LTE-M has a lower bandwidth than the regular 4G LTE on smartphones so the modem would consume less power and data plans would cost less (in Megabytes instead of in Gigabytes). The Digi XBee3 LTE-M modem is rather expensive, at $100, or $150 for a dev kit. Anyway, if you are deploying the logger in a research field close to your lab/office or really don’t mind regularly visiting remote areas to collect data logs, you may skip the modem.

SDI-12 USB adapter upgraded

2018-07-02 15.25.47

After some more firmware development and testing, I am happy to announce that the upgraded SDI-12 USB adapter is now available. The above is the first batch of these adapters and one hi-res analog input add-on board.

The upgraded board features the following:

  1. 4 analog inputs. 12-pole terminal block that features 4 analog voltage inputs. These are 10-bit or 5mV resolution inputs without differential reading. There are there to provide basic voltage inputs for projects that don’t require hi-res analog voltage inputs.
  2. Pulse counters. Alternatively, these 4 inputs can be used as pulse counters. Say you have a rain gauge or flow meter that outputs pulses, these pins can count the pulses. You may need additional filtering (one capacitor and two resistors) if the pulses are noisy. Each time you read the pulse counts, you get the counts since you last read and the adapter will start counting from zero again. This way, if you collect data every minute, the counts will be counts/min. Because each data point is accompanied with date-time information, you can always calculate the count rate with your data set.
  3. Extension port. There is now an 8-pin extension port for add-on boards. The first extension board I have designed and tested is a hi-res analog voltage input board. This board features the same four 16-bit auto-scaling inputs and differential inputs as the SDI-12 + Analog USB adapter, with an added benefit of address jumper. You can add as many as 4 such extension boards to the new SDI-12 USB adapter, with each extension board taking a different address. That is up to 16 hi-res voltage inputs.
  4. Serial port. There is also a serial port connector with RX, TX, etc. This port helps you connect the adapter to an arduino or a micropython board that don’t have USB connections but have serial ports.

With the added features, comes added costs of parts, quality checking, and development times. So I am currently offering two-tier pricing:

  1. The board with everything included and tested at $55
  2. The board without the 12-pole terminal block or these pins tested at $45
  3. If you need the serial port, I can solder the header and configure it free of charge.
  4. The hi-res analog add-on board is $35 each. If you want the 4 additional SDI-12 port soldered on it, I can do it for $5 extra.

The full-featured board has the additional terminal block and needs to be tested with all the analog pins to make sure they are all properly connected (reflowed). In the photo, the bread board and 4 blue potentiometer is the test rig I use to test hi-res analog inputs for the SDI-12 + Analog USB adapter. You CAN solder on your own 12-pole terminal block and test the pins yourself too. You will have to do a lot of screw/unscrew of a potentiometer or resistors though. The firmware is the same so once you solder the connector on, you can use the features.

2018-07-02 14.05.33

Although the analog add-on + new adapter costs the same as the original SDI-12 Analog USB adapter, the stack of two boards does add to its height so the board needs more space. My intention is to add flexibility to the adapter so I can later add more features to the ecosystem without scraping the existing devices. I did a custom board for someone that wants magnetometer, accelerometer, and gyroscope with SDI-12 sensors:

2018-06-17 12.15.47.jpg

I glued the sensor board on top of the adapter board and wired them together via the extension header. This way I didn’t have to spend time designing a new board, which will likely cost more time and money. Also, designing a new extension board is easier than designing a whole new adapter with the sensors on it. I can do more custom sensor boards even if there is only market for a few.

The purchase links to the full-featured adapter and the hi-res add-on board have not been established yet. I’ll get them up and running on my blog and at inmojo soon.

Second update on the new SDI-12 USB adapter

I found sometime to assemble a batch of the new boards. I populated the 12-pole terminal block on top and a row of headers on this one board for firmware development. Since not everyone will need these new features, the 12-pole block and the header for extension board or UART serial port will be optional and you can specify with your order that you need them. Adding these components adds more cost due to parts cost, assembly, and testing time. You could solder these headers yourself if you have some basic soldering skills. The UART serial port header is soldered on the underside of the board with a right-angle header to avoid the extension board and keep wires tidy.

If you need to use these boards over UART serial port such as connecting them to an Arduino or MicroPython board, please let me know with your order. I will place a solder blob between two pins on the USB serial IC so that it is placed in RESET mode to not interfere with UART serial communication with your microcontroller off board.

Here is the high-precision analog input extension board:

I assembled two extension boards, stacked them on top of the adapter, and set them to address 0 and address 1. These extension boards with come with a stacking header soldered on and four M3 standoffs, washers, and nuts. This ensures the proper spacing between boards to prevent short circuiting. I also need to trimming 20 pins on the underside of the board so that the underside of one extension board won’t touch the top side of another extension board below it. If you want, you can buy a set of 4 3-pole terminal blocks and populate them on the extension board to connect to more SDI-12 sensors, although I don’t recommend more than about 8 SDI-12 sensors from any vendor on the same adapter and extension board. A basic test running the SDI-12 + Analog USB adapter firmware on this adapter and extension board was successful, which was how I tested the extension board’s assembling quality.

My next steps are:

Extension board:

  • Expend the firmware to talk to as many as 4 such extension boards for a total of 16 high-precision analog inputs
  • Test address-setting jumpers (don’t expect any issues)
  • Populate SDI-12 headers on one extension board and test it (don’t expect any issues)

With one extension board and its address set to 0, getting high-precision analog readings is the same as using the SDI-12 + Analog USB adapter, by sending zM! and zM1! (differential reading), then using zD0! to retrieve data. With more extension boards, reading the 4 channels on board with address 1 will be zM2! and zM3! (differential reading) then the same zD0! to retrieve data. Board address 2 will have zM4! and zM5!, while board address 3 will have zM6! and zM7!. Then zM8! is reserved for the on-board basic analog channel read, while zM9! retrieves number of pulses from these channels.

Main adapter:

  • Develop firmware to read analog channels on the adapter itself (for basic analog signals at around 5mV precision).
  • Develop firmware to read pulses from the analog channels on the adapter itself (for rain gauges, flow meters etc. that output pulses).

Then I’ll test everything with a test rig. Stay tuned!

Upgrades to the SDI-12 USB adapter

I have been working on some updates to the SDI-12 USB adapter so that it would add more features to a data logging system. So far, I’ve updated the PCB (left board) to include additional connectors. The top of the board will have 4 analog channels. This is not as accurate as the red SDI-12 + Analog USB adapter boards. The SDI-12 + Analog USB adapter has practical accuracy of 20 microvolts and has differential input channels. The 4 channels on the basic SDI-12 USB adapter have accuracy of about 5 millivolts. Also there is not a voltage reference so the measurement will be affected by the USB voltage, which is only nominally 5V. Nevertheless, if there are some sensors that output voltages in 0-5V range you want to log with moderate accuracy, such as a potentiometer, or a thermistor for approximate temperature calculation, you can use these channels. The breakout looks the same as the SDI-12 + Analog USB adapter. There are no serial resistors so you have to add yours if you want to convert resistance to voltage.

I will release a new firmware version on these newer boards. At the same time, I am considering adding digital counting features to these analog channels so if someone wants to count pulses such as flow meters or rain gauges, they can use these channels for such purpose. I plan to develop this part in the summer.

Another connector (bottom one on left board) I have added will connect the adapter to an analog extension board (right board), which sports the same 20 microvolt accuracy as the SDI-12 + Analog USB adapter, in case you want to add these channels say for pyronometers or other low voltage and high precision measurements after initially getting the basic adapter. You can stack up to 4 such analog extension boards. Each board has an address jumper (right board, white rectangle) for one of the four addresses the analog-to-digital converter supports. That gives you a total of 16 high-precision analog input channels. Each extension board also comes with a few additional connectors for SDI-12 sensors as an option. You can more easily wire up more SDI-12 sensors to the adapter. I don’t recommend wiring up more than 6-8 SDI-12 sensors to the same adapter. Some sensors come with strong pull-down resistors. When too many of them are wired to the same adapter, they may prevent some other sensors from correctly communicating on the bus.

The last connector (left board middle) I have added will help developers using MicroPython platforms easily connect to it via serial ports, since most MicroPython boards don’t have USB hosts. I will start shipping these newer SDI-12 USB adapter boards soon although new firmware that makes use of these features will have to wait until later.

Will this affect your existing projects? Very unlikely. The new adapter has all the features of the old adapter. The SDI-12 + Analog adapter will still be around since it is a nice compact form factor. The new SDI-12 adapter plus the analog extension board will be approximately the same price as as the SDI-12 + Analog adapter.

More SDI-12 sensors tested with the adapters

As more researchers and developers are using my SDI-12 USB adapters, more sensors have been tested to run with the adapter. Here are some of the new additions recently:

Gill Instruments:

  • WindSonic Ultrasonic Wind Sensor (Thanks MG!)

HSTI:

  • HydraSCOUT multi-sensor soil moisture and temperature probe (Thanks Bertrand!)

Sutron:

  • Accubar SDI-12 Barometric Pressure Sensor, Model 5600-0120-3 (Thanks Meidad!)

Unidata:

  • Starflow QSD Ultrasonic Doppler Velocity And Depth sensor (Thanks Yiren!)

Since Decagon has merged with a German company UMS into METER Group, some of their product lines are renamed and other products are added to these new lines. Here is a list of tested sensors from them:

METER group (formally Decagon and UMS):

  • Atmos 22 (DS-2) (Sonic Anemometer)
  • Atmos 14 (VP-4) (temperature, vapor pressure, relative humidity sensor)
  • Atmos 41 weather station (solar radiation, precipitation, vapor pressure, relative humidity, air temperature, barometric pressor, horizontal wind speed, wind gust, wind direction, compass heading, tilt, lightning strike count, lightning average distance) (Thanks CD!)
  • PRI (spectral reflectance sensor)
  • NDVI SRS (spectral reflectance sensor)
  • GS3 (Ruggedized Soil Moisture, Temperature, and Electrical Conductivity Sensor)
  • TEROS 21 (MPS-6) (Calibrated Water Potential Sensor)
  • ECH2O 5TE (5TE) (Volumetric Water Content, Electrical Conductivity, and Temperature)
  • ECH2O 5TM (5TM) (Soil Moisture and Temperature Sensor)

If you are using the SDI-12 USB adapters, I’d love to include your SDI-12 sensors on my list of tested sensors. Leave me a message.

Data logging with Wipy and SDI-12 dongle

If you have heard of the Python programming language and its easy-to-learn and easy-to-use fyou’ll be pleasantly surprised that someone has successfully implemented Python on microcontrollers a few years ago, appropriately named MicroPython! He has developed his own MicroPython board and ported the code to a number of similar microcontrollers. Imagine a low-power microcontroller with “lots” of memory (compared with Arduino) happily running Python code that talks to the internet etc. I’ll start writing about MicroPython and how you may use it for DIY electronics and data logging in a number of posts but this post is an announcement related to MicroPython and the SDI-12 serial dongle:

This dongle has a similar set of features as the SDI-12 USB adapters but lacks the USB connectivity, just having serial connection to arduino. Since all MicroPython compatible boards have serial ports and library to run the port, you can connect a MicroPython compatible board to one of these dongles and run almost the same Python data logging code I provide to SDI-12 USB adapters. Here is the first successful attempt that made it happen. Jason is a developer on the Wipy platform, which is a microcontroller supporting MicroPython. He is developing a data logger. In order to talk to SDI-12 sensors, he got one of my SDI-12 dongles. The Wipy board has serial ports but at 3.3V logic. The dongle has 5V logic. I built a simple voltage divider on the dongle for him and he was able to communicate with it using the Wipy board. Here is the code that he wishes to share:

Wipy_logger.py

This script runs on Wipy (or similar MicroPython platforms) and logs data from SDI-12 sensors to the on-board SD card. Here is what a Wipy board V 3.0 looks:

SDI-12 USB adapter and manual updates

If you have been looking at the SDI-12 USB adapters lately, you may have noticed that the manual got updated a bunch of times between February and March (latest version being 3/19/2018). I’ve been updating the manual to:

1. Introduce the newer SDI-12 USB adapter (black) that replaced the original adapter (green).

Original SDI-12 USB adapter since 2015:

slide2

Updated SDI-12 USB adapter:

SDI-12 USB + adapter

Notice that now the updated adapter is the same size as the other flavors of the adapters, such as the SDI-12 + Analog adapter and the SDI-12 + GPS adapter. These are the features the updated adapter has:

  • Four SDI-12 sensor connectors
  • External power connection and sensor power selector (5V USB or external)

2. I updated several sections of the manual to make the manual easier to understand.

  • Bookmarks in the .PDF file
  • More detailed description of the updated adapter
  • Details of how to configure an SDI-12 adapter
  • Other places have been tidied up as well

By the way, did I mention that I reduced all my adapters by $5 to $10?

  • SDI-12 USB adapter (updated) $45 ($5 cheaper)
  • SDI-12 + GPS USB adapter $60 ($10 cheaper)
  • SDI-12 + Analog USB adapter $80 ($10 cheaper)
  • There will be discounts for bulk purchases, such as 10, 20, 50 etc. Please contact me if you intend to buy more adapters so I can send you an accurate quote of prices with discounts and shipping cost.
  • If you still need the green adapter for its simplicity or form factor, you can contact me to do bulk orders, 10 adapters or more.
  • If you need something else to be on your adapter to connect to a particular sensor, contact me to see how I can help you achieve that. The sensor would definitely not be SDI-12 but I’ll make it so that reading from it is just like reading from any other SDI-12 sensor so your program needs little to no change.

Since I am assembling and testing these adapters myself and use trustworthy US parts vendors, I will have to get more efficient in my work to make up the difference. I made some improvements on a reflow oven that I will be using for new batches of adapters. I hope the effort pays off. I would be able to assemble a batch, load into the reflow oven, and assemble the next batch or do cleanup or soldering thru-hole components such as the screw terminal blocks while the oven reflows automatically. I’ve been using a manual control (Variac transformer) for reflow for the past several years, which requires constant attention.

One negative impact from the US postal service, since the end of January 2018, USPS will no longer ship merchandise via international first-class letter. The only alternative is first-class package, which is a much more expensive service. A package containing one adapter that used to cost less than $4 will now cost $9 to ship to Canada (come on!) or $14 to most of the rest of the world. I felt that I’m being squeezed out of the international market by the postal service. The only upside is that now the package seems to come with complete tracking in both countries. I made a small change to the first-item shipping cost and additional item shipping cost.

I am also considering designing a complete low-cost data-logging system so it would be more integrated than the adapters. The adapters have proved to be very successful indeed. It shows up as number-1 search for keywords “SDI-12 adapter” or “SDI-12 USB adapter”. I didn’t pay google anything for the free advertisement. Everyone that searched and read my blog helped spread the word! Thank you!

I plan to keep improving the adapters so that anyone that is integrating SDI-12 sensors will find it easy to just get an adapter and add SDI-12 sensors. For anyone else that is interested in low-cost data logging solutions but are more interested in a turn-key solution instead of investing time and effort in learning python programming or raspberry pi, this logger will make things easy for you!

Any comments, suggestions, your use case, including what sensors and telemetry solutions you want that you would like to share? Leave a comment! The design of the data logger is still very fluid so that your opinions WILL influence its development!

The development of a turn-key solution WILL positively affect the adapter development as well. I expect to have a 3-prone approach ultimately: USB adapters to add SDI-12 sensors to existing logger projects that use computers/raspberry pi, serial dongles/shields to do the same for Arduino-based existing logger projects, and a complete logger for those seeking turn-key solutions.

Have a nice spring/fall day!

SDI-12 USB adapter manual and logging script updated

Due to the discontinuation of data.sparkfun.com, I moved data logging to ThingSpeak.com

I have other updates that I rolled up in the manual, such as more details on telemetry. New manual is posted on SDI-12 USB adapter page as well as the updated data logging code. Here is a snapshot of data I logged to ThingSpeak.com:

The full data stream is here:

https://thingspeak.com/channels/359964

SDI-12 + GPS USB adapter

After a final revision, I am happy to release the SDI-12 GPS USB adapter! This adapter is the latest one to add to the line of SDI-12 USB adapters. In August 2015, I released my first SDI-12 USB adapter with this post. It was an idea that I thought about while traveling. I was working on data logger designs that use SDI-12 sensors and felt that interacting with SDI-12 sensors is not easy for agricultural or water resource researchers. Having an adapter that connects a computer to an SDI-12 sensor and reads measurements directly from the sensor would be very useful. So I made the adapter to simplify lab tests and data logger deployments. Since then, I’ve written free Python scripts for basic data logging (read the SDI-12 USB adapter main page). The demand for the adapter since then has been high enough to support my continued update on the data logging script, expanding from PC/Mac/Linux to single-board computers such as Raspberry Pi and Beagle Bone Bone. I have also expanded the adapter with an SDI-12 + Analog USB adapter that includes four high-precision analog inputs.

Later I found some need to add GPS modules to the existing SDI-12 USB adapter so that mobile data loggers such as those mounted on tractors will be able to produce with Geo-tagged data that can be made into maps. After some initial struggle using the new ATMEGA328PB processor that sports two hardware serial ports (one to talk to PC and the other with GPS), I realized that the GPS module actually interfered with the processor and caused program freeze-up. Then I made some hardware revisions and was able to prevent interference. It turned out that the new ATMEGA328PB processor that I used in my initial prototype was especially susceptible to interference when I used its second hardware serial port that have the same pins as the SPI pins that program the processor. So I switched to the ATMEGA1284P processor that I have been using on my open source physics laboratory design.

After extensive tests, I am happy to add this adapter to the product line. You can purchase (small quantity at the moment) at inmojo.com or on my blog (in the middle of the page). The adapter requires a separate purchase of the GPS module that Adafruit makes and sells, the Ultimate GPS module part number 746. You only need to solder four pins on the GPS module, the TX, RX, GND, and VIN, and the same pins on the adapter. Since the GPS module is relatively expensive, I can’t stock them up. But if you really need it assembled, you may have a GPS unit sent to me and a few extra dollars for assembly and testing. Just contact me once you make a purchase if you want assembly.

Adding Beagle bone to the mix

I was recently contacted by someone who was interested in using the SDI-12 USB adapter on a Beagle Bone Black single board computer. I’ve never used a Beagle Boards but I know that they are ARM-based computers running linux thus should operate similarly to the Raspberry pi boards that I’ve been playing since 2012. So I took the dive and got a Beagle Bone Black from MCM electronics and gave it a try. Right out of the box the board boots into a version of Linux. I was able to test its connectivity with the SDI-12 USB adapter successfully using the “screen” command. Later I ran a simple Python script under Python 2.7 and got very nice results:

There are a few differences that I noticed while exploring BBB:

  1. There is a “serial” module included in Python that is not available on other platforms, such as windows, linux, Mac OS, or Raspberry pi. It functions like the pyserial module used on all these systems.
  2. The board boots much faster than raspberry pi 3B, maybe in 15 seconds. RPI 3B takes about 30 seconds. This is a good thing.
  3. There are a lot fewer instructions on basic operations for Beagle boards than Raspberry pi, which was the primary reason I got my raspberry pi B instead of Beagle board back in 2012.

When I have more time, I will test my open-source python data logger on BBB to make sure it works just as it does on all other systems. For now, one more box is checked: “compatible with Beagle Bone Black”.

 

%d bloggers like this: