SDI-12 USB adapter manual and logging script updated

Due to the discontinuation of data.sparkfun.com, I moved data logging to ThingSpeak.com

I have other updates that I rolled up in the manual, such as more details on telemetry. New manual is posted on SDI-12 USB adapter page as well as the updated data logging code. Here is a snapshot of data I logged to ThingSpeak.com:

The full data stream is here:

https://thingspeak.com/channels/359964

Upload data to ThingSpeak

I have been using sparkfun’s Phant server for data upload and retrieval for a few years since they started the service. The service was easy to use and was free. Several months back they discontinued the service, unfortunately. I started looking for suitable alternatives.

Sparkfun recommended three, ThingSpeak, Cayenne, and Blynk. I went ahead and did some research on these services. My goal is to be able to log data online and later retrieve them and possibly visualize them using google charts, like before. I am not at the moment interested in automating my home with actuators or smart phone apps to turn on and off my hall lights. Here are my findings and why I decided that ThingSpeak was the best fit. If you are logging data for later processing or visualization, read on.

ThingSpeak.com

This service is provided by the company behind Matlab. I am not a fan of expensive commercial data manipulation tools such as Matlab but they do have a fair amount of business between universities and industry so their service might be a safer way to go against sudden discontinuation of service such as sparkfun. Basically you create a data stream and send data to the stream. It’s very similar to sparkfun. You can also retrieve your data, possibly good for running your data through google chart. They also provide some basic graphs and matlab analysis tools that I have yet tried.

There are two types of application programming interfaces (APIs) you can use: a REST API, and an MQTT API.

The REST API is based on HTTP so it’s very similar to existing services elsewhere. You use HTTP GET or POST command to send data using an API key, like a private key with sparkfun. Your data are limited to up eight values per data point. If you need more, then you need to create more streams. They also have a bulk update feature that you can use to send multiple sets of data instead of one set of data. This method allows a device to collect data and sleep in between data points. Then when it collects a fair amount of data, it connects to the Internet and sends all data in one shot. It saves power and network bandwidth. You can also create and modify the properties of streams with this API.

With the MQTT API, the underlying protocol is TCP/IP. There is no acknowledgement of data received and it is intended for low-powered devices to just wake up, take data, send it out, and go back to sleep. From my tests, data sent via this method were lost over 50% of the time. Unless future holds differently, I am not recommending this API.

Getting start is easy with ThingSpeak. Just set up an account and follow their tutorial to create a new data stream. Then the following bash code should get you started posting code:


curl "https://api.thingspeak.com/update/?api_key=your_appkey&field1=1.23&field2=2.34"

You can post any number of field values between 1 and 8. You will receive a zero as a positive response. Then you will see your results like the plot on the top of this page.

I have updated my Python data logging code to use ThingSpeak.com instead of the now discontinued data.sparkfun.com. The plot in this post is from Thingspeak.com. Here is a link to the data stream:

https://thingspeak.com/channels/359964

Can’t upgrade pyserial in latest raspbian distribution?

This is just for your information if you are a Raspberry Pi user and playing with Python code from my blog. If you are trying to use the latest distro of raspbian with pyserial for some serial port project, you may have come across this issue that regardless how you upgrade pyserial using pip3, your python3 will always call up the old pyserial 2.6 that came with the distribution. I am a bit disappointed that the foundation has included such an old version of pyserial, couldn’t they just try a pyserial 3.0 instead? My solution was to remove the python3-serial module using apt-get and then install pyserial 3.3 using pip3.

sudo apt-get remove python3-serial
sudo pip3 install pyserial

Hope this helps.

Augmented reality sand box

I have been involved in constructing augmented reality sand box (ARSandbox) lately. It is a beautiful project created by Dr. Oliver Kreylos at UC Davis. The system uses a Microsoft XBox 360 Kinect sensor to digitize the sand in a box and then uses a projector to project color-coded elevation and contours on the sand, thus augmenting the sand with colors and contours. Here is a photo I took on a prototype that we replicated from the ARSandbox created by Dr. Kreylos:

When someone manipulates the sand, thus changing the topography, the projected colors and contours change accordingly. You can also rain over the terrain by a hand gesture over the terrain.

For those that didn’t know, the Microsoft XBox 360 Kinect sensor is a sophiscated set of sensors that include IR projector and camera for depth sensing, body movement and gesture capture, and regular RGB camera and microphone array. The software Dr. Kreylos developed takes the depth image and calculates a topography map and projects it onto the very same sand using a calibrated projector. To show you how good the simulation is, here is a photo:

Did you notice the white cone and the colors/contours on it? The contours are depicting 1cm heights and the cone is about 4cm tall. Here is a close-up of the cone placed at a different location:

It shows roughly 4cm tall and the contours are very well centered around the tip of the cone.

Here is a video:

Like it? The whole setup is not cheap. It needs an expensive video card for the simulation, especially the water. It also needs a decent desktop computer and projector, sand box, frames etc. Here is what my setup looks like:

I didn’t buy a more expensive (thus shorter throw ratio) projector so my setup is very tall even without any legs. I am hoping to develop it into a portable system so I can take a few of them to teachers’ training workshops, museums, schools, fairs etc. for basic education and outreach for water resources.

What I’m thinking about doing using my Arduino/Raspberry pi skills is to add sensors to help preserve the projector’s bulb and have kids and operators interact with the sand box without having to use the keyboard and mouse or understanding linux. Big buttons will do certain predefined things such as rain, drought, etc.

SDI-12 + GPS USB adapter

After a final revision, I am happy to release the SDI-12 GPS USB adapter! This adapter is the latest one to add to the line of SDI-12 USB adapters. In August 2015, I released my first SDI-12 USB adapter with this post. It was an idea that I thought about while traveling. I was working on data logger designs that use SDI-12 sensors and felt that interacting with SDI-12 sensors is not easy for agricultural or water resource researchers. Having an adapter that connects a computer to an SDI-12 sensor and reads measurements directly from the sensor would be very useful. So I made the adapter to simplify lab tests and data logger deployments. Since then, I’ve written free Python scripts for basic data logging (read the SDI-12 USB adapter main page). The demand for the adapter since then has been high enough to support my continued update on the data logging script, expanding from PC/Mac/Linux to single-board computers such as Raspberry Pi and Beagle Bone Bone. I have also expanded the adapter with an SDI-12 + Analog USB adapter that includes four high-precision analog inputs.

Later I found some need to add GPS modules to the existing SDI-12 USB adapter so that mobile data loggers such as those mounted on tractors will be able to produce with Geo-tagged data that can be made into maps. After some initial struggle using the new ATMEGA328PB processor that sports two hardware serial ports (one to talk to PC and the other with GPS), I realized that the GPS module actually interfered with the processor and caused program freeze-up. Then I made some hardware revisions and was able to prevent interference. It turned out that the new ATMEGA328PB processor that I used in my initial prototype was especially susceptible to interference when I used its second hardware serial port that have the same pins as the SPI pins that program the processor. So I switched to the ATMEGA1284P processor that I have been using on my open source physics laboratory design.

After extensive tests, I am happy to add this adapter to the product line. You can purchase (small quantity at the moment) at inmojo.com or on my blog (in the middle of the page). The adapter requires a separate purchase of the GPS module that Adafruit makes and sells, the Ultimate GPS module part number 746. You only need to solder four pins on the GPS module, the TX, RX, GND, and VIN, and the same pins on the adapter. Since the GPS module is relatively expensive, I can’t stock them up. But if you really need it assembled, you may have a GPS unit sent to me and a few extra dollars for assembly and testing. Just contact me once you make a purchase if you want assembly.

SDI-12 data logging on Beagle Bone Black

In my previous post, I wrote about my initial success reading SDI-12 sensors using my SDI-12 USB adapter and a Beagle Bone Black (BBB), via a simple linux command “screen”. Upon further testing, I had trouble running my open-source python data logging script on BBB, because a number of python modules including pyserial and urllib3 are missing and I couldn’t install them the way I used to do on a Raspberry Pi. So after searching online for the past few days, I finally found how to solve the problem and here is a screen shot of the data logging script happily logging data on BBB:

BBB_logger_screenshot

The SDI-12 sensor I was using was the GPS sensor from my newest SDI-12 + GPS USB adapter. It has an on-board GPS module (separate purchase from Adafruit and its vendors). I was logging coordinates.

Here are the steps to prepare your BBB to run the data logging script:

  1. Install pip for python3: sudo apt-get install python3-pip
  2. Upgrade pip: sudo pip3 install –upgrade pip
  3. Install pyserial module: sudo pip3 install pyserial
  4. Install dropbox module: sudo pip3 install dropbox

The reason that I installed dropbox is because that I can use it to send data files from remote logger to my desktop, and installing it upgrades urllib3, which I use to generate URI-safe address for sending data to sparkfun’s phant server. If you are not doing telemetry or you prefer logging into your BBB to get your data files, you don’t have to install dropbox or upgrade urllib3. Now all I have to do is to update my documentation and welcome BBB to the club!

Adding Beagle bone to the mix

I was recently contacted by someone who was interested in using the SDI-12 USB adapter on a Beagle Bone Black single board computer. I’ve never used a Beagle Boards but I know that they are ARM-based computers running linux thus should operate similarly to the Raspberry pi boards that I’ve been playing since 2012. So I took the dive and got a Beagle Bone Black from MCM electronics and gave it a try. Right out of the box the board boots into a version of Linux. I was able to test its connectivity with the SDI-12 USB adapter successfully using the “screen” command. Later I ran a simple Python script under Python 2.7 and got very nice results:

There are a few differences that I noticed while exploring BBB:

  1. There is a “serial” module included in Python that is not available on other platforms, such as windows, linux, Mac OS, or Raspberry pi. It functions like the pyserial module used on all these systems.
  2. The board boots much faster than raspberry pi 3B, maybe in 15 seconds. RPI 3B takes about 30 seconds. This is a good thing.
  3. There are a lot fewer instructions on basic operations for Beagle boards than Raspberry pi, which was the primary reason I got my raspberry pi B instead of Beagle board back in 2012.

When I have more time, I will test my open-source python data logger on BBB to make sure it works just as it does on all other systems. For now, one more box is checked: “compatible with Beagle Bone Black”.

 

Ordering PCB from seeedstudio.com

This is an updated short tutorial on how to order your PCB from seeedstudio.com

I am not associated with seeedstudio but just a satisfied customer. They have competitive pricing and decent quality. They also recently automated design file submission process, although there is still some kinks.

Let’s assume that you have finished designing your PCB with EAGLE CAD, you should use seeedstudio’s CAM file to export your design into several files. The CAM exports many files but only the following are useful and should be zipped in a .zip file, per their instruction (http://support.seeedstudio.com/knowledgebase/articles/1176532-how-to-generate-gerber-file):

Copper traces:

  • Top Layer: pcb_name.GTL
  • Bottom Layer: pcb_name.GBL

Solder masks to apply solder resist around pads and pins so solder bridges don’t form and short circuits:

  • Solder Mask Top: pcb_name.GTS
  • Solder Mask Bottom: pcb_name.GBS

White text information for human assemblers and end users to read

  • Silk screen Top: pcb_name.GTO
  • Silk screen Bottom: pcb_name.GBO

Drill file for drilling pads and vias:

  • Drill Drawing: pcbname.TXT

The boarder of the circuit:

  • pcb_name.GML

Once you have a .zip file, direct your web browser to https://www.seeedstudio.com/fusion_pcb.html  and you will see their PCB printing service:

Seeedstudio PCB

There are a lot of options on this page. I will list what they are below:

Seeedstudio PCB options

  • Add your gerber file: Pick the .zip file that contains your design. You may examine your design after the upload. As of the writing, they are experimenting with a new gerber viewer that has some bugs. For instance, a board I designed 178mm wide only shows about 155mm of area. If I view with the old gerber viewer, everything shows up.
  • Material: typical boards are made of fire-retardant materials (default FR-4)
  • Layer: You will pick 2 layers. This means you will have circuits both on top and bottom of your circuit boards. I don’t see many single layer PCB any more but they were popular decades ago.
  • PCB Dimension: This is the rectangular size that encloses your PCB. Typical sizes in their price tiers are 5cm*5cm (about 2″*2″) for $10 (10 boards), and 10cm*10cm (about 4″*4″) for $25 (10 boards) although they usually run discounts on them. In the past, the board dimensions are taken automatically from your PCB design files but not anymore. You have to supply it yourself now. But you also get a “gerber viewer” after you upload your design.
  • PCB Qty: Seeedstudio PCB service doesn’t make one PCB, the least amount is 5 although 10 usually has the deepest discounts. Go ahead and pick 10, unless you want 50 or 100. Sometimes if I want 20 and the price of 20 doesn’t make sense, I would just go with 10 and then add another PCB design, which is the same as the first 10 and just order another 10.
  • Different design(s): you don’t want to change it to anything other than 1.
  • PCB Thickness: Go ahead and pick 1.6mm. Most common thickness is 1.6mm. If you want some added mechanical strength on your larger boards, you can go with 2.0mm.
  • PCB Color: Green is the default and other colors used to cost $10 extra. Now with smaller 10 QTY board all color cost the same, more or less. Your PCB will be painted with color paint although text is always white. I’ve never tried white or yellow but I assume the text may be black. I’ve used green, red, blue, and black. They all come out fine.
  • Surface Finish: Pick HASL (Hot Air Surface Leveling). It’s a technique that plates holes with hot molten solder and the excess is removed by hot air. You can also pick lead-free HASL if you want to go ROHS. You may upgrade to ENIG, which is Electroless Nickel Immersion Gold. Holes will be covered by a thin layer of gold to prevent oxidation.
  • Min Solder Mask Dam: I don’t know this enough so I’m going with default 0.4mm.
  • Copper Weight: this is the weight of copper clad per square foot. 1oz is a typical choice. Thicker copper clad offers less resistance with the same trace size. It’s simple physics.
  • Min Hole Size: this is the least drill size. 0.3mm is good enough. We are talking about very small via holes. Only multi-layer boards with a large IC that has lots of pins or ball grid array ICs will need more vias and smaller vias.
  • Min Tracking/Spacing: this is the thinnest copper trace width. 6mil is good enough. Unless you have multi-layer board and ICs with hundreds of pins, you don’t need any thinner traces.
  • Blind Vias: oh my, just go with default NO.
  • Half-cut: go with NO. Never done it.
  • Impedance Control: unless you have specifically designed your board to control impedance of your traces, go with NO.

After everything is selected and the service is added to cart, I would proceed to check out. You need to set up an account with seeedstudio and have a paypal account. Regular shipping is slow and not that cheap. I average about 3 weeks between design submission and delivery (to Mid-west, USA). You can also pay DHL and save maybe a week or more, although I never used it. I guess that’s it.

Free assistance on data logger projects

Summer is finally coming to my backyard and my spring semester is coming to an end. Thinking ahead (skipping over all the final papers to grade), with the whole summer ahead of me, starting 5/15/17, I can provide some free assistance to those that are working on your data logger projects using my devices, such as the SDI-12 data logging shield and SDI-12 USB adapters.

My goal is to get you started so you can quickly work on your own after my help. I’ve used Teamviewer to remotely help people install software, test their adapters with their own sensors, and modified my Python data logging code in the past. As long as I have some time to spare, I am willing to keep providing help. I appreciate it if you could help me spread the word. I might ask you to provide a blurb such as what sensors you use and what type of project you are working on etc. as a form of exchange for my free help.

Temporary connections?

I bet once or twice in the past you wish that you could find a simple way to just hold a module with male headers like this Adafruit GPS module in the pin holes of a circuit board without soldering the pins so you can later take the module and put it elsewhere. Here I found a neat solution, wedging the module with some ESD foam. You get ESD foam pieces from buying ICs so they come for free and have some springy-ness. Then you put in between the module and the board and push in. The force from the foam will make the module form a wedge and there is enough strength to tlit the pins against the holes to make connection. Here are some photos:

The first photo shows the underside of my new phi-3 shield. The GPS module is supposed to be on the top side but unless I solder it to the shield, there is not enough space to make a wedge with the module while it is under the display. So I hang it below the shield board and stuck a piece of ESD foam between the module and the shield board. Here is a side view. You can’t see the foam, which is too dark but you do see the pins are all tilted and thus pushed hard enough to make electrical connection. I only recommend this on a temporary basis. Tomorrow the GPS module is going on another board, the SDI-12 + GPS USB adapter. I’ll solder this time.

%d bloggers like this: