## Calibrate a magnetic sensor

I recently worked on a project that required a magnetic sensor (MPU-9250) be calibrated to get best accuracy. I had some basic understanding on how to calibrate a magnetic sensor, although I’ve not done calibration before. The calibration turned out to be a bit complicated and took a while to understand. I got some help, including method to get best calibration, from the author of the MPU9250 Arduino library, Kris Winer. I thought that if I shared my experience here, others that are planning to calibrate their magnetic sensors may find it useful.

First of all, what is calibration? In a general sense, calibrating a sensor makes the sensor provide the most accurate readings allowed by the sensor’s own precision. As an example, let’s assume for a moment that the earth’s magnetic field and any other stray magnetic fields are shielded and you have a uniform magnetic field generated artificially for the sole purpose of calibration. Let’s say that the field strength is 400 mG (milliGauss), equivalent to 40,000 nT (nanoTesla). Now if you align one axis of your magnetic sensor parallel to the direction of the field, it should read 400mG. If you then carefully rotate your sensor so that the axis is anti-parallel with your field, it will read -400mG. If you didn’t do a good job in either alignments, you will read less values, say 390mG, if you’re off by about 13 degrees, because only a portion of the field, which is a vector, is projected along your magnetic sensor’s axis.

In the diagram above, the thick blue arrows represent the constant magnetic field of 400mG pointing to the right. The thin arrows represent various orientations your sensor could take. If your sensor + axis is also pointing to the right, you get the full 400mG. If your sensor + axis points to the left, you get -400mG. If your sensor + axis makes an angle, it reads a projection of the field, which is less. You can figure out the angle:

The above was assuming that there IS a constant magnetic field and the sensor’s reading IS symmetric along its positive and negative axis, meaning with zero magnetic field, the sensor reads zero. When not calibrated, the sensor reading will NOT be zero under zero field. It could read say 10mG. As a result, you might get say 510mG and -490mG with the field on. You know what that means. There is an offset (bias) of 10mG that should be subtracted from your reading to get the correct reading of +-400mG.

The above was the basis for calibration to remove the offset (bias) on each axis. In order to get the maximal and minimal value, you need to write your code to store max/min out of a stream of live data while you rotate your sensor in space, trying to maximize or minimize the readout. Then repeat two more times for a 3-D magnetic sensor. Since you don’t have a magnetic shield, you are relying on the earth’s magnetic field as the constant field. The earth’s magnetic field is not horizontal, or pointing from north to south. In most areas, the field either has a vertical up component, or a down component. And in most cases the field points from south to north as the rotational north pole (AKA the north pole) is near the magnetic south pole, where field lines go in, not coming out. In my area for example, the earth magnetic field points primarily downwards, only slightly towards north, making an angle over 70 degrees with the horizontal. The magnetic field has very little component in the east direction. The relative strength between East, North, and Downward is about 1:64:195. The angle the magnetic field vector makes with the horizontal is called magnetic inclination, with downward being positive. This is approximately atan(195/64)=72 degrees. The angle the magnetic field vector’s horizontal component makes with the true north is called magnetic declination, with east being positive. This is approximately atan(1/64)=0.9 degrees. The properties of magnetic field varies greatly from place to place and also changes from time to time. To find out the magnetic field in your area, visit noaa.gov:

https://www.ngdc.noaa.gov/geomag-web/#igrfwmm

The following is from my area:

The next calibration is for sensitivity. The sensor either returns an analog voltage or a digital value. How do we convert this return into actual magnetic field in mG? This means finding the relation between the sensor readout and actual physical values. Say the sensor is digital and returns values between 0 and 32767, which represents magnetic field between 0 and 49150mG. Then you can use the conversion vactor 49150/32767=1.5mG/LSB to convert your readout. Here LSB means one digit (least significant bit). For an analog sensor, you will need an x.xx mG/V.

All sensors provide this factor in their spec sheets so you can just use this factor to get the actual magnetic field. But since not all sensors were made identical, some sensors should use larger or smaller values than the spec’s factor. Some manufacturers test their sensors at factory and store a correction factor for each axis in the sensor for better accuracy. For example, the MPU-9250 sensor (the magnetic sensor is AK8963) has digital magnetic field sensor output. One of the sensors I got has the following factory trims:

X-Axis sensitivity 1.18

Y-Axis sensitivity 1.19

Z-Axis sensitivity 1.14

So instead of a straight 1.5mG/LSB, the x-axis has 1.5*1.18=1.77mG/LSB. We’ll multiply this factor to the x-axis readout to get x magnetic field in mG. Same for y and z axes.

And yet sometimes these adjustments are still not able to make all 3 axes read the “400mG” value. They are very close to be identical already so we can apply a small correction. We average the maximal readings from all three axes, then divide by the maximal reading of each individual axis to get three factors. If say the x-axis reads slightly higher maximum than y and z axes. Then the avg_max/x_max will be slightly less than 1. We apply this factor to the the final result:

If the sensor you’re using doesn’t have the factory trim, then you’re out of luck unless you have both a magnetic shield and a nice uniform magnetic field inside the shield so you can find out the trim.

The following are steps for the AK8963 magnetic sensor calibration I did, using Kris Winer’s MPU9250 Arduino library:

1. Extract factory sensitivity factor by calling initAK8963().
2. Call readMagData() to extract raw data repeatedly enough to get accurate max/min for all axes.
3. Calculate bias in raw counts by (max+min)/2, such as (510+ (-490))/2=10
4. Combine factory sensitivity factor and Kris’s scale factor described above.
5. Keep these biases and factors in program.

As a testimony to the method that works, here are two graphs.

This graph has three separate plots from raw data that represent mx vs. my, mx vs. mz, and my vs. mz. I rotated my sensor as much as I could for a few minutes before I got bored and couldn’t think of any other ways of rotation. The fact that all three plots are near circular means that the sensor’s three axes have very similar sensitivities. I was simply rotating the sensor around, giving all three axes opportunities to read the whole magnetic field. This means when the x axis reads the whole field, y and z axes read nothing. Some trigonometry can show that the result is a series or circles making a disc. But the discs were not centered as I expected, because the z axis had a very large bias (this means the blue is mx vs. my). After my bias calibration, here is what I got:

Now all three plots are centered at zero pretty well visually, although I didn’t multiple the factory sensitivity or the final factor from Kris’s calculation. This shows that bias calibration is the most crucial. If your sensor doesn’t have stored factory sensitivity factors, getting it calibrated for bias alone will go a long way.

FYI, this is my sensor board inside an enclosure. I found it much easier to hold and rotate when it’s in a box. The cable became much less of an issue when I was rotating the box. The black square board is my SDI-12 USB adapter. The purple board is the sensor board. I customized the SDI-12 USB adapter by gluing the sensor board to it and connecting it to the I2C bus on the adapter.

## Second update on the new SDI-12 USB adapter

I found sometime to assemble a batch of the new boards. I populated the 12-pole terminal block on top and a row of headers on this one board for firmware development. Since not everyone will need these new features, the 12-pole block and the header for extension board or UART serial port will be optional and you can specify with your order that you need them. Adding these components adds more cost due to parts cost, assembly, and testing time. You could solder these headers yourself if you have some basic soldering skills. The UART serial port header is soldered on the underside of the board with a right-angle header to avoid the extension board and keep wires tidy.

If you need to use these boards over UART serial port such as connecting them to an Arduino or MicroPython board, please let me know with your order. I will place a solder blob between two pins on the USB serial IC so that it is placed in RESET mode to not interfere with UART serial communication with your microcontroller off board.

I assembled two extension boards, stacked them on top of the adapter, and set them to address 0 and address 1. These extension boards with come with a stacking header soldered on and four M3 standoffs, washers, and nuts. This ensures the proper spacing between boards to prevent short circuiting. I also need to trimming 20 pins on the underside of the board so that the underside of one extension board won’t touch the top side of another extension board below it. If you want, you can buy a set of 4 3-pole terminal blocks and populate them on the extension board to connect to more SDI-12 sensors, although I don’t recommend more than about 8 SDI-12 sensors from any vendor on the same adapter and extension board. A basic test running the SDI-12 + Analog USB adapter firmware on this adapter and extension board was successful, which was how I tested the extension board’s assembling quality.

My next steps are:

Extension board:

• Expend the firmware to talk to as many as 4 such extension boards for a total of 16 high-precision analog inputs
• Test address-setting jumpers (don’t expect any issues)
• Populate SDI-12 headers on one extension board and test it (don’t expect any issues)

With one extension board and its address set to 0, getting high-precision analog readings is the same as using the SDI-12 + Analog USB adapter, by sending zM! and zM1! (differential reading), then using zD0! to retrieve data. With more extension boards, reading the 4 channels on board with address 1 will be zM2! and zM3! (differential reading) then the same zD0! to retrieve data. Board address 2 will have zM4! and zM5!, while board address 3 will have zM6! and zM7!. Then zM8! is reserved for the on-board basic analog channel read, while zM9! retrieves number of pulses from these channels.

• Develop firmware to read analog channels on the adapter itself (for basic analog signals at around 5mV precision).
• Develop firmware to read pulses from the analog channels on the adapter itself (for rain gauges, flow meters etc. that output pulses).

Then I’ll test everything with a test rig. Stay tuned!

I have been working on some updates to the SDI-12 USB adapter so that it would add more features to a data logging system. So far, I’ve updated the PCB (left board) to include additional connectors. The top of the board will have 4 analog channels. This is not as accurate as the red SDI-12 + Analog USB adapter boards. The SDI-12 + Analog USB adapter has practical accuracy of 20 microvolts and has differential input channels. The 4 channels on the basic SDI-12 USB adapter have accuracy of about 5 millivolts. Also there is not a voltage reference so the measurement will be affected by the USB voltage, which is only nominally 5V. Nevertheless, if there are some sensors that output voltages in 0-5V range you want to log with moderate accuracy, such as a potentiometer, or a thermistor for approximate temperature calculation, you can use these channels. The breakout looks the same as the SDI-12 + Analog USB adapter. There are no serial resistors so you have to add yours if you want to convert resistance to voltage.

I will release a new firmware version on these newer boards. At the same time, I am considering adding digital counting features to these analog channels so if someone wants to count pulses such as flow meters or rain gauges, they can use these channels for such purpose. I plan to develop this part in the summer.

The last connector (left board middle) I have added will help developers using MicroPython platforms easily connect to it via serial ports, since most MicroPython boards don’t have USB hosts. I will start shipping these newer SDI-12 USB adapter boards soon although new firmware that makes use of these features will have to wait until later.

Will this affect your existing projects? Very unlikely. The new adapter has all the features of the old adapter. The SDI-12 + Analog adapter will still be around since it is a nice compact form factor. The new SDI-12 adapter plus the analog extension board will be approximately the same price as as the SDI-12 + Analog adapter.

## More SDI-12 sensors tested with the adapters

As more researchers and developers are using my SDI-12 USB adapters, more sensors have been tested to run with the adapter. Here are some of the new additions recently:

Gill Instruments:

• WindSonic Ultrasonic Wind Sensor (Thanks MG!)

HSTI:

• HydraSCOUT multi-sensor soil moisture and temperature probe (Thanks Bertrand!)

Sutron:

• Accubar SDI-12 Barometric Pressure Sensor, Model 5600-0120-3 (Thanks Meidad!)

Unidata:

• Starflow QSD Ultrasonic Doppler Velocity And Depth sensor (Thanks Yiren!)

Since Decagon has merged with a German company UMS into METER Group, some of their product lines are renamed and other products are added to these new lines. Here is a list of tested sensors from them:

METER group (formally Decagon and UMS):

• Atmos 22 (DS-2) (Sonic Anemometer)
• Atmos 14 (VP-4) (temperature, vapor pressure, relative humidity sensor)
• Atmos 41 weather station (solar radiation, precipitation, vapor pressure, relative humidity, air temperature, barometric pressor, horizontal wind speed, wind gust, wind direction, compass heading, tilt, lightning strike count, lightning average distance) (Thanks CD!)
• PRI (spectral reflectance sensor)
• NDVI SRS (spectral reflectance sensor)
• GS3 (Ruggedized Soil Moisture, Temperature, and Electrical Conductivity Sensor)
• TEROS 21 (MPS-6) (Calibrated Water Potential Sensor)
• ECH2O 5TE (5TE) (Volumetric Water Content, Electrical Conductivity, and Temperature)
• ECH2O 5TM (5TM) (Soil Moisture and Temperature Sensor)

If you are using the SDI-12 USB adapters, I’d love to include your SDI-12 sensors on my list of tested sensors. Leave me a message.

## Data logging with Wipy and SDI-12 dongle

If you have heard of the Python programming language and its easy-to-learn and easy-to-use fyou’ll be pleasantly surprised that someone has successfully implemented Python on microcontrollers a few years ago, appropriately named MicroPython! He has developed his own MicroPython board and ported the code to a number of similar microcontrollers. Imagine a low-power microcontroller with “lots” of memory (compared with Arduino) happily running Python code that talks to the internet etc. I’ll start writing about MicroPython and how you may use it for DIY electronics and data logging in a number of posts but this post is an announcement related to MicroPython and the SDI-12 serial dongle:

This dongle has a similar set of features as the SDI-12 USB adapters but lacks the USB connectivity, just having serial connection to arduino. Since all MicroPython compatible boards have serial ports and library to run the port, you can connect a MicroPython compatible board to one of these dongles and run almost the same Python data logging code I provide to SDI-12 USB adapters. Here is the first successful attempt that made it happen. Jason is a developer on the Wipy platform, which is a microcontroller supporting MicroPython. He is developing a data logger. In order to talk to SDI-12 sensors, he got one of my SDI-12 dongles. The Wipy board has serial ports but at 3.3V logic. The dongle has 5V logic. I built a simple voltage divider on the dongle for him and he was able to communicate with it using the Wipy board. Here is the code that he wishes to share:

Wipy_logger.py

This script runs on Wipy (or similar MicroPython platforms) and logs data from SDI-12 sensors to the on-board SD card. Here is what a Wipy board V 3.0 looks:

If you have been looking at the SDI-12 USB adapters lately, you may have noticed that the manual got updated a bunch of times between February and March (latest version being 3/19/2018). I’ve been updating the manual to:

Original SDI-12 USB adapter since 2015:

Notice that now the updated adapter is the same size as the other flavors of the adapters, such as the SDI-12 + Analog adapter and the SDI-12 + GPS adapter. These are the features the updated adapter has:

• Four SDI-12 sensor connectors
• External power connection and sensor power selector (5V USB or external)

2. I updated several sections of the manual to make the manual easier to understand.

• Bookmarks in the .PDF file
• More detailed description of the updated adapter
• Details of how to configure an SDI-12 adapter
• Other places have been tidied up as well

By the way, did I mention that I reduced all my adapters by \$5 to \$10?

• SDI-12 USB adapter (updated) \$45 (\$5 cheaper)
• SDI-12 + GPS USB adapter \$60 (\$10 cheaper)
• SDI-12 + Analog USB adapter \$80 (\$10 cheaper)
• There will be discounts for bulk purchases, such as 10, 20, 50 etc. Please contact me if you intend to buy more adapters so I can send you an accurate quote of prices with discounts and shipping cost.
• If you still need the green adapter for its simplicity or form factor, you can contact me to do bulk orders, 10 adapters or more.
• If you need something else to be on your adapter to connect to a particular sensor, contact me to see how I can help you achieve that. The sensor would definitely not be SDI-12 but I’ll make it so that reading from it is just like reading from any other SDI-12 sensor so your program needs little to no change.

Since I am assembling and testing these adapters myself and use trustworthy US parts vendors, I will have to get more efficient in my work to make up the difference. I made some improvements on a reflow oven that I will be using for new batches of adapters. I hope the effort pays off. I would be able to assemble a batch, load into the reflow oven, and assemble the next batch or do cleanup or soldering thru-hole components such as the screw terminal blocks while the oven reflows automatically. I’ve been using a manual control (Variac transformer) for reflow for the past several years, which requires constant attention.

One negative impact from the US postal service, since the end of January 2018, USPS will no longer ship merchandise via international first-class letter. The only alternative is first-class package, which is a much more expensive service. A package containing one adapter that used to cost less than \$4 will now cost \$9 to ship to Canada (come on!) or \$14 to most of the rest of the world. I felt that I’m being squeezed out of the international market by the postal service. The only upside is that now the package seems to come with complete tracking in both countries. I made a small change to the first-item shipping cost and additional item shipping cost.

I am also considering designing a complete low-cost data-logging system so it would be more integrated than the adapters. The adapters have proved to be very successful indeed. It shows up as number-1 search for keywords “SDI-12 adapter” or “SDI-12 USB adapter”. I didn’t pay google anything for the free advertisement. Everyone that searched and read my blog helped spread the word! Thank you!

I plan to keep improving the adapters so that anyone that is integrating SDI-12 sensors will find it easy to just get an adapter and add SDI-12 sensors. For anyone else that is interested in low-cost data logging solutions but are more interested in a turn-key solution instead of investing time and effort in learning python programming or raspberry pi, this logger will make things easy for you!

Any comments, suggestions, your use case, including what sensors and telemetry solutions you want that you would like to share? Leave a comment! The design of the data logger is still very fluid so that your opinions WILL influence its development!

The development of a turn-key solution WILL positively affect the adapter development as well. I expect to have a 3-prone approach ultimately: USB adapters to add SDI-12 sensors to existing logger projects that use computers/raspberry pi, serial dongles/shields to do the same for Arduino-based existing logger projects, and a complete logger for those seeking turn-key solutions.

Have a nice spring/fall day!

## SDI-12 USB adapter manual and logging script updated

Due to the discontinuation of data.sparkfun.com, I moved data logging to ThingSpeak.com

I have other updates that I rolled up in the manual, such as more details on telemetry. New manual is posted on SDI-12 USB adapter page as well as the updated data logging code. Here is a snapshot of data I logged to ThingSpeak.com:

The full data stream is here:

https://thingspeak.com/channels/359964

## SDI-12 + GPS USB adapter

Later I found some need to add GPS modules to the existing SDI-12 USB adapter so that mobile data loggers such as those mounted on tractors will be able to produce with Geo-tagged data that can be made into maps. After some initial struggle using the new ATMEGA328PB processor that sports two hardware serial ports (one to talk to PC and the other with GPS), I realized that the GPS module actually interfered with the processor and caused program freeze-up. Then I made some hardware revisions and was able to prevent interference. It turned out that the new ATMEGA328PB processor that I used in my initial prototype was especially susceptible to interference when I used its second hardware serial port that have the same pins as the SPI pins that program the processor. So I switched to the ATMEGA1284P processor that I have been using on my open source physics laboratory design.

After extensive tests, I am happy to add this adapter to the product line. You can purchase (small quantity at the moment) at inmojo.com or on my blog (in the middle of the page). The adapter requires a separate purchase of the GPS module that Adafruit makes and sells, the Ultimate GPS module part number 746. You only need to solder four pins on the GPS module, the TX, RX, GND, and VIN, and the same pins on the adapter. Since the GPS module is relatively expensive, I can’t stock them up. But if you really need it assembled, you may have a GPS unit sent to me and a few extra dollars for assembly and testing. Just contact me once you make a purchase if you want assembly.

## Adding Beagle bone to the mix

I was recently contacted by someone who was interested in using the SDI-12 USB adapter on a Beagle Bone Black single board computer. I’ve never used a Beagle Boards but I know that they are ARM-based computers running linux thus should operate similarly to the Raspberry pi boards that I’ve been playing since 2012. So I took the dive and got a Beagle Bone Black from MCM electronics and gave it a try. Right out of the box the board boots into a version of Linux. I was able to test its connectivity with the SDI-12 USB adapter successfully using the “screen” command. Later I ran a simple Python script under Python 2.7 and got very nice results:

There are a few differences that I noticed while exploring BBB:

1. There is a “serial” module included in Python that is not available on other platforms, such as windows, linux, Mac OS, or Raspberry pi. It functions like the pyserial module used on all these systems.
2. The board boots much faster than raspberry pi 3B, maybe in 15 seconds. RPI 3B takes about 30 seconds. This is a good thing.
3. There are a lot fewer instructions on basic operations for Beagle boards than Raspberry pi, which was the primary reason I got my raspberry pi B instead of Beagle board back in 2012.

When I have more time, I will test my open-source python data logger on BBB to make sure it works just as it does on all other systems. For now, one more box is checked: “compatible with Beagle Bone Black”.

## Free assistance on data logger projects

Summer is finally coming to my backyard and my spring semester is coming to an end. Thinking ahead (skipping over all the final papers to grade), with the whole summer ahead of me, starting 5/15/17, I can provide some free assistance to those that are working on your data logger projects using my devices, such as the SDI-12 data logging shield and SDI-12 USB adapters.

My goal is to get you started so you can quickly work on your own after my help. I’ve used Teamviewer to remotely help people install software, test their adapters with their own sensors, and modified my Python data logging code in the past. As long as I have some time to spare, I am willing to keep providing help. I appreciate it if you could help me spread the word. I might ask you to provide a blurb such as what sensors you use and what type of project you are working on etc. as a form of exchange for my free help.