Careful with Python indentation

If you are like me, using the default Python integrated development environment, IDLE, you probably feel like me, desiring more features, such as line numbers, horizontal scrolling, code folding, etc. I am aware of a Python module called IDLEX, which has lots of these features. On the other hand, if you are already familiar with text editors such as Notepad ++ (Npp in short), which is excellent, you may want to just switch over to Npp for your Python script editing. You can either use their default color theme or edit it to look like IDLE. There is an essential feature you need to set in order to produce correct indentation though.

Because unlike other major programming languages such as C/C++, Java, etc. Python uses indentation to indicate structure. If you write an if-statement, you do this:


if (statement==True):
    print('It is true!')
else:
    print('It is not true!')

I’ve learned a lesson of mixing tab characters and white spaces as indentation. It’s a bad mix. Although I love the tab character more than spaces, I consider spaces better in Python since it is the Python standard. Here is some tab vs. space discussion. They each have advantages. So how do we set up Npp to produce the Python standard 4-spaces whenever we press the TAB key on the keyboard? Here it is:

This “Preferences” dialog is under “Settings” menu. Just make sure you select “Language” on the left and then “Python” on the right (not the middle), deselect “use default value”, and select “Replace by space”. This produces Python 4-space indentations. Once you’re done, you may want to inspect your exist script to see if there are any tabs by selecting option “Show all characters” under “View” menu’s “Show Symbol” option. This way tab characters look like red arrows while white spaces look as red dots.

Now you are using Npp for editing, you can create a “Run” command inside Npp to run your Python code:

Under “Run” menu “Run…” option, you can type up the command python -i “$(FULL_CURRENT_PATH)”

Save a short cut such as Shift + F5. Then you can run the code with the shortcut.

Interesting indoor temperature data

As a demonstration for my SDI-12 USB adapter, I have a raspberry pi zero-w log temperature from a Decagon (METER group) 5TM soil temperature probe using the adapter. The probe is not buried in soil so it is just sensing the indoor temperature. The sensor is located away from direct sun light, just in case you wonder. Here are two snapshots of the data stream:

https://thingspeak.com/channels/462583

One-day

Multi-day

Except for the daily dip-and-rise combinations (why?), the temperature of my home is rather constant near 21.5 Degree Celsius. That is good to know that there is no issue with my thermostat in stabilizing temperature. So why the daily dip-and-rise combinations? The top graph was from Sunday. Notice two dip-and-rise combinations on the plot instead of one? The lowest points occurred around 2:30pm and 8:30pm but that’s not important. What’s important are when the dips started, around 1pm and 7pm. Those were the times we were cooking! The extra heat from the kitchen must have triggered the thermostat to turn off the heater, which made the room temperature drop before it kicked back in and heated the room back up. Without the heater on, the room drops steadily by 0.3 degC per hour. When heat is turned back on, the heater raises temperature at 0.5 degC per hour. You can also see that we were cooking dinner every night. Who would have thought that by posting room temperature online they are giving up a lot of privacy? 🙂

Data logging with Wipy and SDI-12 dongle

If you have heard of the Python programming language and its easy-to-learn and easy-to-use fyou’ll be pleasantly surprised that someone has successfully implemented Python on microcontrollers a few years ago, appropriately named MicroPython! He has developed his own MicroPython board and ported the code to a number of similar microcontrollers. Imagine a low-power microcontroller with “lots” of memory (compared with Arduino) happily running Python code that talks to the internet etc. I’ll start writing about MicroPython and how you may use it for DIY electronics and data logging in a number of posts but this post is an announcement related to MicroPython and the SDI-12 serial dongle:

This dongle has a similar set of features as the SDI-12 USB adapters but lacks the USB connectivity, just having serial connection to arduino. Since all MicroPython compatible boards have serial ports and library to run the port, you can connect a MicroPython compatible board to one of these dongles and run almost the same Python data logging code I provide to SDI-12 USB adapters. Here is the first successful attempt that made it happen. Jason is a developer on the Wipy platform, which is a microcontroller supporting MicroPython. He is developing a data logger. In order to talk to SDI-12 sensors, he got one of my SDI-12 dongles. The Wipy board has serial ports but at 3.3V logic. The dongle has 5V logic. I built a simple voltage divider on the dongle for him and he was able to communicate with it using the Wipy board. Here is the code that he wishes to share:

Wipy_logger.py

This script runs on Wipy (or similar MicroPython platforms) and logs data from SDI-12 sensors to the on-board SD card. Here is what a Wipy board V 3.0 looks:

SDI-12 USB adapter and manual updates

If you have been looking at the SDI-12 USB adapters lately, you may have noticed that the manual got updated a bunch of times between February and March (latest version being 3/19/2018). I’ve been updating the manual to:

1. Introduce the newer SDI-12 USB adapter (black) that replaced the original adapter (green).

Original SDI-12 USB adapter since 2015:

slide2

Updated SDI-12 USB adapter:

SDI-12 USB + adapter

Notice that now the updated adapter is the same size as the other flavors of the adapters, such as the SDI-12 + Analog adapter and the SDI-12 + GPS adapter. These are the features the updated adapter has:

  • Four SDI-12 sensor connectors
  • External power connection and sensor power selector (5V USB or external)

2. I updated several sections of the manual to make the manual easier to understand.

  • Bookmarks in the .PDF file
  • More detailed description of the updated adapter
  • Details of how to configure an SDI-12 adapter
  • Other places have been tidied up as well

By the way, did I mention that I reduced all my adapters by $5 to $10?

  • SDI-12 USB adapter (updated) $45 ($5 cheaper)
  • SDI-12 + GPS USB adapter $60 ($10 cheaper)
  • SDI-12 + Analog USB adapter $80 ($10 cheaper)
  • There will be discounts for bulk purchases, such as 10, 20, 50 etc. Please contact me if you intend to buy more adapters so I can send you an accurate quote of prices with discounts and shipping cost.
  • If you still need the green adapter for its simplicity or form factor, you can contact me to do bulk orders, 10 adapters or more.
  • If you need something else to be on your adapter to connect to a particular sensor, contact me to see how I can help you achieve that. The sensor would definitely not be SDI-12 but I’ll make it so that reading from it is just like reading from any other SDI-12 sensor so your program needs little to no change.

Since I am assembling and testing these adapters myself and use trustworthy US parts vendors, I will have to get more efficient in my work to make up the difference. I made some improvements on a reflow oven that I will be using for new batches of adapters. I hope the effort pays off. I would be able to assemble a batch, load into the reflow oven, and assemble the next batch or do cleanup or soldering thru-hole components such as the screw terminal blocks while the oven reflows automatically. I’ve been using a manual control (Variac transformer) for reflow for the past several years, which requires constant attention.

One negative impact from the US postal service, since the end of January 2018, USPS will no longer ship merchandise via international first-class letter. The only alternative is first-class package, which is a much more expensive service. A package containing one adapter that used to cost less than $4 will now cost $9 to ship to Canada (come on!) or $14 to most of the rest of the world. I felt that I’m being squeezed out of the international market by the postal service. The only upside is that now the package seems to come with complete tracking in both countries. I made a small change to the first-item shipping cost and additional item shipping cost.

I am also considering designing a complete low-cost data-logging system so it would be more integrated than the adapters. The adapters have proved to be very successful indeed. It shows up as number-1 search for keywords “SDI-12 adapter” or “SDI-12 USB adapter”. I didn’t pay google anything for the free advertisement. Everyone that searched and read my blog helped spread the word! Thank you!

I plan to keep improving the adapters so that anyone that is integrating SDI-12 sensors will find it easy to just get an adapter and add SDI-12 sensors. For anyone else that is interested in low-cost data logging solutions but are more interested in a turn-key solution instead of investing time and effort in learning python programming or raspberry pi, this logger will make things easy for you!

Any comments, suggestions, your use case, including what sensors and telemetry solutions you want that you would like to share? Leave a comment! The design of the data logger is still very fluid so that your opinions WILL influence its development!

The development of a turn-key solution WILL positively affect the adapter development as well. I expect to have a 3-prone approach ultimately: USB adapters to add SDI-12 sensors to existing logger projects that use computers/raspberry pi, serial dongles/shields to do the same for Arduino-based existing logger projects, and a complete logger for those seeking turn-key solutions.

Have a nice spring/fall day!

JLCPCB – A new PCB fabrication house

I was recently approached by a new PCB fabrication house JLCPCB (technically they’ve been in business for some time but just started advertising to USA customers) to write some reviews on their service. Given my experience designing PCB and using PCB fabrication services (batchpcb, seeedstudio, iteadstudio, oshpark etc.) over the past decade, writing a review shouldn’t be too hard. If they are any good, I could use their service myself and recommend them to others in the hobby electronics community. So I sent them two of my designs and got them back relatively fast, with DHL. Since I still have some older versions of one of the boards, I could make a comparison with the two fabrication services. JLCPCB has offered their service free of charge for the exchange of a fair review. So I decide to not mention which competition I pitched JLCPCB against. What I primarily looked at was how accurately each layer of board is printed and how well they are aligned with one another from each fab service. JLCPCB‘s results are very good. Its competition, a long-standing name among DIYers, doesn’t look quite as good.

In case you’re still learning how to design PCBs, especially surface-mount components, here are some terms I’m going to use:

Top layer: this logic layer contains all copper traces, pads, and vias on the top side of the two-sided circuit board. A process is used to protect all aforementioned features specified in this layer when the entire board is etched in acid. If the process isn’t accurate, then your features aren’t exactly where they are supposed to be.

Top solder paste: this logic layer contains only pads for surface-mount components. It is a subset of top layer and is used to generate stencils for reflow soldering. Again if the fabrication isn’t very good, these features tend to not align with other features.

Top solder resist: this logic layer contains similar information to the pads and vias contained in top layer but the sizes of the features in this layer are slightly enlarged to a peel-back amount so they don’t accidentally apply solder resist on top of your pads where solder should go. Solder resist is a lacquer that prevents solder from adhering to the copper traces. They also protect the bare copper from rusting away. You will want them to be applied to all your traces and only leave the pads exposed so later process will cover them with a very thin layer of solder (the shiny looking stuff) and you then reflow solder your components to these pads. If the registration of this layer of lacquer is not well registered with the top layer, you will see visually under a magnifier. This is why fabrication houses usually do quite a bit of peel-back so they leave room for themselves to be less-aligned but still the lacquer won’t cover up the pads.

The following images are from an FTDI chip (FT232RL). The pitch was the finest on my board so I selected these pads for comparison of how well the different layers register with one another on these two service providers.

Top: JLCPCB Bottom: competition


They look similar. I’ve unfortunately scratched the pads on the competition’s board. I applied solder paste to that board and later cleaned the paste off so I could photograph its pads. My bad.
On a closer look there is a difference:

SMD pads (SSOP 0.65mm pitch):

Top: JLCPCB Bottom: competition
You can see that JLCPCB‘s boards (top) have SMD pads (shiny metal pads) that are very symmetrically situated inside the solder-resist masks (slightly larger dark rectangle). The solder resist also goes closer to the board than its competition. These are manufactured from the same designs! The engineers at the competition must have increased the peel-off (sizes of feature not covered by solder resists) to offset their less-perfect layer registrations and/or accuracy on each layer. As you can see, not all pads and their solder resist have the same offset. Some look better than others. This is clear with the second to the last pad and the last pad. The pads are not centered at all. This is very consistent across the board made by competition. JLCPCB has better overall registration than competition. Better registration translates into better chance to prevent solder bridges and less chances to reworking on your boards after you reflow them, that means time and money saved.

Thru-hole pads (o.1″ spacing):

Top: JLCPCB Bottom: competition

The top one has less size solder resist layer, the edges of the red lacquer surrounding the shiny pads (almost same size as the thru-hole pads) and very symmetrical. The competition has again increased the solder resist layer and couldn’t keep the layer registered well with the pad.

The via to the right of the bottom right through hole pin hole will be compared next. On JLCPCB‘s board, its solder resist has 2.8mm diameter when displayed on my computer monitor. On the competition board, it is 3.1mm. The images were taken under a high-magnification lens and I checked the images to be exactly the same zoom, measuring the same across the same features on screen. What this means is that if you have a lot of vias in one area, very close to one another, you may get some solder bridges between the vias if there is not enough solder resist to separate them. Not a problem on this board but a problem if you happen to have vias very close and also close to thru-hole components. You solder the thru-hole and inevitably fill the adjacent vias with some solder. This could short vias.

Overlay (white texts):

The quality of the white overlay texts are about the same between the two fabricators, although at some places you see one board having better quality than the other board while at different places the quality is reversed. This is not a crucial feature to look at though.

JLCPCB: top, Competition: bottom

With the explanation above, you can easily distinguish these two photos. The bottom one has so much space between the edges of the shiny pads and the red lacquer (less qualty). The top one has so much less and so symmetric.

So the results are clear. JLCPCB is a pretty decent PCB fab house and I will order my next batch from them. The shipping cost is also slightly less than competition if you use DHL. I always use DHL. They do a good job delivering to small cities like the one I live in!

Here is a link to JLCPCB’s website:

www.jlcpcb.com

I am not getting any commissions for your purchases. The link has no “trackers” 🙂

P.S.: I had a research student hand solder this board (yes, every single chip resistor and the FTDI chip) as a good test of his skills and it turned out fine. This is a proof that having the right size solder resist helps, really. This is what the board looks like after assembly:

Augmented reality sandbox control box updated

This slideshow requires JavaScript.

I constructed a couple of these control boxes for my ARsandbox project using Arduino Micro and Adafruit Feather 32u4. The main goal is to make the interaction between a user and the software more intuitive. The two flashing buttons add and remove water from the simulation, which is pretty nice. I wanted to add another feature, to move the elevation up and down. Say you have a lot of sand in the box but you want to display some “water”. You have to remove a lot of sand (time consuming) from the box or edit a config file (near impossible for an average user). On the other hand, if I can change the base plane location in the calculation, I can achieve the above goal without removing sand or editing files. With help of the original developer Dr. Oliver Kreylos, I started messing with the source code. It took me a day to understand what I could do to change the base plane and how to do it. I ended up creating a tool to manipulate the plane. Now that the plane can be moved, I added a rotary encoder to the control box to emulate the keyboard keys assigned to move the plane. Voila!

The photo gallery above shows 5 shots of different elevation values. You can clearly see the “sea” on bottom right is shrinking after each turn of the knob.

Here is a short video. I was slowly raising the elevation so more and more land became above water. Then I quickly returned the land back into water:

Here is a photo of the inside of the box:

I added a breakout board for rotary encoder and buttons. It’s very tightly sandwiched between the two buttons. There is no space between the board and the buttons, not even a fraction of millimeter. I don’t know how these pieces just managed to fit. Here is the breakout board:

I made this a while ago as an interface for my phi-panels LCD backpacks. This breakout board is pretty simple. There are two encoder channels and one shaft button, plus six push buttons, which I am not using. I just connected to the encoder channels, shaft button, and the common. I had to cut off a strip on the right side of the board to fit into the box. I was using my phi_interfaces library to read the encoder and emulate a button push if the encoder is rotated one way, another button if the encoder is rotated the other way. I didn’t assign any function for the shaft button. I am thinking that I should use it to switch between elevation adjustment and water speed adjustment.

I haven’t updated the Adafruit Feature 32u4 version but will probably design a new printed circuit board so assembling will be easier, and not involving cutting off part of a board. Some code in case anyone wants to replicate it.


/*
 * Credit: Dr. John Liu
 * Purpose: This sketch emulates keyboard keys "1" and "2" with two push buttons with LEDs. It also flashes the LEDs
 * If a rotary encoder is present, it will emulate '5' and '6' if you rotate the encoder's shaft. This combined with my modification on ARsandbox source code will shift color mapping up and down.
 * This version uses p-channel mosfets and open drain with 10Kohm pull-up to 5V to control LEDs with the 3.3V Adafruit Feather32u4.
 * Notes: On Adafruit Feather 32u4, pin 9 is connected to battery sensing voltage divider.
 * 2018-01-28
 * Visit https://liudr.wordpress.com for more information
 */
#include "Keyboard.h"
#include <phi_interfaces.h>
#define USING_MICRO
#ifdef USING_MICRO
const int button_1=2; // 11 for Adafruit feather 32u4, 2 for Arduino Micro;
const int button_2=3; // 10 for Adafruit feather 32u4, 3 for Arduino Micro
const int EncoderChnA=7;
const int ChnCommon=8;
const int EncoderChnB=9;
const int ShaftBtn=10;
const int EncoderDetent=18;
#endif

#ifdef USING_FEATHER
const int button_1=11; // 11 for Adafruit feather 32u4, 2 for Arduino Micro;
const int button_2=10; // 10 for Adafruit feather 32u4, 3 for Arduino Micro
#endif

const int led_1=6;
const int led_2=5;
const unsigned long led_on_ms=300;
const unsigned long led_off_ms=1700;
const unsigned int button_1_key='1'; //KEY_LEFT_ARROW;
const unsigned int button_2_key='2'; //KEY_RIGHT_ARROW;
const unsigned int UpKeyOut='5';
const unsigned int DownKeyOut='6';

int prev_1=HIGH;
int prev_2=HIGH;
int led_stat=LOW;

unsigned long prev_1_ms=0;
unsigned long prev_2_ms=0;
unsigned long blink_timer_0_ms=0;
unsigned long blink_timer_1_ms=0;

int debounce_ms=25;

char mapping[]={'U','D'}; // This is a rotary encoder so it returns U for up and D for down on the dial.
phi_rotary_encoders MyEncoder(mapping, EncoderChnA, EncoderChnB, EncoderDetent);
//multiple_button_input* dial1=&my_encoder1;

void setup()
{
  // make pin 2 an input and turn on the
  // pullup resistor so it goes high unless
  // connected to ground:
  pinMode(button_1, INPUT_PULLUP);
  pinMode(button_2, INPUT_PULLUP);
  pinMode(led_1,OUTPUT);
  pinMode(led_2,OUTPUT);
  pinMode(ChnCommon,OUTPUT);
  digitalWrite(led_1,LOW);
  digitalWrite(led_2,LOW);
  digitalWrite(ChnCommon,LOW); // Using this pin as ground since some prototypes don't have enough gnd pins.
  Keyboard.begin();
  blink_timer_0_ms=millis();
}

void loop()
{
  unsigned char ch=MyEncoder.getKey(); // Rotary encoder emulates two buttons.
  if (ch==mapping[0])
  {
    Keyboard.write(UpKeyOut);
  }
  else if (ch==mapping[1])
  {
    Keyboard.write(DownKeyOut);
  }

  switch (led_stat)
  {
    case LOW:
    if (millis()-blink_timer_0_ms>led_on_ms)
    {
      blink_timer_0_ms=millis();
      led_stat=HIGH;
      pinMode(led_1,INPUT); // Open drain to let pull-up resistor pull drain to 5V.
      pinMode(led_2,INPUT); // Open drain to let pull-up resistor pull drain to 5V.
    }
    break;

    case HIGH:
    if (millis()-blink_timer_0_ms>led_off_ms)
    {
      blink_timer_0_ms=millis();
      led_stat=LOW;
      pinMode(led_1,OUTPUT); // Pull drain to GND.
      pinMode(led_2,OUTPUT); // Pull drain to GND
    }
    break;
  }

  if ((digitalRead(button_1) == HIGH)&&(prev_1==LOW))
  {
    if (millis()-prev_1_ms>debounce_ms)
    {
      prev_1_ms=millis();
      Keyboard.release(button_1_key);
      //Serial.println("1 released");
    }
    prev_1=HIGH;
  }

  if ((digitalRead(button_1) == LOW)&&(prev_1==HIGH))
  {
    if (millis()-prev_1_ms>debounce_ms)
    {
      prev_1_ms=millis();
      Keyboard.press(button_1_key);
      //Serial.println("1 pressed");
    }
    prev_1=LOW;
  }

  if ((digitalRead(button_2) == HIGH)&&(prev_2==LOW))
  {
    if (millis()-prev_2_ms>debounce_ms)
    {
      prev_2_ms=millis();
      Keyboard.release(button_2_key);
      //Serial.println("2 released");
    }
    prev_2=HIGH;
  }

  if ((digitalRead(button_2) == LOW)&&(prev_2==HIGH))
  {
    if (millis()-prev_2_ms>debounce_ms)
    {
      prev_2_ms=millis();
      Keyboard.press(button_2_key);
      //Serial.println("2 pressed");
    }
    prev_2=LOW;
  }
}

Augmented reality sandbox control using Arduino Feather

It’s been a while since I completed this project. I promised the Augmented reality sandbox community to write something in details so someone can replicate it. Here is the gist of it:

The Augmented reality sandbox (ARsandbox) is an awesome simulator that blends sand landscaping with augmented reality. A projector casts color-coded elevation, contours, and simulated water flow over regular sand. A 3D sensor measures the sand landscape in real time so the computer knows where sand is high and how it slopes down and renders color and contours according the the 3D data. You can rain over the landscape by pressing a button or hold out your palm over a certain area.

Here is a video of one of my builds:

This is where the control box comes in. The sand box is a 40″ by 30″ box with up to 200lb of sand. The function to rain over the augmented terrain is to press the “1” key on a keyboard. Sand and keyboard shouldn’t mix, not to mention if the system is deployed in a museum, the keyboard probably should be hidden from patrons. So a box with rain and dry buttons will be most convenient. You can make these buttons by following this forum post. You buy and assemble a USB gamepad kit and doing some scripting but it’s not that easy to follow if you are a Linux beginner. I want a plug-n-play solution so that anyone that wants to add a button box can add it without any knowledge of Linux and minimal skills in circuit assembly. In order to achieve my goals, the electronics inside the box has to emulate keyboard keys “1” and “2” for rain and dry. This ensures that the control box requires no software setup on the Linux system. As a bonus, the buttons should flash blue and orange so it would invite patrons to press and is color-coded for rain (blue) or sun (orange). Enter the ARsandbox control box:

Here is a short video of the box in action:

Features of the box:

1. Two buttons. Blue button on the left emulates the “1” key. It starts the rain in the main simulation, and is used in the projector calibration program to enter tie points, and can be assigned other functions in other programs such as extracting plane. Orange button on the right emulates the “2” key. It drys the rain in the main simulator, and can be assigned functions in other programs.

2. Both buttons have LEDs that flash to invite patrons to press. My 5-yr old son figured out what they do simply by the colors they flash because he knew the rain and dry features already but wasn’t told what these buttons would do.

3. No software setup is needed. This box works like another keyboard, with only 2 keys though.

 

Here are the parts I used, in case you want to build your own:

  1. Adafruit Arduino feather 32u4 basic board (1)
  2. LED push buttons 16mm from adafruit (2)
  3. USB-micro cable (1)
  4. USB extension cable (1)
  5. P-channel MOSEFT BS250 from mouser (2)
  6. 330 ohm resistors (2)
  7. 1K ohm resistors (2)
  8. DuPont 30cm female-female jumper wires (4)
  9. Prototype board or my printed circuit board. Click here to download (1)
  10. Enclosure SK-15 and flange kit SK-99 from polycase (1)
  11. Cable gland M3198GBH from Heiland for pre-assembled cables with USB connectors (1)
  12. Arduino sketch. Click here to download

Here is the schematic of the circuit:

Since the ATMEGA32u4 microcontroller on Adafruit Feather 32u4 is operating at 3.3V, I had to use two transistors to ensure that the LEDs inside the buttons light up with sufficient brightness. Currently the code only blinks both button LEDs at the same rate. In the future, I may update the code to blink more rapidly on the button that is depressed.

The following is a prototype that I built with the parts on a perfboard. It’s a bit messy and also took almost 2 hours to complete. I had to solder 16 wire leads and a bunch of jumper wires on the back side of the perfboard. I wouldn’t recommend this to a beginner.

Here is the printed circuit board that I designed (3 boards, two on the right showing the top, 1 on the left showing bottom):

The board is fairly simple and straightforward to solder. Only 8 wire leads and soldering the rest on the printed board, which is very easy. I also cut in half 4 30cm female-female Dupont jumper wires so I can just use the female side with male pins on the board to cut wire leads in half.

Here is a photo of the assembled guts including the microcontroller soldered to the button box board. Notice the black female Dupont connectors :

Here is the assembled board placed inside the enclosure. This enclosure is not cheap but nice and easy to use. It has various knock-outs so I didn’t have to drill a single hole.

Here is the completed box:

Although I am not aiming to sell a lot of these boxes, I think that designing the printed circuit board was the right way to go. I probably spent a few hours designing and proofing my design but I saved about one hour for each subsequent box I made. Plus, if YOU are interested in making this box, it will save YOU at least that much time.

SDI-12 USB adapter manual and logging script updated

Due to the discontinuation of data.sparkfun.com, I moved data logging to ThingSpeak.com

I have other updates that I rolled up in the manual, such as more details on telemetry. New manual is posted on SDI-12 USB adapter page as well as the updated data logging code. Here is a snapshot of data I logged to ThingSpeak.com:

The full data stream is here:

https://thingspeak.com/channels/359964

Upload data to ThingSpeak

I have been using sparkfun’s Phant server for data upload and retrieval for a few years since they started the service. The service was easy to use and was free. Several months back they discontinued the service, unfortunately. I started looking for suitable alternatives.

Sparkfun recommended three, ThingSpeak, Cayenne, and Blynk. I went ahead and did some research on these services. My goal is to be able to log data online and later retrieve them and possibly visualize them using google charts, like before. I am not at the moment interested in automating my home with actuators or smart phone apps to turn on and off my hall lights. Here are my findings and why I decided that ThingSpeak was the best fit. If you are logging data for later processing or visualization, read on.

ThingSpeak.com

This service is provided by the company behind Matlab. I am not a fan of expensive commercial data manipulation tools such as Matlab but they do have a fair amount of business between universities and industry so their service might be a safer way to go against sudden discontinuation of service such as sparkfun. Basically you create a data stream and send data to the stream. It’s very similar to sparkfun. You can also retrieve your data, possibly good for running your data through google chart. They also provide some basic graphs and matlab analysis tools that I have yet tried.

There are two types of application programming interfaces (APIs) you can use: a REST API, and an MQTT API.

The REST API is based on HTTP so it’s very similar to existing services elsewhere. You use HTTP GET or POST command to send data using an API key, like a private key with sparkfun. Your data are limited to up eight values per data point. If you need more, then you need to create more streams. They also have a bulk update feature that you can use to send multiple sets of data instead of one set of data. This method allows a device to collect data and sleep in between data points. Then when it collects a fair amount of data, it connects to the Internet and sends all data in one shot. It saves power and network bandwidth. You can also create and modify the properties of streams with this API.

With the MQTT API, the underlying protocol is TCP/IP. There is no acknowledgement of data received and it is intended for low-powered devices to just wake up, take data, send it out, and go back to sleep. From my tests, data sent via this method were lost over 50% of the time. Unless future holds differently, I am not recommending this API.

Getting start is easy with ThingSpeak. Just set up an account and follow their tutorial to create a new data stream. Then the following bash code should get you started posting code:


curl "https://api.thingspeak.com/update/?api_key=your_appkey&field1=1.23&field2=2.34"

You can post any number of field values between 1 and 8. You will receive a zero as a positive response. Then you will see your results like the plot on the top of this page.

I have updated my Python data logging code to use ThingSpeak.com instead of the now discontinued data.sparkfun.com. The plot in this post is from Thingspeak.com. Here is a link to the data stream:

https://thingspeak.com/channels/359964

Can’t upgrade pyserial in latest raspbian distribution?

This is just for your information if you are a Raspberry Pi user and playing with Python code from my blog. If you are trying to use the latest distro of raspbian with pyserial for some serial port project, you may have come across this issue that regardless how you upgrade pyserial using pip3, your python3 will always call up the old pyserial 2.6 that came with the distribution. I am a bit disappointed that the foundation has included such an old version of pyserial, couldn’t they just try a pyserial 3.0 instead? My solution was to remove the python3-serial module using apt-get and then install pyserial 3.3 using pip3.

sudo apt-get remove python3-serial
sudo pip3 install pyserial

Hope this helps.

%d bloggers like this: