Install python on raspberry pi or debian

I have just started learning the python programming language. It is very different than C/C++ that I’m used to. So to Arduino and Java folks it will take some getting used to. But the reason that I am learning is the promise of cross-platform portability and powerful libraries/modules. If you could send an HTTP request in a few lines, or imagine that you write and test a program on PC and run it on a raspberry pi with no/minimal modification, would you stop complaining about weird syntax or loose data types and just learn the darn language? Well, I did.

So in order to use Python, you need to install it. Since I am interested in using raspberry pi with an arduino, I need the best serial port support that I can find. My conclusion is to use Python 3.5.1 and pyserial 3.0.1 (the latest of both as of the blog post).

On a windows PC, this would be the easiest. Download the installers and install the program.

https://www.python.org/downloads/release/python-351/

https://pypi.python.org/pypi/pyserial

On a linux PC or raspberry pi, python is included but is a lower version. The tutorial helps you install python on linux machines and raspberry pi’s.

This tutorial is based on this tutorial and pieces of other tutorials and information online:

http://www.extellisys.com/articles/python-on-debian-wheezy

To install the latest version of python, you need to build it from source code.

  1. Download the source code and unzip it in a folder.

https://www.python.org/downloads/source/

  1. Update apt-get first

$ sudo apt-get update

  1. Install required tools to build Python from source. Debian/Raspbian Jessie will install libdb5.3-dev while Wheezy will install libdb5.1-dev. You need to install tk-dev to make Python’s IDE IDLE3.5 work

$ sudo apt-get install build-essential

$ sudo apt-get install tk-dev

$ sudo apt-get install libncurses5-dev libncursesw5-dev libreadline6-dev

$ sudo apt-get install libdb5.1-dev libgdbm-dev libsqlite3-dev libssl-dev

$ sudo apt-get install libbz2-dev libexpat1-dev liblzma-dev zlib1g-dev

  1. Prepare for Pip (I didn’t do it but the tutorial I followed mentioned this, which seems a bit old)

$ mkdir -p ~/.pip/cache

$ echo ‘[global]’ > ~/.pip/pip.conf

$ echo ‘download_cache = ~/.pip/cache’ >> ~/.pip/pip.conf

  1. Make python in the unzipped source code folder (Python-3.5.1), install it in a folder that will not overwrite the current python version, such as in /usr/local/opt, delete upzipped source folder after done

$ cd Python-3.5.1

$ ./configure –prefix=/usr/local/opt/python-3.5.1

$ make

$ sudo make install

$ sudo mkdir /usr/local/opt/python-3.5.1

  1. Download pyserial source code and unzip in a folder.
  2. In the unzipped folder, run python 3.5 to install the module. If you used an alias to call python, then the install is in the default python version (3.4 on RPI) so don’t use an alias.

$ sudo /usr/local/opt/python-3.5.1/bin./python3.5 setup.py install

  1. Make aliases to run python 3.5 and idle more easily. You have to log out and back in after making them.

$ echo ‘alias python35=”/usr/local/opt/python3.5.1/bin/python3.5″‘ >> .bashrc

$ echo ‘alias idle35=”/usr/local/opt/python3.5.1/bin/idle3.5″‘ >> .bashrc

Now you are done! I forgot to mention that if you are installing Python on raspberry pi 1 or zero, the build process will take quite some time to complete. Log out and back in. In a terminal, type idle35 and you will be able to run your new Python.

SDI-12 USB adapter

After some delay, the SDI-12 USB adapter is finally here:

2015-10-03 16.16.57

This adapter is extremely easy to use. Just connect it to your PC and SDI-12 sensor. Then you can use any serial monitor or terminal emulator program to talk with your sensor. Just open the serial port at 9600 Baud rate. You can start by sending device identification command such as ?!. You will see a response from your sensor, which is the one-character address of your sensor. If you have not set its address, it is most likely to be zero (0). Then you can use 0I! to find out the manufacture and model of your sensor, before getting measurements from it. Getting measurement is easy as pie. First send 0M!, then wait for response. Then send 0D0! to fetch the measurements.

For PC users, I even wrote a data logger script that can automatically log data using the popular Tera Term program. You can choose sensor address, total number of data points, delay between points, and time zone when logging, then the program will keep logging data. The following is a screen grab of Tera Term. The sensor is a Decagon 5TM soil temperature and moisture sensor. The address is one (1) and the returned values are relative dielectric permittivity and then temperature in Celsius.

Data logger

Once you get data logging going, you can import the .CSV file into your Excel and plot it. You can choose a proper refresh rate so your data and plot are up to date when you look at them.

Plot

I’m still ordering more circuit boards but should be able to sell these on my inmojo.com store starting now. There is even a quantity discount if you need 10 or more.

Inmojo store sales page

Arduino board install problems

Arduino board is darn close to plug and play. However, there are many arduino compatible boards that come with different USB chips, making the play part just a bit further away from the plug part. So, if you have got your new arduino (or compatible), how do you install the USB chip driver? The following are the steps to identify what driver you need, where to find them, and how to install them.

There are several USB chips different Arduino compatible boards use. An incomplete list of them is supplied at the end of the post.

First of all, if you have good eye sights, you should locate the USB chip on your arduino, which is usually near the USB socket, read its markings. Then find the driver from the list at the end of the post.

On Linux, many of these drivers are included in the distribution so you don’t notice anything except that you now have a new serial port, such as TTYUSB0. On windows, however, you need more work to get the driver installed.

If you are using windows, and you can’t determine (markings too faint) what chip set you have, do the following after plugging in your arduino board:

Go to device manager and find the device with an exclamation mark under “Other devices” as an “Unknown device” (mine actually is already recognized as COM47, yeah, lots of Arduinos).

unknown device

Select properties and view the Details tab. Select under Property “Hardware Ids”. Look at the value.

USB serial port propertiesMine has VID=0403 and PID=6001. It is the FTDI FT232RL. This tells me that I would need the FTDI driver in case the driver is not already installed.

Install the correct driver, remove the arduino, reinsert it and wait for the driver to properly associate with the board. In case that didn’t happen automatically, you want to then right click the unknown device and update driver, then choose “browse my computer”. Then point to the right driver location. This step is necessary for all ATMEGAXXuX chips. The driver is in Arduino IDE under Drivers folder.

Update software driver

The following is a working list of chips, VID, PID and download links. I don’t guarantee the links are current or virus free but I did post official links.

Chip VID PID Board Link Note
Atmel ATMEGA16U2 2341 003D 003F 0042 0043 0044 ADK, DUE, MEGA2560 R3, UNO R3, ADK R3, clones Included in Arduino software under drivers
Atmel ATMEGA32U4 2341 8036 etc. Leonardo, micro, clones Included in Arduino software under drivers
FTDI FT232RL 0403 6001 Nano, Duemilanove, MEGA, clones http://www.ftdichip.com/Drivers/VCP.htm Included in Arduino software under drivers. Many fakes exist so avoid buying cheap Arduino compatible board with this chip.
WCH CH34X 1A86 5523 7523 etc. Many clone boards http://www.wch.cn/download/CH341SER_ZIP.html Supports win 10. Seems best choice for Arduino compatible boards.
Prolific PL2303 10CE Many clone boards http://www.prolific.com.tw/US/ShowProduct.aspx?p_id=225&pcid=41 The company claims that many older PL2303 models on the market are fakes so it has stopped supplying drivers for win 7 and up on these models.
Silicon Labs CP210X 11F6 Many clone boards https://www.silabs.com/products/mcu/Pages/USBtoUARTBridgeVCPDrivers.aspx The driver may not work with win 10 so if you just upgraded to win 10 and your board stops working, …

 

SDI-12 USB adapter coming

I have had a very busy summer. Workshop, conference,projects etc. So the only time I had for making new stuff was on an airplane flying back from a conference. I managed to keep myself awake for enough time to make this :D

What is it?

It is an adapter that can read SDI-12 sensors. FYI, SDI-12 is a standard for sensors and data loggers. It is very popular among agricultural research, including monitoring soil, water, sun light etc. You can find the SDI-12 specification here:

http://www.sdi-12.org/specification.php

You can find SDI-12 sensors in a lot of companies that make and sell sensors for soil and water monitoring such as Decagon, Ponsel, AquaCheck etc. These sensors are expensive, ranging from $100 USD to a few times more, depending on what they measure but they are all rugged for deployment in the field for continuous monitoring.

Photo credit Decagon

For those that aren’t familiar with this protocol, here is a short introduction:

An SDI-12 bus is a serial bus with one communication wire. There is a master, usually the data logger. There are multiple slaves each having a unique address from ‘0’ to ‘9’, then ‘A’ to ‘Z’ and finally ‘a’ to ‘z’. So you can have up to 62 devices all connected to one wire. There will also be a power wire of nominal 12V and a ground. The sensors that are not addressed automatically go to sleep to conserve data logger’s battery.

This protocol is different from the serial port (more specific TTL UART, the one you find on your arduino). There are additional signalling/timing that the serial port doesn’t have. There are also retries that the serial port doesn’t have. A data logger that reads SDI-12 sensors costs hundreds to thousands of USD and is usually not very open to hacks etc. that we see in the Arduino world.

I’ve designed shields and dongles that can talk with SDI-12 sensors and they are popular on inmojo market. But, I still want more simplicity for broader impacts. So with the 2 hours of time that I was awake on the airplane, I made a modification to my dongle and added USB to it. I also removed most connectors and only kept one screw terminal block for one sensor. To use it, just connect the SDI-12 sensor on the right side and plug this into the USB port of your computer. Then you can use your favorite terminal program to send it commands, change its addresses, get measurements etc. I’ve even written some macros for my favorite terminal program so simple inquiries can be done but button pushes. Here is a look at the board:

top

bottom

It is pretty small. About 1″ by 2″. Today I added a pull-down resistor to the board and pushed order at oshpark.com

I order my smaller boards from them because they are domestic and cheap for small boards (3 copies for $5 per sq. in.)

I will be able to get the board back and build it before the semester starts. Will post more stuff soon.

Making your programming jig

I have recently developed an open source device based on arduino, with an ATMEGA1284P chip. I need to build a number of them and program/test them as well. I did some research and found this article from sparkfun to be quite useful:

https://www.sparkfun.com/tutorials/138

For me, my device is rather large, not a small breakout board. So I need something for its size. The device has LCD, rotary encoder, RTC, SD card, etc. All need some tests before I flash a standard firmware. So I need some software that can flash at least two different firmware to a target with some human interactions. Luckly Nick Gammon has done all the great work with hex loader code. You can turn an arduino into a hex loader that programs fuse values on a target, and load hex files from an SD card to a target. Here is his repo:

https://github.com/nickgammon/arduino_sketches

I used his Atmega_hex_uploader code. It is interactive. You open the serial port and type in commands and see printouts. Very nicely done. Thanks Nick!

For my programmer, I thought about just using an arduino and a jumper wire. Then I have to solder male pins to my device’s ICSP pins. It’s a bit unnecessary and time-consuming, say if I need to program 100 devices (at least a couple extra hours to solder the pins). So I based on my original device design, moved the ICSP header to a different location, and placed a target ICSP header where the original device has ICSP header. I then kept the SD card, RTC, power supply, FTDI chip, and the MCU from the original design. The result is a device that can program other devices. I put it on standoffs. Since this device has the same screw hole locations, I can slide a newly-assembled device down the standoffs to make contact with the programmer. Then use nuts to hold the new device. I can then program it using serial port (fuses, hex code etc.) The programmer also has 2 push buttons. In the future, I will get rid of serial port and have the programmer standalone. If I press button 1, it will flash the target with one hex file that runs a series of tests. Then if all tests pass, I press the second button to flash the target with standard firmware and move on to the next board. I also have 4 LEDs (not yet soldered on) and a buzzer to show audio and visual indications of success and failure firmware flash. I also have a 8-dip switch row to further customize what I do with the target, such as program different standard firmware versions to the target for different variations of the device etc.

Here are some pictures:

Front side (left side has programmer’s own ICSP header, right side is 6 pogo sticks)

Back side (SD card, MCU, two buttons, RTC, power supply):

Programmer with a target board on top (angled top view):

Programmer with target board on top (side view). You can see the gold pogo sticks to the left of the right standoffs.

In the future, I will also write a more capable hex programmer with Nick’s code to do:

  • At the push of button 1, program target fuses and flash FLASH/EEPROM with the current time and date saved to target EEPROM. This way the target can program its own RTC based on the time of the programmer’s RTC, with at most a few seconds off. The rest of the program will self test target.
  • At the push of button 2, program target fuses and flash FLASH with a standard firmware.
  • Audio indicator of success/failure. With audio, you don’t have to watch the programmer. You can do other stuff and be reminded by success/failure tunes.
  • Programming mode (which standard firmware, what fuse setting etc.) based on the 8-dip switches.

I’ll post some videos when I get a chance.

My Review of Kingston SDC44GB 4GB microSDHC Memory Card

It works with Arduino. It’s slow on PC

By Liudr from MN on 11/16/2014

 

4out of 5

Cons: Slow On Pc

Pros: Reliable Performance, Fast Enough On Arduino, Easy To Use

Best Uses: Arduino, Maybe Raspberry Pi

Describe Yourself: Casual User

Primary use: Personal

For the price I paid for 10 of them, I am fairly happy. Each came with an adapter, which is nice. I think one or two of the adapters became unreliable. This may be the cause of someone’s grief. I have 10 so I swap adapters when I find the card is unreadable and narrowed down to two bad adapters. I’ve used these on my arduino and raspberry pi projects and for file storage. On arduino, if you use a very large capacity card, you waste a lot of time if you tally remaining free space so 4GB is perfect. I will be upgrading to larger sizes sometime soon but the class 10 16GB review is pretty awful. Maybe a different brand or it’s the adapter issue again.

(legalese)

Debouncing a mechanical rotary encoder

As an active member of the Arduino user community, I felt I should keep up with my library updates. So after some planning and “careful” coding, I’ve added a new digital rotary encoder class to my phi_interfaces library. The phi_interfaces library includes buttons, rotary encoders, analog buttons, joysticks, matrix keypads, serial ports, etc. as input devices and one can use getKey() to sense any of them.

https://liudr.wordpress.com/libraries/phi_interfaces/

The original rotary encoder class already works well. This addition expands the scope, supporting both normally open and normally closed encoders, and also supporting encoders that have twice the detents as they have complete pulses (have some hardware but haven’t tested yet). Also, this addition paves the way of an analog encoder class that will help people using nano/ATMEGA328P-AU save two pins, which has two analog-only pins with no digital functions.

Along the way I met some difficulties that I thought was useful to share. Although it stems from library development, the nature of the problem is actually misunderstanding of hardware.

Hardware I used: Bourns PEC11 series 18 pulses/rev. encoder with 18 detents./rev. (no obsolete, replaced by PEC11R)

So here is what I discovered: the mechanical rotary encoder that I have has a delay between channel A closing and channel B closing about 14 ms, then delay between channel A opening and channel B opening about 8 ms. Not only is opening asymmetric with closing, but they are also faster than my expectation. I started programming the new class expecting to debounce the encoder. I first used 20ms for debouncing, like what I do for tactile switches. That didn’t yield good results. I thought I need longer time, 50 ms, not good, or about the same, only 1/3 detents got detected. I then tried shorter debouncing like 10 ms and still few detents got detected. Running the original rotary encoder code I developed initially, without debouncing, was find, with every detent registered. So I checked the encoder with a logic analyzer. Wow the action was faster than I expected, 14ms and 8 ms. I also looked at the bouncing of the contacts. It was around 0.5 ms after initial contact was made, many bouncing occurred. So if I’m using millis() to debounce, I’d rather not debounce at all.

Here is one part of the pulse when channel A closes then channel B closes, about 14 ms:

Here is another part of the pulse when channel A opens then channel B closes, about 8 ms:

Here is a typical bouncing of the contacts, in the order of 0.5 ms:

An afterthought, I read the spec sheet, which says at 15RPM, close to how fast I was turning the shaft, bouncing is 5ms max. I’ve seen bouncing around that length in time. If I read the spec sheet, I’ve probably avoided the whole learning experience smiley

Here is diagram showing symmetric pulses on the spec sheet:

So, take the spec sheet with a grain of salt.

BTW, I’ve recently designed a system that uses one analog pin to sense a mechanical rotary encoder. The purpose was to save pins. The ATMEGA328P-AU has two analog pins that have no digital functions. Why wasting them if I can hook a rotary encoder to one of them?

Follow

Get every new post delivered to your Inbox.

Join 64 other followers

%d bloggers like this: