Augmented reality sandbox control box updated

This slideshow requires JavaScript.

I constructed a couple of these control boxes for my ARsandbox project using Arduino Micro and Adafruit Feather 32u4. The main goal is to make the interaction between a user and the software more intuitive. The two flashing buttons add and remove water from the simulation, which is pretty nice. I wanted to add another feature, to move the elevation up and down. Say you have a lot of sand in the box but you want to display some “water”. You have to remove a lot of sand (time consuming) from the box or edit a config file (near impossible for an average user). On the other hand, if I can change the base plane location in the calculation, I can achieve the above goal without removing sand or editing files. With help of the original developer Dr. Oliver Kreylos, I started messing with the source code. It took me a day to understand what I could do to change the base plane and how to do it. I ended up creating a tool to manipulate the plane. Now that the plane can be moved, I added a rotary encoder to the control box to emulate the keyboard keys assigned to move the plane. Voila!

The photo gallery above shows 5 shots of different elevation values. You can clearly see the “sea” on bottom right is shrinking after each turn of the knob.

Here is a short video. I was slowly raising the elevation so more and more land became above water. Then I quickly returned the land back into water:

Here is a photo of the inside of the box:

I added a breakout board for rotary encoder and buttons. It’s very tightly sandwiched between the two buttons. There is no space between the board and the buttons, not even a fraction of millimeter. I don’t know how these pieces just managed to fit. Here is the breakout board:

I made this a while ago as an interface for my phi-panels LCD backpacks. This breakout board is pretty simple. There are two encoder channels and one shaft button, plus six push buttons, which I am not using. I just connected to the encoder channels, shaft button, and the common. I had to cut off a strip on the right side of the board to fit into the box. I was using my phi_interfaces library to read the encoder and emulate a button push if the encoder is rotated one way, another button if the encoder is rotated the other way. I didn’t assign any function for the shaft button. I am thinking that I should use it to switch between elevation adjustment and water speed adjustment.

I haven’t updated the Adafruit Feature 32u4 version but will probably design a new printed circuit board so assembling will be easier, and not involving cutting off part of a board. Some code in case anyone wants to replicate it.


/*
 * Credit: Dr. John Liu
 * Purpose: This sketch emulates keyboard keys "1" and "2" with two push buttons with LEDs. It also flashes the LEDs
 * If a rotary encoder is present, it will emulate '5' and '6' if you rotate the encoder's shaft. This combined with my modification on ARsandbox source code will shift color mapping up and down.
 * This version uses p-channel mosfets and open drain with 10Kohm pull-up to 5V to control LEDs with the 3.3V Adafruit Feather32u4.
 * Notes: On Adafruit Feather 32u4, pin 9 is connected to battery sensing voltage divider.
 * 2018-01-28
 * Visit https://liudr.wordpress.com for more information
 */
#include "Keyboard.h"
#include <phi_interfaces.h>
#define USING_MICRO
#ifdef USING_MICRO
const int button_1=2; // 11 for Adafruit feather 32u4, 2 for Arduino Micro;
const int button_2=3; // 10 for Adafruit feather 32u4, 3 for Arduino Micro
const int EncoderChnA=7;
const int ChnCommon=8;
const int EncoderChnB=9;
const int ShaftBtn=10;
const int EncoderDetent=18;
#endif

#ifdef USING_FEATHER
const int button_1=11; // 11 for Adafruit feather 32u4, 2 for Arduino Micro;
const int button_2=10; // 10 for Adafruit feather 32u4, 3 for Arduino Micro
#endif

const int led_1=6;
const int led_2=5;
const unsigned long led_on_ms=300;
const unsigned long led_off_ms=1700;
const unsigned int button_1_key='1'; //KEY_LEFT_ARROW;
const unsigned int button_2_key='2'; //KEY_RIGHT_ARROW;
const unsigned int UpKeyOut='5';
const unsigned int DownKeyOut='6';

int prev_1=HIGH;
int prev_2=HIGH;
int led_stat=LOW;

unsigned long prev_1_ms=0;
unsigned long prev_2_ms=0;
unsigned long blink_timer_0_ms=0;
unsigned long blink_timer_1_ms=0;

int debounce_ms=25;

char mapping[]={'U','D'}; // This is a rotary encoder so it returns U for up and D for down on the dial.
phi_rotary_encoders MyEncoder(mapping, EncoderChnA, EncoderChnB, EncoderDetent);
//multiple_button_input* dial1=&my_encoder1;

void setup()
{
  // make pin 2 an input and turn on the
  // pullup resistor so it goes high unless
  // connected to ground:
  pinMode(button_1, INPUT_PULLUP);
  pinMode(button_2, INPUT_PULLUP);
  pinMode(led_1,OUTPUT);
  pinMode(led_2,OUTPUT);
  pinMode(ChnCommon,OUTPUT);
  digitalWrite(led_1,LOW);
  digitalWrite(led_2,LOW);
  digitalWrite(ChnCommon,LOW); // Using this pin as ground since some prototypes don't have enough gnd pins.
  Keyboard.begin();
  blink_timer_0_ms=millis();
}

void loop()
{
  unsigned char ch=MyEncoder.getKey(); // Rotary encoder emulates two buttons.
  if (ch==mapping[0])
  {
    Keyboard.write(UpKeyOut);
  }
  else if (ch==mapping[1])
  {
    Keyboard.write(DownKeyOut);
  }

  switch (led_stat)
  {
    case LOW:
    if (millis()-blink_timer_0_ms>led_on_ms)
    {
      blink_timer_0_ms=millis();
      led_stat=HIGH;
      pinMode(led_1,INPUT); // Open drain to let pull-up resistor pull drain to 5V.
      pinMode(led_2,INPUT); // Open drain to let pull-up resistor pull drain to 5V.
    }
    break;

    case HIGH:
    if (millis()-blink_timer_0_ms>led_off_ms)
    {
      blink_timer_0_ms=millis();
      led_stat=LOW;
      pinMode(led_1,OUTPUT); // Pull drain to GND.
      pinMode(led_2,OUTPUT); // Pull drain to GND
    }
    break;
  }

  if ((digitalRead(button_1) == HIGH)&&(prev_1==LOW))
  {
    if (millis()-prev_1_ms>debounce_ms)
    {
      prev_1_ms=millis();
      Keyboard.release(button_1_key);
      //Serial.println("1 released");
    }
    prev_1=HIGH;
  }

  if ((digitalRead(button_1) == LOW)&&(prev_1==HIGH))
  {
    if (millis()-prev_1_ms>debounce_ms)
    {
      prev_1_ms=millis();
      Keyboard.press(button_1_key);
      //Serial.println("1 pressed");
    }
    prev_1=LOW;
  }

  if ((digitalRead(button_2) == HIGH)&&(prev_2==LOW))
  {
    if (millis()-prev_2_ms>debounce_ms)
    {
      prev_2_ms=millis();
      Keyboard.release(button_2_key);
      //Serial.println("2 released");
    }
    prev_2=HIGH;
  }

  if ((digitalRead(button_2) == LOW)&&(prev_2==HIGH))
  {
    if (millis()-prev_2_ms>debounce_ms)
    {
      prev_2_ms=millis();
      Keyboard.press(button_2_key);
      //Serial.println("2 pressed");
    }
    prev_2=LOW;
  }
}

Augmented reality sandbox control using Arduino Feather

It’s been a while since I completed this project. I promised the Augmented reality sandbox community to write something in details so someone can replicate it. Here is the gist of it:

The Augmented reality sandbox (ARsandbox) is an awesome simulator that blends sand landscaping with augmented reality. A projector casts color-coded elevation, contours, and simulated water flow over regular sand. A 3D sensor measures the sand landscape in real time so the computer knows where sand is high and how it slopes down and renders color and contours according the the 3D data. You can rain over the landscape by pressing a button or hold out your palm over a certain area.

Here is a video of one of my builds:

This is where the control box comes in. The sand box is a 40″ by 30″ box with up to 200lb of sand. The function to rain over the augmented terrain is to press the “1” key on a keyboard. Sand and keyboard shouldn’t mix, not to mention if the system is deployed in a museum, the keyboard probably should be hidden from patrons. So a box with rain and dry buttons will be most convenient. You can make these buttons by following this forum post. You buy and assemble a USB gamepad kit and doing some scripting but it’s not that easy to follow if you are a Linux beginner. I want a plug-n-play solution so that anyone that wants to add a button box can add it without any knowledge of Linux and minimal skills in circuit assembly. In order to achieve my goals, the electronics inside the box has to emulate keyboard keys “1” and “2” for rain and dry. This ensures that the control box requires no software setup on the Linux system. As a bonus, the buttons should flash blue and orange so it would invite patrons to press and is color-coded for rain (blue) or sun (orange). Enter the ARsandbox control box:

Here is a short video of the box in action:

Features of the box:

1. Two buttons. Blue button on the left emulates the “1” key. It starts the rain in the main simulation, and is used in the projector calibration program to enter tie points, and can be assigned other functions in other programs such as extracting plane. Orange button on the right emulates the “2” key. It drys the rain in the main simulator, and can be assigned functions in other programs.

2. Both buttons have LEDs that flash to invite patrons to press. My 5-yr old son figured out what they do simply by the colors they flash because he knew the rain and dry features already but wasn’t told what these buttons would do.

3. No software setup is needed. This box works like another keyboard, with only 2 keys though.

 

Here are the parts I used, in case you want to build your own:

  1. Adafruit Arduino feather 32u4 basic board (1)
  2. LED push buttons 16mm from adafruit (2)
  3. USB-micro cable (1)
  4. USB extension cable (1)
  5. P-channel MOSEFT BS250 from mouser (2)
  6. 330 ohm resistors (2)
  7. 1K ohm resistors (2)
  8. DuPont 30cm female-female jumper wires (4)
  9. Prototype board or my printed circuit board. Click here to download (1)
  10. Enclosure SK-15 and flange kit SK-99 from polycase (1)
  11. Cable gland M3198GBH from Heiland for pre-assembled cables with USB connectors (1)
  12. Arduino sketch. Click here to download

Here is the schematic of the circuit:

Since the ATMEGA32u4 microcontroller on Adafruit Feather 32u4 is operating at 3.3V, I had to use two transistors to ensure that the LEDs inside the buttons light up with sufficient brightness. Currently the code only blinks both button LEDs at the same rate. In the future, I may update the code to blink more rapidly on the button that is depressed.

The following is a prototype that I built with the parts on a perfboard. It’s a bit messy and also took almost 2 hours to complete. I had to solder 16 wire leads and a bunch of jumper wires on the back side of the perfboard. I wouldn’t recommend this to a beginner.

Here is the printed circuit board that I designed (3 boards, two on the right showing the top, 1 on the left showing bottom):

The board is fairly simple and straightforward to solder. Only 8 wire leads and soldering the rest on the printed board, which is very easy. I also cut in half 4 30cm female-female Dupont jumper wires so I can just use the female side with male pins on the board to cut wire leads in half.

Here is a photo of the assembled guts including the microcontroller soldered to the button box board. Notice the black female Dupont connectors :

Here is the assembled board placed inside the enclosure. This enclosure is not cheap but nice and easy to use. It has various knock-outs so I didn’t have to drill a single hole.

Here is the completed box:

Although I am not aiming to sell a lot of these boxes, I think that designing the printed circuit board was the right way to go. I probably spent a few hours designing and proofing my design but I saved about one hour for each subsequent box I made. Plus, if YOU are interested in making this box, it will save YOU at least that much time.

I2C LCDs

There are many types of I2C character LCDs on the hobby electronics market. To design my new open source data loggers and Phi-3 Arduino shield, I decided to move away from the bare parallel HD44780 character LCDs and go with character LCDs and I2C backpacks (aka I2C LCDs). I found out a few popular designs and thought that I would summarize them for your convenience.

Most I2C LCDs are based on the following two ICs, all of which are I2C I/O (port) expanders:

  • PCF8574 or PCF8574A
  • MCP23008

Hardware:

Both ICs have 8 I/O pins. MCP23008 is more versatile but that is irrelevant to simple applications in LCDs.

Adafruit designed an I2C LCD backpack and Arduino LCD shields based on MCP23008. There are compatible devices sold on ebay. I can’t tell without seeing the sample code to decide whether an ebay seller is actually selling a compatible product. But if you do want to get one on ebay, make sure you find their library code and confirm that the library contains Adafruit’s names. Libraries you find from ebay sellers are likely out of date though. One good thing is that the compatible ones are very likely using the same pin assignments as Adafruit’s so it’s easy to get it to work once you get the library installed.

FM (Francisco Malpartida) designed an I2C LCD backpack based on PCF8574. There are lots of compatible devices sold on ebay and they don’t have the same pin assignments! This creates issues when you are making purchases thinking that they have certain pin assignments. The pin assignments refer to which PCF8574 pin is connected to which HD44780 display pin. Also the I2C addresses are all different. I don’t mean one might have an address of 0x3F and another might have 0x3E. What I mean is that one might have 0x3F and another one may be 0x20. There is no way to set one display that has address 0x20 to address 0x3F! PCF8574 has address space of 0x20 to 0x27. PCF8574A has address space of 0x38-0x3F. Most common addresses I’ve seen are 0x20, 0x27, and 0x3F, with the latter two sharing pin assignments that are different from the ones with 0x20 address. Most of these displays allow you to cut traces or solder pads to change addresses. Why would you if you don’t have multiple LCDs?

Software:

Adafruit has its own library Adafruit_LiquidCrystal. This library is decent. It can take different pin assignments as parameters. On the other hand, it is a different library than Arduino’s included LiquidCrystal library. So code you wrote for LiquidCrystal library may need some change when you switch to an Adafruit compatible I2C LCD.

FM wrote a library New LiquidCrystal. This library is pretty good. You can use a number of different LCDs including parallel HD44780 LCDs, I2C LCDs using PCF8574, LCDs using shift registers etc. A nice feature is that there is a base class LCD so regardless what actual type of LCD you are using, as long as it’s supported by this library, it works the same way on the software level as another supported LCD.

Since not all PCF8574/74A I2C lcds have the same pin assignment, or even back light polarity, using the correct definition will be crucial. I found the following three definitions. Each seems to work with the particular I2C address, although there is no relation between I2C address and how the pins are assigned (by circuit designer):

The first two work on backpacks that look like this:

Notice that only the address is different. Pin assignments and back light polarity are all the same.

LiquidCrystal_I2C lcd(0x3F, 2, 1, 0, 4, 5, 6, 7, 3, POSITIVE); // Blue potentiometer with back light jumper.

LiquidCrystal_I2C lcd(0x27, 2, 1, 0, 4, 5, 6, 7, 3, POSITIVE); // Blue potentiometer with back light jumper.

The last definition works on backpacks that look like this:

Notice that pins are very different and back light polarity is negative.

LiquidCrystal_I2C lcd(0x20, 4, 5, 6, 0, 1, 2, 3, 7, NEGATIVE); // Tiny mental potentiometer no back light jumper.

Detection:

In case you can’t determine the address or pin out, say none of the above definitions work, but you’re sure the IC is PCF8574/74A, you should first scan the I2C bus for the address, and then use your meter to map out the pin assignments, and then use your definition. The lcd constructor has the following parameters: lcd(add, En, Rw, Rs, d4, d5, d6, d7, Bl, Pol).

Here is the I2C scanner I use by Tod E. Kurt:

https://github.com/todbot/arduino-i2c-scanner/

 

Phi-shield revised and released

phi-3-shield-on-in-hand

It has been a while since I gave the phi-shield a major revision. I’ve been working on this for a while and now I am releasing the Phi-3 shield. This shield continues to support user interaction with LCDs and buttons. Here is a list of the features:

The following hardware are provided by the shield:

  • 20X4 LCD with back light on/off control
  • Six buttons (up/down/left/right/B/A)
  • Two LED indicators
  • Speaker
  • MicroSD card slot
  • Real-time clock (DS3231)
  • EEPROM (32KB 24LC256)
  • Connector for Adafruit Ultimate GPS module or Bluetooth module
  • Stacking headers for easy access to all pins.
  • Recessed board right edge for easy access to MEGA’s 18X2 pin headers on the right side.
  • Reset button

phi-3-shield-lcd-side-by-side

The following software functions are provided by various supporting libraries:

  • User-selectable menu (LCD + buttons)
  • Number and text entry (LCD + buttons)
  • Scrollable long text (LCD + buttons)
  • Date and time (DS3231 or GPS)
  • Location (GPS)
  • Data and configuration storage (MicroSD card and EEPROM)
  • Playing simple tones (speaker)
  • Indicators (LEDs)
  • Wireless connection (Bluetooth module)

phi-3-shield-lcd-removed-annotated

There are three tiers of Phi-3 shield kits: kit0, kit1, and kit2, none of which includes a GPS module. The kits are immediately available. Buttons with color caps as pictured will be included while supplies last.

Here is the Phi-3 shield’s own page. There are links on the page to make purchases. Or you can visit the BUY page to see what stores carry this shield.

Phi-3 shield

Video demonstrations will be available next week. Meanwhile, the support of Phi-2 shield will remain. If you need Phi-2 shields, I have them available.

phi-3-shield-bottom-rtc-lcd-wire-removed

Directional keypad for phi-panel

This is a directional keypad I developed for phi-panel, together with the rotary encoder keypad.

There are two sizes, 20X4 and 16X2.

Features:

  • Laser-cut 3mm-thick glossy/matte black acrylic face plate with window to mount a 20X4 or 16X2 character LCD and a 8 push buttons. You choose which side to use, glassy or matte. Brown/white color is removable protective layer on both sides.
  • 4 push buttons are arranged in up/down/left/right
  • 2 buttons above the directional buttons and 2 above.
  • 8 short standoffs for LCD and keypad circuit board, 4 long standoffs for the face plate, 12 screws and 12 nuts, all M3 in size
  • 6 button covers in case you want to cover up the button holes you don’t use
  • PCB for the keypad and 8 buttons

Currently the 20X4 version is out of stock. I only made 2 in a batch of 4 new keypad face plates. If you want the 20X4 kit please respond with a comment.

Initial tests are all positive with phi-panel functions with new firmware. The directional keypad is very nice if you don’t need the 0-9 numbers provided by the matrix keypad. There is no more setup menu you can call up with the escape key.

It took me forever to make those white connectors. The female crimp end was so hard to make that if you add just a bit too much solder the spring is soldered on and can’t accept pins. These are not the best ways to connect the keypad to phi-panels. I did it only because I want to switch between keypads between photos and videos so I don’t have to tie up two assembled panels with keypads.

In the kit, I included regular male headers. You can simply solder wires to the pins or use male headers and wire wrap to the pins for connection, much easier than the white wafer connectors.

I am writing documents for these keypads but you can already purchase these kits and tinker with them. It’s pretty easy to hook them up to phi-panels or your arduino in case you don’t want a phi-panel.

Videos (20X4 version):

Still shots:

New phi-panels documentation

I have been procrastinating but there is no shame to release a documentation with one or two things not yet documented, out of hundreds that have. So here we go. This documentation matches firmware version 1.6.2

http://code.google.com/p/phi-panel/downloads/detail?name=Phi-panel-serial documentation 20121029.pdf

New functions are added to family of phi-panels

Arend requested some new features that I thought was useful to everyone:

1) Have up to 32 items on a menu/list

2) Have a way to set which item is high lighted so the user will not have to go through the entire list from item 1.

This resulted in quite a bit of work and time (I have about 3,000 lines of code that I have not touched for a while). Anyway now let’s welcome firmeware 1.6.2 with the following additions:

Dennis also requested some discrete button options but that will not be released to everyone (unless you all want that).

Updates:

1) Added support for 8 discrete buttons with back_pack version. Firmware will be uploaded in a separate file

2) Updated EEPROM key to 0x61 for back_pack_8_keys and 0x62 for this version

3) Added support for up to 32 items on a list.

4) Added support for up to 660 characters for long messages or menus.

5) Added option to preset highlighted menu item with menu_highlighted_item. It updates with each selection.

6) Added a serial command to set this menu_highlighted_item value with “\eWn~”. Eg. “\eW3~” highlights item 3 (1 based) for the next menu phi-panel receives.

Download the firmware here if you want to get the update and MAKE SURE YOU DOWNLOAD THE RIGHT ONE FOR YOUR PANEL OR BACKPACK SIZE.

First read the wiki about how to upload firmware then download the correct 1.6.2 version for your hardware 🙂

http://code.google.com/p/phi-panel/downloads/list

Arduino Emulation on PC

I have worked on this for a while and was able to run arduino code on my PC. This is to make debugging much easier and compiling much faster (1 second compared with minutes of compile and upload). Since the purpose was to develop phi_prompt programs, I only made emulations on key Arduino functions, liquidcrystal, DS1307, phi_interface, and phi_prompt libraries. There are no real hardware hooked up to arduino. Future development may include some limited feature such as digitalWrite or other hardware that I might need for projects. Here are some pictures to accompany my previous post about the professional development kit for phi_prompt:

The supported Arduino stuff is here:

millis();
microSeconds(); //Good luck with this on a multitasking operating system!
delay();

PROGMEM related definitions and functions. Seems to be working with “c-strings in PROGMEM and a pointer array in PROGMEM to point to these strings”. I have not tested other data types. Be careful with this though, pitfalls may still exist
strcpy_P();
strlen_P();
strlcpy(); // This is only available with GNU C not a standard one but I need it.
pgm_read_byte();
pgm_read_word();

LiquidCrystal library. Will work on write(char), print(int) and print(float)

Everything except for below functions, which I don’t use
        void noDisplay();                                                         ///< Turns off the display
        void display();                                                           ///< Turn on the display
        void scrollDisplayLeft(); // Hate these scroll functions. Useless
        void scrollDisplayRight();
        void leftToRight();
        void rightToLeft();
        void autoscroll();
        void noAutoscroll();
        void createChar(uint8_t, unit8_t*); // I can’t create char on PC

DS1307 all working except for this one I didn’t implement. Easy to do though. Also start and stop have no actions. You can’t change DOW without changing dates.
    int get(int, boolean);

phi_interfaces input device library only has the phi_serial_keypads class which takes inputs from keyboard. Function keys are wsadfg of course (who didn’t play some FPS games?)

phi_prompt user interface library works except that the right arrow looks different and the scroll bar is not as awesome as on real LCD

Phi_prompt advanced features video demo

I have developed many features with my famous phi_prompt user interface library. These features are too specific to be turned into library function calls but I have a well-documented set of examples for lots of these features that will open your eyes.

These feature code and some professional consultation will be offered under “Arduino User Interface Professional development kit for phi_prompt” together with a PC emulation engine that you can emulate arduino and tweak on a PC with debugger and use your keyboard to operate the emulated code and copy paste to arduino to load to arduino when you are finally satisfied. It saves you tons of time for waiting for uploads and saves your hair from pulled off when debugging with arduino IDE.

Here is a video, take a look at these eye candies features:

I’ll be adding more features including variable length of list (say you adjust one parameter and the list expands to show its details).

By no means can I support newbies with these features that can’t even be turned into generic library function calls but to more experienced people trying to make money with their products, these features add values. Hope I can use fund I get from professional help with professional projects using these features to support the free end of my library (actually all library function calls are free to use but how you use them to make advanced features are not yet open).

Phi-panel face plates

Face plates are coming out! These are white 3mm acrylic face plates made from laser cutting. I didn’t choose color since I thought I could just paint the white to any color I wanted. Well, I don’t really have a spray painting can :((

If you want to make them, just leave a message here. I’ll be posting the design file in a while.

This is the design file in Corel Draw 11 (super old version). It should export into the .svg file for laser cutting at ponoko.com with the 180mm*180mm service (cheapest) if you also have Corel Draw. Otherwise, download the .svg file. I am not sure what software to use to modify .svg if you want some changes though.

To make it, set up an account at ponoko.com and choose acrylic (3mm thickness suggested).

Contains 2 16×2 face plate, 1 20×4 face plate, and one arduino uno support.

Corel Draw file:

http://code.google.com/p/phi-panel/downloads/detail?name=phi-panel_faceplate_v3.cdr&can=2&q=

Exported .svg file:

http://code.google.com/p/phi-panel/downloads/detail?name=phi-panel_faceplate_v3.svg&can=2&q=

Suggested push buttons: EG1861-ND, caps: EG1882-ND, EG1881-ND, EG1880-ND. All these are digikey.com part numbers. These are the cheapest ones I could find.
Suggested spacers: M3 thread 8mm spacing and M3 screw: 25509K-ND, H742-ND.

%d bloggers like this: