A couple weeks ago I shared a project that gives the blind sonar. It’s turned into one of my more poplar projects. At the time I wrote a pile of words about it but I didn’t tell the story of how it happened. Which regular readers will know is unheard of.  So here’s the story. I think it’s as important as the end result, though for different reasons.

Last year John Park posted a video on the Make: Blog showing the WaveShield, a bit of electronics that plays back sounds on demand. He had hooked up to a joystick to make a sort of quicky sound board. (The article and video is here.)  It’s a nifty project and it got me thinking about what I would do with the same components—something that can talk and a joystick. Since it had a joystick and I’ve spent much of the last decade or so doing video game design and development the obvious idea was to make an audio-only video game.

The first idea on the front door of my imagination was to record a bunch of concatenated audio and make a dungeon crawler. “You are in a small room. There are walls to the east and west. There is a door to the north. There is a grue to the south.”  The idea made me smile. I’d also written a little bit of radio drama way back when, and the audio-only adventure appealed to me.

But when I started thinking about it, I knew it would be very slow to play. Every time the player moved a step they would listen to a full situation report. (>Move east “You can’t move east. You are in a small room. There are walls to the east and west….” Etc, etc, etc.) Navigating menus would be a unique challenge. This applied to almost every other game type I could think of.

Hmm.

But I still really liked the idea of a game that didn’t rely on vision. And one of the great things about working with microcontrollers is that you can plug in all sorts of things. Like vibration motors, which I happen to have a pile of. If a player could tell where things were by sensing vibrations I could cut out most of the repetitive, long-winded audio.

Elastic headband with vibration motors (housed in the large binder clips) and ultrasonic sensors (Attached with the small binder clips). It looked like even more of a nightmare when on ones head.

This seemed like a good enough idea to try. When you’re in unknown territory it’s best to work iteratively, taking small steps, so I came up with a basic way to test the idea of vibration-based navigation first.  So I hooked up four vibration motors, like the one that makes your phone vibrate, to a strap of elastic and wrapped it around my head, and whipped up a virtual maze inside an Arduino microcontroller. A quick test showed it worked!  Not great, but sort-of. Not bad for a couple hours work.  And the idea of wandering around a maze by touch got me excited. Maybe it would be possible (and not too difficult) to make a two player game where both players were in the same maze, blind, searching for each other.  Sounded like fun and a great interactive installation.

I spent a few evenings trying to get the maze navigation smooth and natural.  It turned out to be very difficult.  Moving around naturally with a joystick was a lot more difficult than in a vision-based game.  It was difficult to get a sense for how fast you were moving and smaller subtleties were missing. I was getting frustrated.  I wrote and rewrote code, swore, took an angry nap, played with the idea of adding accelerometers, compass modules and other complications.

What if my idea was a stinker? What if it was unworkable?

Maybe it was.  To decide if it was workable or not I needed to find the best case for navigating with the headband.  The “best case” being what would it be like to walk around the real world using only vibration motors on my head.  It turns out it wasn’t too hard to hack it together.  The fine folks at Parallax make an easy to use ultrasonic sensor and I had several lying around. In about an hour I hacked them onto my headband and ended up with something that would make Peter Venkman jealous. It read the distance in each direction with an ultrasonic pulse and vibrated the motor faster if something was close, slower if it was far.

It worked!  And walking around a dark house with my eyes closed I suddenly realized I was seeing without using my eyes! That was much more fascinating than games and mazes and slaying beasts in dank dungeons! The grue would have to find someone else to eat, I was off in a new direction.

Well at least I was for a few minutes after which I  had to take the thing off since I had strapped 4 constantly vibrating motors to my head and they drove me batty. I also had a stubbed toe and a bruised hip from hitting obstacles that weren’t at head height—but it worked!  A whole new (to me) way of sensing the environment!

It had problems, pretty well documented in the original article. I chased after them, replacing the vibration motors with pressure from servos and moving the whole thing to the hand to better sense objects at shin level.  It still can be better, but it’s good. In developing it I got a crash course in haptics, the perceptions of the blind, the current state of assistive technology, and biology lesson. I’ve met a ton of smart people who know more than I do, but who have been generous with their time and knowledge.  It has evolved into a truly helpful project that has interested a great many people, and hopefully will help many more.

Why do I think that’s an important story?

Because, as my grade-school art teacher Mrs Hoover said, booboo’s can be beautiful. In other words: When reality doesn’t match expectations, chase the outcome, not the expectation.

Read the story of how the Microwave oven was created by Percy Spencer while working on radar. Or how the Slinky was created while Richard James was working on stablilizing shipboard instruments.  Or how Teflon was discovered by Roy Plunkett when he was researching refrigerants. Silly putty, X-rays, penicillin, Corn Flakes, Super Glue, Play-doh… Out world is filled with results of this process. Sometimes it can be hard to turn away from the effort we’ve put into a project, it can be crushing when our hard work doesn’t pan out.  But it’s only if we continue to smash our heads against reality that our work is truly wasted.

• www.facebook.com/bibi915 David Mejia

  HOW INTRESTING  THANKS

• Robert

  Thank you for this write up Steven. I love watching how your trial and errors evolve your project into something even more interesting and useful. One of the hardest part of making things is when my results don’t meet up with my expectations for the project. It was fascinating to watch you pick up the interesting parts to make something new. Thank you for making your work open source, I hope it inspires many more assistive technology projects.

• JT

  Saw an article on your device.  Nice invention.  Congrats!  I dabble with micro electronics myself and have used the Parallax Ultrasonic sensor myself.  One problem that you are going to have to address is that the sensor, since it’s based on sound waves, really irritates animals in the immediate area.  I have cats.  When I turn on a project that uses this sensor, they go crazy trying to get away from the immediate area.  They act like they are being hurt in some way and run away.  Being ultrasonic, that makes sense.  I have not used this sensor since I found that.  I have not tested it on dogs.  But blind people might have pets not to mention helper animals that could be affected like this too.  Might have to swap out the sensor for another that is not ultrasonic in nature.

  • grathio.com Steve Hoefer

    Hi JT,

    I did some pretty extensive tests around dogs of multiple breeds and never got a reaction from any of them, but we never though to test around cats. I’ll see if I can find some cats who are willing to test around it.

    Other ultrasonic sensors use different frequencies and might give you better luck.

    As far as other sensors go a laser range finder would probably provide the best results with out much interference, but they are vastly more expensive.

    I’ve been contacted by people who have had luck with IR reflectance sensors, but I haven’t been able to reproduce their results. They always had way too much noise outside.