This is a project I’m calling Tacit. No, I didn’t bother making an awkward backronym for it, it just seemed like an appropriate name that’s a lot shorter (though less descriptive) than “Hand-Mounted Haptic Feedback Sonar Obstacle Avoidance Asstance Device”. It measures the distance to things and translates that into pressure on the wrist.
It’s wrist mounted and senses objects from about 1 inch (2 cm) to 10 feet (3.5m). It has generally fast response time (fractions of a second) to quickly navigate complex environments. It’s designed to help a vision impaired person to navigate complex environments. Mounted to the back of the hand, the force feedback means it doesn’t interfere with other assistance devices that mount elsewhere and use audio feedback cues. The learning curve is measured in seconds, everyone who has worn it has figured it out immediately.
Here’s a short video that give more information and shows it in action.
Click to enlarge
This is the first public prototype. It’s not perfect, but it works, and it can be better. For example it could easily be made about half the size, and the replaceable batteries should be replaced by rechargeables with a blind-friendly charging method, either wireless or magnetically-aligning power plug.
This is vastly different from the first version which was a headband, very similar to the (independently developed) haptic headband. Just like Instructables user polymythic, my headband had ultrasonic sensors and vibration motors around the circumference of a headband with the motors vibrating faster the closer they came to an obstacle. I looked at infrared sensors too but found that sunlight, remote controls, security cameras, and absorbent surfaces easily confused the sensors. Ultrasonics are much larger and harder to focus but they’re good in just about any environment. I plan to do some more tests with IR using different wavelengths and polarizing filters, but for now ultrasonics work best at this price. Lasers would be the most accurate, but would increase the cost dramatically.
The headband was a great first test, and it did work, but it had two fatal flaws:
- The most dangerous obstacles are not at head level. Furniture and most of the other things that can be tripped over and stubbed on are waist level or lower.
- Vibrating motors stuck on your skull will drive you insane quickly.
In addition it would be a challenge to disguise it as anything but some mad-science-looking headband, and blind people do care how they look.
I looked at a number of solutions to getting the sensors to look in a more useful direction. For the blind, vision-simulating sensors don’t need to be attached to the head, that’s a sighted prejudice. There is not shortage of other projects to look at, like haptic vests and vision processing systems, but they were generally cumbersome and universally very expensive. I don’t see the point of an accessibility device that has an inaccessible price tag. (The cost of materials in the prototype is around $65USD retail.) In the end mounting it to the hand was the best solution. It’s in about the right location, lets the wearer quickly and easily point it at anything they’re curious about, and generally won’t interfere or be interfered with by most clothes and other limbs.
Even though it’s no longer head mounted the vibration motors are still a problem in several ways. They’re noisy and audio feedback is important for the visually impaired. More importantly even small vibrations, if they’re constant, can cause nerve damage. The idea of solenoids came up since they can be small and have a fast response time, but they are either on or off, and don’t provide any in-between. Other solutions like pneumatics and linear actuators were slower than I wanted. There are some experimental muscle materials out there which are intriguing, but not generally available at the moment. After some testing I decided to use some small servomotors with cushions on the ends, these can be quickly and precisely positioned in a lot of positions and give higher fidelity than solenoids. And small ones are cheap.
Designing the supporting gauntlet to hold the hardware went through a number of iterations as well. To the visually impaired the sense of touch is important, so gloves were out, even fingerless ones as the fingers and palms have some of the highest density of touch-sensitive nerves in the body. (For the curious, there are three different types of touch-sensitive nerves, Merkel cells which detect pressure, Pacinian and Messiner’s which sense the start and finish of touches.) I decided to use neoprene as a base fabric since it’s a solid, durable, shock absorbent base for electronics and is flexible and stretchy so it won’t resist movement. It comes in tons of colors (The prototyp ie red because that’s what was on sale at the fabric store.) This prototype uses 3mm thickness, but future versions will use 2mm since it can get a little warm after extended wear in warmer weather.
The first design was crafted after a medical wrist support. It was anchored at the thumb and wrapped around the wrist. This didn’t provide enough support for the electronics, required a different version for righties and lefties, and was awkward to get on and off, even for a sighted person. Its latest form uses a middle finger loop to support the device and a velcro wrist strap. This is took care of the handedness problem (Though it’s still slightly better on the right since velcro is less likely to catch on the wearer’s clothes.) It’s also one-size fits most and simple to wear correctly using only touch. The version shown makes some stylistic sacrifices to maintain easy access to the hardware inside. Keep reading for more information, build notes, parts list, schematics, and code. The electronics are very simple, they’re all self contained units with no other passive components.
Important Note #1: This is not a hard project, but it’s not for beginners. Read through everything carefully before starting and do not try to build this project if you do not undertand it fully.
Important Note #2: (February 12, 2012) The circuit and diagrams are released under a Creative Commons Attribution 3.0 Unported license. Feel free to use, modify and distribute as long as you attribute it to me. The source code is released under the MIT License which says pretty much the same thing. This is an change from the original, more restrictive (and less appropriate) by-nc-sa license and makes the project fully Open Source. Feel free to contact me if you have any questions.
Parts:
- (1) Arduino Mini Pro 5v (I also use a FTDI 5v basic adapter to program the Arduino but any 5v FTDI cable should work.)
- (2) Parallax PING))) ultrasonic sensors.
- (2) small hobby servo like a Turnigy TG9 or Hextronix HXT900 (there are other identical servos with only the label being different.)
- (1) 9v battery connector
- (1) slide switch.
- Shapelock or other Polycaprolactone plastic to build solid mounts for the sensors and servos. (It’s plastic that melts in hot water and can be formed by hand.)
I used some headers and bits of perfboard to make it easier to modify parts for prototyping, essentially making the whole thing a shield for the Arduino Mini, but they’re not necessary, especially if you’re a skilled and confident solderer.
The pressure applying parts are made from:
- Small pieces of polyester sheeting, heat formed.
- 6″ (15cm) of 1″ (2.5cm) wide rubber (I used a replacement squeegee blade from the hardware store. Gasket material might work well too.)
- (4) 2.5mm x 8mm bolts with matching nuts and washers to hold the rubber in place.
- Epoxy to hold the polyester sheet to the servo horn.
The gauntlet is made from:
- (1) 12″x12″ (30cmx30cm) piece of 3mm neoprene.
- 12″ (30cm) of hook and loop fastener (Velcro)
- 4″ (10cm) of bias tape.
Warning: Neoprene is a big challenge to sew. It likes to drop stitches like crazy on a sewing machine. Use the biggest needle you can, thick thread, and if possible place a non-neoprene strip where you sew. (Special thanks to Bethany Shorb of Cyberoptix TieLab for the sewing tip!)
When sewing the gauntlet here’s a couple things to keep in mind:
1) Don’t attach the electronics permanently to the neoprene. Use snaps or velcro or something so that it can be easily removed and laundered. Stuff that is near the hand can get dirty surprisingly fast. When attaching velcro to the electronics or plastic supports, super glue worked better than any other adhesive or epoxy.
2) Be serious about ruggedized connections and strain relief. Sewing electronics into flexible cloth means they have to flex and not break. No matter how much work you put into this, it could use more durable flexibility. Wire can take a lot of bending, but joints and connections can’t. Everywhere I could manage to I looped the wire through a hole or two to releve the flex strain before making the final connection. (This lesson courtesy of the Rock Paper Scissors Playing Glove, which ripped its self apart every 500 uses despite a fair amount of effort put into ruggedization.)
The basic connections look like this, though this is mostly to keep things modular and accessible during the prototype stage and can be improved.
The circuits are pretty simple. Here’s a rough diagram and schematic:
| |
|
Arduino pins:
RAW: Positive (+) side of the 9v battery and power pins on the servos.
GND: Battery ground (-) and grounds for servos, ultrasonic sensors.
VCC: Ultrasonic sensors +5V
2: Right ultrasonic sensor SIG
3: Left ultrasonics sensor SIG
7: Right Servo signal.
8: Left Servo signal.
Here’s the original version of the source code.
Don’t cut and paste this text, it won’t copy correctly. Instead get the latest version from GitHub at this link.
// Tacit, Wrist mounted tactile feedback for the blind.
// By Steve Hoefer at Grathio Labs (grathio.com)
// Version 12.02.04
//
// Copyright (c) 2012 Steve Hoefer
// Permission is hereby granted, free of charge, to any person obtaining a copy of this software and
// associated documentation files (the "Software"), to deal in the Software without restriction,
// including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so,
// subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all copies or substantial
// portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT
// LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN
// NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
// WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
// SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
//
//
// Written for Arduino authoring environment version 0022 and a Arduino Mini Pro 5V but should work on any Arduino/Arduino compatible that provides 5 volts.
//
// This version supports the following hardware:
// Parallax PING))) ultrasonic sensors for range finding
// Connect the GND pin to ground, +5V pin to +5V, and SIG to pin 2 or 3.
// Just about any small hobby servo. (Specifically Turnigy TG9)
// Connect the ground to ground, +V to RAW and the signal to pins 7 or 8.
//
//
// Version history:
// 11.08.07 - original
// 11.08.14 - Added code so servos apply constant pressure even when readings are not changing.
// 11.10.11 - Added pause after each sensor reading to fix reported issue with left sensor occasionally giving garbage responses.
// 12.02.04 - Changed license from CC BY-NC-SA to the MIT license.
#include
const int MaxSensors = 2; // The number of sensor/servo pairs.
const int ServoPins[MaxSensors] = {7, 8}; // The pins they're on
const int RangingPins[MaxSensors] = {3, 2}; // The pins they're on
const int ReadingsPerSensor = 5; // The number of historic readings to consider when determining position.
const int TimePerDegree = 9; // ms per degree rotation on the servo to prevent servo motor electrical noise from interfering with the ultrasonic sensor readings
const int MinimumTurnDistance = 3; // Minimum number of degrees that the servo will turn. Keeps the servos from being too twitchy.
// Variables
Servo ServoList[MaxSensors]; // Array of servo objects for manipulating easily.
int sensorReadings[MaxSensors][ReadingsPerSensor]; // Hold past readings for each sensor.
int calculatedSenorReadings[MaxSensors]; // The calculated distance for each sensor.
int latestReading = 0; // Current position in the array for the most recent reading.
int servoLocations[MaxSensors]; // The current position of each sensor.
int SenorClose = 500; // Closest value we detect with the PING sensor. (Soundwave travel time in milliseconds.)
int SensorFar = 14000; // Furthest distance we register on the PING sensor. (Soundwave travel time in milliseconds.)
int ServoClose[MaxSensors] = {0, 160}; // Angle the servo turns to when something is closest.
int ServoFar[MaxSensors] = {70,110}; // Angle the servo turns to when something is at its furthest.
void setup() {
//Serial.begin(115200); // Uncomment the Serial.foo lines for testing.
//Serial.println("Begin...");
// Initialize the servo location and move them through a full range of motion so we know they work.
for (int i = 0; i < MaxSensors; i++){
ServoList[i].attach(ServoPins[i]);
delay(10);
ServoList[i].write(ServoClose[i]);
delay(500);
ServoList[i].write(ServoFar[i]);
delay(500);
ServoList[i].detach();
}
delay(100);
}
void loop(){
int i, j, oldLocation;
unsigned long delayTime;
// Loop through each range sensor
for (i = 0; i < MaxSensors; i++){ // Get the current sensor's range. sensorReadings[i][latestReading] = getDistance(i); // Figure out an averaged/smoothed readings based on this and past data. calculatedSenorReadings[i] = calculateNewDistace(i); // Set the servo to the correct angle. oldLocation = servoLocations[i]; servoLocations[i] = map(calculatedSenorReadings[i], 0, 100, ServoClose[i], ServoFar[i]); if (latestReading >= ReadingsPerSensor-1){ // Don't do anything until we have enough data to trend.
if (abs(servoLocations[i]-oldLocation) >= MinimumTurnDistance){ // Only try to turn it if we have somewhere to go.
ServoList[i].attach(ServoPins[i]);
delay(10);
ServoList[i].write(servoLocations[i]);
delayTime = (TimePerDegree * (abs(servoLocations[i]-oldLocation))+20); // Set a delay for the next reading so motor noise doesn't interfere with senor readings.
if (abs(delayTime)>500){ // If it can't do it in this amount of time // It's based on how far it has to turn to keep the delay to a minimum, response time at a maximum.
delayTime=500; // we'll get it next time. Keep it responsive.
}
delay(delayTime);
ServoList[i].detach();
} else { // Otherwise if the reading hasn't changed enough write the old value to
ServoList[i].attach(ServoPins[i]); // the servo so that it will hold in place if it's applying pressure.
delay(10);
ServoList[i].write(oldLocation);
delay(50);
ServoList[i].detach();
servoLocations[i]=oldLocation;
}
}
delay(20); // Added to fix left sensor misbehavior reported by Rob.
}
latestReading++; // Increment the reading counter so we know where we're at.
if (latestReading >= ReadingsPerSensor){ // Make sure we don't record more readings than we have space to hold.
latestReading = ReadingsPerSensor-1;
// Pop the oldest reading off the list.
for (i = 0; i < MaxSensors; i++){
for (j=0; j < ReadingsPerSensor-1; j++){ sensorReadings[i][j] = sensorReadings[i][j+1]; } } } } // function: calculateNewDistace(sensorNumber: Which sensor's data to process): Calculated distance in 0-100 range. // Apply some averaging and smoothing to the recorded distance readings // to take care of noisy data. int calculateNewDistace(int sensorNumber){ int output = SensorFar; // Default value is the furthest distance. float weightingFactor = 0.5; // How fast the reading's importance tapers off in time. (1= no taper, 0 = divide by zero error.) float flickerFactor = 30; // When the change is greater than this, ignore it unless its two in a row. (It's probably noise.) if (latestReading >= ReadingsPerSensor-1) { // Only do this if we have a full set of readings to sample.
int total = 0; // Average them with a weighting.
float currentWeight = 1; // New readings count more than older readings.
float percentagePossible = 0;
boolean flickered = false;
for (int i=ReadingsPerSensor-1; i >=0 ;i--){ // Check for flicker (This reduces jitter with something right on the threshold.)
flickered = false;
if (i==ReadingsPerSensor-1){
if ((abs(sensorReadings[sensorNumber][i])-abs(sensorReadings[sensorNumber][i-1]) > flickerFactor) &&
(abs(sensorReadings[sensorNumber][i-1])-abs(sensorReadings[sensorNumber][i-2]) > flickerFactor)){
flickered = true;
}
}
if (flickered==false){
total += (sensorReadings[sensorNumber][i] * currentWeight);
percentagePossible += currentWeight;
currentWeight *= weightingFactor;
}
}
output = total / percentagePossible;
}
return output;
}
// function: getDistance
// Take a sensor number (not pin number) and returns an int in the 0-100 range
// 0 = closest, 100= furthest. (It's a percentage of the distance that the software
//
// Note: Function is designed to be generic so that it can be swapped out for
// different kinds of ranging sensors.
// This version of the function is made for Parallax PING))) sensors
// For more info see arduino.cc/en/Tutorial/Ping
// and www.parallax.com/tabid/768/ProductID/92/Default.aspx
int getDistance(int sensorNumber){
long duration; // How long it takes a sonic pulse to reflect back.
int out; // The value we send back from the function
// Initialize the sensor and tell it to send out a ping.
pinMode(RangingPins[sensorNumber], OUTPUT);
digitalWrite(RangingPins[sensorNumber], LOW);
delayMicroseconds(2);
digitalWrite(RangingPins[sensorNumber], HIGH);
delayMicroseconds(5);
digitalWrite(RangingPins[sensorNumber], LOW);
// Read the time in milliseconds until the value comes back.
pinMode(RangingPins[sensorNumber], INPUT);
duration = pulseIn(RangingPins[sensorNumber], HIGH);
// Trim the data into minimums and maximums and map it to the 0-100 output range.
duration = constrain(duration, SenorClose, SensorFar);
out = map(duration, SenorClose, SensorFar, 0, 100);
return out;
}//
And lastly here is a link to a PDF outline of the gauntlet. It fits most size of hands.
A huge thanks to the design professionals attending Device Design Day who were among the first to see it in action and who gave tons of positive feedback. As mentioned there are a number of improvements and changes left to be made. I’m curious to see what happens.
What software is needed to upload the code onto the Arduino Mini Pro 5v.?
arduino.cc/en/Main/Software
I am building this for my school science project and I wanted to know if I am missing any parts:
(1) Arduino Mini Pro 5v (2) Parallax PING))) ultrasonic sensors.(2) Turnigy TG9 servo(1) 9v battery connector(1) slide switch.Shapelock
Sounds like you have all the main components. you might also want to get a breadboard and connectors to try it out. You will also need the usb board or cable to program it.
Very nice idea with the change of pressure! I also like the hands-free design. We’ve been working with something very similar we called the Enactive Torch:
enactivetorch.wordpress.com/
Hello from France. I made a cheap version using playstation motors (i know, vibrations is not the best), and a arduino duemilanove. Many many people interested, so we are setting up a week of prototyping with various disabled people in april here in our local Lafab (french speaking fablab) : labfab.fr
I think i am going to add a flex sensor to define on demand the range of the ultrasonic sensors. For eg, you touch a wall with lleft hand and then close the right hand to choose about 1 m range. And so on.
@hugobiwan:twitter
That’s fantastic! If you have a chance please follow-up with what you discover in your April prototyping.
This is really cool! I plan to make one, but modify it slightly to make the on/off more touch-friendly. I also want to play around with the code. I also have one question: could an intermediate seam-man(is that the term for a male that sews?) work with 2mm neoprene?
I think you’ll be fine sewing. The only real sewing you have to do is the Velcro, and if you get matching color thread you can be surprisingly sloppy without anyone finding out. If you look really closely at the prototype you’ll see the machine dropped about half the stitches and so I just went back over it again. (You can do this by hand too, but pushing a needle through Velcro and neoprene is tough on fingers.)
Another option is to glue it. Most glues won’t work at all in a flexible, stretchable environment, but one I’ve found that works is Shoe GOO. It isn’t as durable as sewing, but held up pretty well.
Thanks!
Hi- My wife is totally blind and getting around the house is rough and out side is worse. She is 92 years old and would love to try something like this. Where can i buy one and what is the cost?
Contact me at warnerlarry@hotmail:disqus .com Thanks to anyone who can help me out.
Thank you. Larry Warner
Hi Larry,
I don’t make them or sell them at this point, but a number of people have found electrical engineering students at their local college to build one for them. The plans are all here and it should be pretty straightforward for anyone with basic electronics and simple tinkering skills.
This is such a great project! My sister is blind, and her birthday is in 3 days. I’m just finishing making this to give her. I bought a lot of the parts around February, but never had time to really work on it. I have everything soldered together and seems to be working now. I’m making the glove tonight. This is my first time ever using an Ardiuno, or really doing any extensive soldering, so I’m having a lot of fun! I ignored your warning about this not being a good “First project”, and things have gone well. I do have one question, which I may be able to figure out better once I get the glove assembled and can really test it.. The servo on the left side doesn’t seem to move NEARLY as much as the one on the right.. Is there anything I should look at for this? They both work, and both ping sensors are flashing away, but the right one seems to move a lot more regardless of which sensor is nearing an obstacle.
I used all the exact parts you suggested, so it shouldn’t be a difference in hardware..
Thanks,
Jon
That’s a new problem I haven’t heard before. The first thing I would try is swapping the servos to see if it’s a hardware problem or a software problem. (you can just switch the plugs if you have it partially assembled. If its a hardware problem the same servo will have the problem no matter where it’s plugged.
If it looks like a software problem try changing the values in the “ServoClose” and “ServoFar” arrays. The numbers in those arrays are the number of degrees the left and right (respectively) servos turn for the closest and furthest readings.
(I may have right and left switched around in those arrays. It may require a bit of experimentation.)
Steve,
Thanks for your reply! So I have everything fully assembled now (I couldn’t manage to sew the neoprene with anything I had, so I used self adhering velcro and super glue).. The left sensor and servo are working great. The right one is not.. I tried swapping the servo connections as you suggested, but had no luck.. Just now (in the middle of typing this) I swapped the ping sensors, and the problem followed the right sensor.. I must have some problems with it.. My sister’s birthday party is tomorrow, so I’ll have to give it to her as-is, but it’s functioning with the left sensor. I will order another ping and try to figure out how to melt the shape lock and swap out the bad ones.. I’m glad I decided to solder plugs onto the wires for the ping sensors, that will make the replacement easier. Do you have any tips for removing the ping sensor after having it molded in shape lock?
I apologize if I’m rambling.. It’s way past my bed time.. haha.. I’ve just been working my tail off trying to get this finished.
Thanks again for all you do!
Jon
I’m sorry you’ve having problem with the sensors, especially since they’re by far the most expensive part of the project.
As for removing Shapelock I haven’t tried it myself. First I’d take a Q-tip, dip it in isopropyl alcohol and apply it along the edge of the Shapelock where it contacts the sensor or servo. This should help break the tight hold. Then, after it dries thoroughly go after it with a hair dryer to soften the shapelock and bend/pry it with pliers or screwdriver. It should withstand much more heat than the Shapelock. Best of luck, and happy birthday to your sister.
Steve,
Thanks for your replies! I got my replacement sensor, and used a hair dryer to warm the shapelock and replace it as you suggested. It worked great! Thanks again! Everything seems to be functioning now. I just contacted Parralax to try and get a refund or replacement for my bad sensor. Thanks again for all your hard work on this project! You’re making a difference in the world!
Sincerely,
Jon
I am an Egyptian …. I want to buy it for my grand father ….. but I don’t know where can I buy it ?????
Can any body help me , please ???????
Hi, I’m trying to create one by my self but I’m having some issues. Could you please help me just a little bit