How to teach image processing and computer graphics to a blind student?

Question

I'm not (yet) in this situation, so this is a hypothetical question. What would be viable approaches to teaching image processing (IP) and computer graphics (CG) to a blind college student? IP could possibly be approached by explaining images purely as 2D signals, and yet I'm not sure about how effective that would be. CG seems like an even harder task.

Answer 1

I think you can teach 2D graphics using a tactile approach on a phone or tablet.

Students are very likely to be used to tracing their fingers over a mobile device (I have watched awesome videos of blind iPhone users in action).

So position and shape change of an object could be detected by them dragging their fingers over to get auditory feedback at the edges. Ideally you want a truly tactile surface but I don't know of any such devices. I have my own reasons for wanting a computer-controlled version of one of these

For actual image processing maybe map colours to tone, so as they move their finger over an image they hear a tone vary depending on the average pixels underneath. Then, as transforms are applied, the tones would change. You could possibly teach colour theory by varying the mapping to only one channel of RGB.

Update in one of those great examples of synchronicity, a few days after writing the above I was listening to Matthias Tretter of MindNode on a podcast mention how he had been convinced to add accessibility to MindNode, a visual mindmapping product by a blind user.

His blog posting has videos showing VoiceOver in action. This is a bit tangential to the original answer but a nice anecdote about how preconceptions about the intended purpose of a tool as being visual doesn't rule it out for blind users.

Answer 2

Just some ideas

I imagine that the biggest problem, may be motivation.

It can be all done abstractly: in maths this can be taught, with no reference to application. However this is not ideal. Some (most) people are not good at abstract ideas (I am good at abstraction, but poor at abstract ideas ­— at least in maths).

I remember when I did image processing. I remember the words “it is just the 1d processing, you did in sound processing, but 2d. For a lot of it, it is just done twice, once for x and again for y”

Therefore start with sound processing.

Another Idea is using a texture map. However this may cost a lot to produce these.

Answer 3

A computer is blind conceptually speaking. Even if you gave it a camera, it cannot "see" per se. It has a translation of what is viewed to bits.

Now to the blind student. Image Processing is similar to how folks without sight process things - they "feel" their environment for "signals" and make a conclusion about what it is.

A computer does the exact same thing, but with math. Ask them to feel a stick. Easy. Now draw the stick on the floor/paper and ask them to say if a stick or triangle exists on the paper. Really hard. Ask them how would they know if a triangle is on the paper. Feel the edges? Anything else? How about everytime there's a triangle a new sound is heard? Be creative.

Now talk to them about "rasterization". Think of handling them one sliver at a time, and if anything is different by feeling, they note it. Can they "conclude" a triangle after seeing various cuts of the paper?

This exercise can be naturally extended to Image Processing and then math can be explained.

Computer Graphics is different. It lives in the visual plane and the idea is to "draw" it. A good introduction is this:

Problem statement: Draw a line on a graph/matrix and each "cell" must be filled in. Which cells to fill in between points A and B when "drawing this line".

Extending this to blind students could be done with toys/pins etc., How would they know which spots to "fill in" to get as straight a line as possible? What does a "straight line mean"?

This discussion could lead to things about using a line's slope (i.e., math) to figure which pixels to color on a raster display. Same thing with Circles.

I'd start here first. Once that's done, to move to transformations - moving things in space. Easy to feel and then take to the math.

I haven't worked with the blind, but I imagine such exercises/parallels with their sensory world could be useful.