Human Perception and Memory Semester 2, 2009 1 Vision Human - - PDF document

Human Perception and Memory

Semester 2, 2009

1 Vision

Human Visual Perception

• Humans are visual creatures.
• While your eye is like a camera, there’s no little guy inside your head

watching T.V.

• Rather, your subjective perception of the visual world is created by

your brain: John Ross: “Vision is an hallucination triggered by sense inputs”. Notes For most people, vision is the dominant sense, at least in terms of gath- ering information and interacting with the world. This is particularly true

• f conventional user interfaces, although in part that’s due to limitations of
• technology. We have pretty good widely available technology, at least for cre-

ating flat, 2D, imagery (LCD display screens, etc.), and for stereo sound. But at the moment we have only limited, and mostly experimental, technology for smell and touch. Human vision, like any other mode of perception, is far from simple. While the creation of your subjective perception is fundamentally a myste- rious process, we know a lot about it, enough to provide practical guidance for designing GUIs. The Human Eye See Figure 1. Notes The fovea (the term I’ll use), is also called the macula (as in Figure 1).

Figure 1: Cut-away view of the human eye. From http://en.wikipedia.

• rg/wiki/Image:Human_eye_cross-sectional_view_grayscale.png.

The Human Eye Functioning

• focussing (cornea, lens)
• aperture control (iris)
• image capture (retina)

– variable resolution (fovea) – photoreceptors (rods, cones) – integration time (1/15 sec.) – preprocessing

• pointing (saccades)

Focussing

• Cornea and lens form an image on retina.
• The lens is made out of special transparent crystalline protein.
• Lens and ciliary muscles adjust focus for different distances—accommodation.
• Focussing problems

2

• Age effects: presbyopia, yellowing, stiffening, “floaters”, . . .

Notes

• Just as a camera’s lens forms an image of the outside scene on its film or

CCD array, so the cornea and lens of the human eye working together form an image on the retina.

• The lens is made out of special transparent crystalline protein.
• Since the difference in refractive index between the lens and its sur-

rounding fluids is fairly small, it can’t bend light very much.

• Most of the bending of light to form the image is actually done by the

cornea (the curved, transparent front surface of the eyeball), because

• f the substantial difference in refractive index between the cornea and

the air. This is why you can’t see clearly under water: The cornea loses almost all of its focussing ability, because its refractive index is not much different from water’s. Wearing a diving mask restores this cornea-air interface, so your eye can focus.

• The lens functions mainly to adjust the focussing already done by the

cornea. This is why people can still see after they’ve had the lens surgically removed because of cataracts (a clouding of the lens), though they’ll need spectacles or a contact lens to compensate.

• And just as with a camera, for a particular setting, only objects at a

certain distance will be perfectly in focus, therefore the focus setting generally needs to be changed depending on whether you’re looking at nearby or far away objects. Commonly, because of the shape of your cornea, lens or eyeball, you may not be able to focus properly. Usually this can be corrected by wearing eye glasses or contact lenses.

• Cameras adjust focus by changing the distance between the lens and

the image; the eye instead does it by changing the shape of the lens. The lens is attached by the zonular ligaments. In the normal, resting state, the ciliary muscle is relaxed, the ligaments are taut and pull the lens into a thinner shape that bends light less. This is for focussing on far-away objects. 3

When the ciliary muscle contracts, it loosens the ligaments, allowing the lens to spring back into a fatter shape that bends light more. This is for focussing on close-up objects. This process of focus adjustment is called accommodation.

• Age effects: The protein of your lenses has to last your entire life.

It never gets replaced, and deteriorates with time. It loses its trans- parency and elasticity and becomes yellowish with age. Think about what effects this will have. In particular, the lens loses its ability to relax into the fat shape needed to focus on close-up objects. This phe- nomenon is called presbyopia, from Greek words meaning “elder see- ing”. (The name of the Christian denomination, Presbyterian, comes from one of the same Greek words, since their churches are governed by “elders”.) This is why most people need reading glasses as they get older. Another thing that happens any time, but increasingly with age, is that little bits of stuff become detached and drift around inside the eyeball (“floaters”), casting a faint shadow on the retina. You can sometimes see these when you look up at a clear blue sky. Aperture Control

• The iris is like the aperture control of a camera.
• Part of adjustment for varying light levels.
• Bigger aperture:

– Harder to focus. – More effect of lens defects.

• So vision much harder under low-light conditions.

4

Notes

• The iris is the band of muscles in front of the lens. The opening in the

iris is the pupil. It looks black because you’re looking into the relatively dark interior of the eyeball. The iris is pigmented and patterned: the “color” of somebody’s eyes is the color of his’r’her iris. Some security systems recognize people by the pattern in their iris, like a fingerprint.

• The iris controls the size of the pupil and hence the amount of light

entering the eye. It’s one of ways the eye adjusts to varying light levels: In bright light, the pupil contracts to reduce the amount of light getting into the eye. In low-light conditions, the pupil expands (to about 7mm across) to gather as much light as possible. The eye also adjusts to different light levels by chemically altering the reponsiveness of the photoreceptors.

• Pupil adjustment is like the aperture control (f-ratio) of a camera.
• As with a camera, the aperture also affects depth of focus (aka depth
• f field). To see an object clearly, your eye has to accommodate to

focus on that object. Objects nearer or further away will be out of focus, blurry. This focus adjustment is much more critical for a large aperture than a small one. (This is how you can get cheap “focus-free” cameras: They have a small enough aperture that everything over some reasonable range of distances is acceptably in focus.)

• Another issue is that with a large aperture (big pupil) the light is

passing through more of your lens, and will be affected more by any

• ptical defects in your lens.
• The practical effect of these two effects is that vision is much more

difficult under low-light conditions. In bright light, your pupil shuts down to a small aperture: this makes focussing easier and reduces the impact of lens defects. In low light, though, your pupil opens up to a large aperture: focussing becomes more difficult and lens defects show up more. Retina

• Retina covers inside surface of back of eyeball.
• Like CCD sensor of a camera.

5

• Specialized light-sensitive nerve cells, photoreceptors.
• Rods and cones
• Rods—“night vision”, scotopic vision, dark adaptation, blue sensitive.
• Cones—normal light, photopic vision, color perception.
• Retina: neural image-processing front-end computer.
• Blind spot

Notes

• The photoreceptors contain special pigment molecules, which react to

light, setting up a chain of chemical reactions in the cell causing it to send off nerve signals.

• The photoreceptors come in two varieties, rods and cones. (The names

come from the shapes of the cells.)

• The rods are very sensitive to light, and operate only under low-light
• conditions. They are responsible for our “night vision” (known techni-

cally as scotopic vision, from Greek for “dark seeing”). Rod-based night vision is of importance only in some applications, like for visual astronomers, police, emergency workers, military, and the like, also for ordinary citizens driving cars at night, particularly in the country with no street lights. In some of these situations, it can be of life-and-death importance. So it is useful to be aware of the following characteristics of rod-based vision: – Rods are active only in low-light conditions and are inactived by bright light. Worse than that, it takes some time in the dark for rods to become active, about 10 minutes—even longer, half an hour or more, for them to reach peak sensitivity. This is why, when you step out into a dark night from a well-lit room, you at first can’t see anything. Only as your eyes “get used to the dark” will you start to see anything. The reverse process, however, is not gradual. Even a brief expo- sure to bright light is enough to switch off your rods and disable your night vision. – Rod-based night vision is adversely affected by such common drugs as alcohol, caffeine, nicotine, even at quite low doses. 6

– Rods give us no perception of color. (What little color you might see walking about at night comes from residual low-level response

• f your cones.)

– While rods are not connected to any color perception, they are in fact more sensitive to blue light and practically unresponsive to red light. You can sometimes notice such color-based intensity shifts around

• dusk. When he was a toddler, my son had a jumper with repeated

equal-width red, blue, and white stripes. One evening in the back- yard, as it grew dark, I was puzzled that now it appeared to be jumper with alternating thin dark stripes and thick light stripes. This was because my rod-based night vision was starting to kick in, and to the rods the single red stripe looked dark (because they didn’t respond to red), while the blue and white stripes looked like a single, thick light stripe, since to my rods, the blue and white looked much the same: they responded well to the blue light re- flected from the blue stripes and to the blue component of the light reflected from the white stripes. Similarly if you have a red sock and a blue sock that seem to be

• f the same shade (if different hues) in bright light, then by dim

light, once you’re adapted to the dark and have your night vision, the red sock will now appear much darker than the blue. Try it

• ut some time. This is called the Purkinje Effect.

So people who rely on their night vision use a red light to read their maps, etc., (or a red-only computer display). The red light is bright enough to stimulate the cones for normal vision, but is effectively invisible to the rods, so they are unaffected, and stay

• active. (They “think” it’s still dark, since they can’t “see” the

red light.) If such people had used an ordinary white light, or full-color display, then they would have lost their night vision for ten minutes or more. – Many animals, particularly nocturnal animals, have only rod vi-

• sion. So you can use a red light to observe them in the dark, and

they won’t even notice it.

• The cones are not so sensitive to light, and operate under normal light-

ing conditions. They are responsible for our everyday vision under daylight and normal artificial light (known technically as photopic vi- sion, from Greek for “light seeing”). Of great importance is that cones provide us with color vision—the ability to perceive and discriminate 7

• colors. (You can remember this by the mnemonic: both “cone” and

“color” start with “co”.) So it’s cone-based vision that’s important for most GUI applications.

• People with normal color vision have three kinds of cones in their eyes,

nominally sensitive to blue, green, and red light, respectively. However, the wavelengths of light at which these cone types have their peak sensitivities do not correspond with those color names. So among vision scientists it is now the custom to refer to them as the S, M, and L cones, respectively for short, medium and long wavelength (of light). For example, while the L cones are responsible for our perception of red, their peak sensitivity actually occurs at a wavelength of light that would be perceived as a slightly greenish yellow. Regardless of the terminology, human color perception is based, ini- tially, on the responses of the three cone types to incoming light. This is the basis of color photography and color displays. Within limits, you can mix red, green, and blue light to create almost any perceivable

• color. You just have to adjust the relative mix of red, green and blue

light so as to stimulate the S, M, and L cones in the same way as the desired color.

• If you don’t have a full complement of working cone receptors, then

you’ll have color blindness (or at least some degree of color-vision de- ficiency), which effects about 8% of the male population (and about 0.4% of the female population).

• However, the retina is much more than just a sensor: It’s better thought
• f as a neural image-processing front-end computer: The image received

by the photoreceptors goes through a few stages of processing before being relayed to the brain via the optic nerve.

• By some accident of evolution, the vertebrate retina is “wired up” the

“wrong way”: The output connections are actually on the front of the retina, on the inside of the eye-ball. All the output nerve fibers get from the inside to the outside at a single point where they join the

• ptic nerve, which connects on to the brain. This point is the so called

“blind spot”, because there are no photoreceptors there. But in normal situations we are totally unaware of this blind spot, because later visual processing “fills in” the gap.

• Eyes of cephalopods (like squid and octopus) are remarkably similar

to vertebrate eyes (like humans’). However, this is an instance of con- 8

Figure 2: Drawing of a small section of the human retina by Santiago Ram´

• n

y Cajal, from http://en.wikipedia.org/wiki/Image:Cajal_Retina.jpg vergent evolution, since cephalopods and vertebrates have no common ancestor with any eyes to speak of. One difference is that the neural connections of cephalopod eyes are on the back of the retina, so they have no blind spot. Retina Neurocomputer See Figure 2, which shows the photoreceptors (rods and cones) at the top, and the various layers of nerve cells they connect to perform the initial neural preprocessing that’s done before the visual signals are passed back to the brain through the optic nerves. Note, this diagram is oriented so that the front of the eye is at the bottom. The Fovea versus Peripheral Vision

• Retina: variable-resolution sensor.
• Fovea: central small patch of high-resolution vision.

Cones only, no rods. 9

• Illusion of clarity.
• Peripheral vision.

Notes

• Ultimately, the human vision system has only limited processing re-

sources available, so those resources need to be deployed where they’re most useful.

• Normally, what you’re looking directly at is of most importance, so the

vision system devotes more resources to the center of your field of view, and less to the periphery.

• In a CCD array in say a digital camera, the pixels are distributed

regularly and uniformly across the entire image.

• But the retina is a variable-resolution sensor: There’s a much higher

density of photoreceptors in a special, central area of the retina, called the fovea, and much lower density in the periphery.

• This means we really see things in full detail only in a quite small

region close to the direction we’re looking in. In visual terms, it’s about the size of your thumbnail held out at arm’s length (about one degree across).

• Your impression that you see everything clearly is just an illusion: As

we’ll see later, your eyes are continually moving, looking in different

• directions. Whatever you’re looking right at now you see in full de-

tail, and to some extent what you’ve just looked at your visual system “remembers” in full detail. Try this: Look directly ahead at something, and fix your gaze on it. You’ll see it in full detail, courtesy of your fovea. Now, without shifting your gaze, pay attention to objects off to the side. This is tricky to do, because your normal reaction will be for your gaze to automatically follow your attention. Resist this. Shift your attention without shifting your gaze. You’ll appreciate that objects off to the side are “blurry” and ill-defined, compared with how they appear when you look straight at them. Or you can fix your gaze on a word on the page in front of you, and without shifting your gaze, try to read a few words to the left and the right. 10

• It’s not merely the density of photoreceptors that changes, but also

their nature. There are fewer cones in the periphery, so our color perception is modified and our ability to discriminate colors is weaker. I remember one long night drive I went on, looking ahead at the road (as one does in such situations). I was puzzled by an unusual orange indicator light I saw down on the dashboard. The more so, because every time I looked straight at it, there was no orange light there, just some normal red indicator. It took me a while to realise that it was in fact the same light. Because of different color perception between the fovea and periphery, the same light that appeared red when looked at directly, appeared orange to me when seen off to the side. Conversely, there are no rods in the fovea (all the space is used up with cones). This means that a faint light may paradoxically be invisible when you look straight at it, but appear “mysteriously” when you look slightly away from it. This is an old trick used by astronomers.

• I’ll say a bit more about form and motion perception later, but we also

have more motion detectors out in the periphery, out beyond where we have form (shape) detectors. Try this: Look straight ahead, point your arm out in front of you, make a fist and stick your thumb out pointing upwards. Now, while keeping your gaze fixed ahead, gradually swing your arm horizontally to move your thumb off to the side. At first this will be just like the previous party trick: as your thumb moves off to the side, it will become less and less distinct. But at some stage (probably a little past 90 degrees from dead ahead), you’ll just reach the point where you can no longer see your thumb at all. Now, wiggle your thumb. You’ll be able to clearly see your thumb moving (or at least see something moving), even though you can’t see it at all when it’s still. Integration Time and the Movies

• Integration time, about 1/15 second.
• Motion pictures, animations.
• Screen refresh.

Trade-offs. Peripheral vision. Notes 11

• Any sensor has to make its measurements over a finite interval of time,

its integration time.

• The photoreceptors in your eye are no exception, and being made out
• f electrochemical wetware rather than electronic hardware, they are

relatively slow.

• As always, it’s not simple, but roughly speaking, the integration time
• f photoreceptors under normal conditions is about 1/15 second.
• In simplest terms, this means that if you have a light flashing on and off

slower than 15 times a second, you’ll be able to perceive the individual

• flashes. But if the light flashes on and off much faster than 15 times a

second, then the flashes will all blend together to create a more or less continuously glowing light of brightness averaged between on and off.

• This is the whole basis of motion pictures: If you show a sequence of

still images rapidly enough (24 frames a second for standard movies, 25 for PAL TV, very close to 30 for NTSC TV), then the successive images will blend together to create the illusion of continous motion.

• In user interfaces, this is clearly important if you’re doing animations.

But even if you aren’t explicitly doing animations, there’s still the is- sue of the refresh rate of your computer screen, which is redrawn many times a second. Because of various effects, which I won’t go into here (mainly the brightness and large size of the screen) “flicker”, the fluc- tuation in brightness caused by the redrawing of the screen, can still be noticeable at refresh rates considerably above the nominal 15Hz. Even low-end screens would normally refresh at around 50–60Hz; higher-end at 80–90Hz or more, and some would say these high refresh rates lead to better appearance. Since one bottleneck is usually total hardware bandwidth, there’s often a trade-off involved: You can get higher screen resolution (more dots per inch), but only at lower refresh rates. To go to a higher refresh rate, within the bandwidth limits of your hardware, you’d need to drop the screen resolution and perhaps the color resolution (number of bits of precision devoted to storing color information). Which would be best would depend on your intended usage: If you were doing something like desktop publishing, you’d probably go for the highest screeen and color resolution, even if it meant a relatively low refresh rate (so long as it didn’t get down to the level of objectionable flicker). If you were doing animations or computer gaming, then maybe the best choice would 12

be to opt for higher refresh rates at the expense of screen and color resolution.

• Integration time depends on various factors like brightness and position.

In general, we’re more sensitive to motion and flicker in our peripheral

• vision. For example, a screen which is adjusted to be flicker-free when

you look at it directly, may still exhibit a noticeable degree of flicker when you view it “out of the corner of your eye”, as you may do when you look away to read a document on your desk. Same with TV. Saccades

• Eye muscles.
• Gaze jumps—saccades.
• Retinally stabilized images.

Notes

• Attached to the eyeball are various muscles that allow it to be turned

in different directions.

• In normal circumstances, your eyes are continually moving, pointing at

different parts of what you’re looking at.

• Remember that the fovea, the only part where we see full detail, cor-

responds to only a very small part of our field of view. To a large extent, our illusion of seeing everything more or less clearly is achieved by stitching together lots of little, partial views.

• This gaze motion is usually not continuous and smooth, but our gaze

moves in jumps, called saccades, from fixation point to fixation point. Most of this occurs below the level of conscious awareness, but it can be measured in psychology labs.

• It’s a fairly complicated process, but essentially what happens is that

from the unclear view in the periphery, your vision system picks the next interesting point to look at, and jumps to that point, then repeats, building up an overall perception in the process. Of course, for more specialized vision tasks like reading, the saccades follow a more stereo- typical pattern, from word to word along a line (or more likely, from word group to word group) and then a jump back to the beginning of the line to start the next line. 13

• The effect of this is that, even if you’re looking at something stationary,

the pattern of light projected onto your retina is always changing.

• Why doesn’t our perception jump about then? The reason is that our

visual system “knows about” the eye movements, and compensates for them in constructing our perception.

• Moreover, any image that doesn’t move about relative to the retina

actually fades away fairly quickly, in a second or so. These are retinally stabilized images. This doesn’t happen in the real world: because the eyeball is continually moving, nothing “out there” could produce a stabilized image. In the

• riginal experiments, this was achieved by putting a marker on a special

contact lens, tracking this marker to track the eye movement, and then moving a projected image on a screen to follow this eye movement, so that the image on the retina stayed the same. Kids, you can try this at home. Do it in a reasonably dark room— doesn’t have to be perfectly dark. You’ll need a small light. One of those little key-ring red LED lights works well. A small torch will probably do, but you’ll probably need to put a cardboard disk over the lens with a small hole cut in it, so you get only smallish beam of light coming out. Hold the light close to the corner of your eye and jiggle it around a

• bit. Be careful doing this. Don’t do it at a party where you’re likely

to get bumped in the arm and poke yourself in the eye. If you get the position just right, you’ll see an amazing thing: this dendritic pattern, which is mainly the shadows cast onto your photoreceptors by blood vessels on the front of your retina. The image is elusive: It is a retinally stabilized image. If you don’t move the light, it’ll fade after a second or so. Even though your eyeballs are still moving, as always, the blood vessels are so close to the photo- receptors that their shadows fall in almost exactly the same place on the retina. If you jiggle the light around a little, you can get the image to persist for longer. The reason is that as you move the light, different parts become lighter or darker, and that’s enough of a change stop the fading. This demonstration also points to the reason for this apparently weird behavior of retinally stabilized images: In normal circumstances, if an image doesn’t move when the eyeball moves, then it must come from something actually inside the eye (like shadows of retinal blood vessels). 14

Figure 3: Back view of human brain. Visual cortex shown as color

• verlay,

from http://en.wikipedia.org/wiki/Image:Brodmann_areas_ 17_18_19.png. The fading is actually the visual system’s way of ignoring such things. It’s part of the computation done in the retina. Higher-Level Processing

• Several “subsystems”

– form, color, motion, stereo,. . .

• higher-level perception
• pre-attentive and attentive vision

Notes

• After leaving each eye the separate optic nerves meet under the brain

at the optic chiasm (named from its shape, which looks like the Greek letter χ), where the nerve pathways crossover. Signals from the right side of the left eye cross over to the right side of the brain; similarly signals from the left side of the right eye cross over to the left side of the brain. Why this strange arrangement? Well, the human brain is arranged so that, for the most part, sensory and motor processing areas for the right side of the body are in the left hemisphere of the brain, and vice-versa. Because of the inverted projection onto your retina, things in the right side of your field of vision are imaged on the left side of both your eyes (and vice-versa). The crossover at the optic chiasma means that vision for the right side of your field of view (from both eyes) is processed in the same hemisphere (the left) as the motor control for the right side of 15

your body. Similarly for the left side of your field of view (and body). This is a more efficient arrangement, because it means there is a more direct connection between visual processing and action on each side. It also means that the vision centers in each hemisphere get input from both eyes, which is important for stereo vision. After the optic chiasma, the main visual pathways on each side pass through the LGN (Lateral Geniculate Nucleus) and then on to the visual cortex at the back of the brain. (There are, however, other visual pathways.) Visual cortex is shown in Figure 3. You can see it takes up quite a large fraction of the brain. In the figure, different colors indicate different areas of visual cortex, but we need not worry about those distinctions here.

• There is pretty strong evidence that there are separate subsystems re-

sponsible for processing different aspects of vision, such as form (shape), color, motion, and stereo perception. It’s almost as if we had a num- ber of distinct visual senses, all running off input from our eyes, but processed separately, and only later integrated into a unified conscious visual perception.

• On top of all this is higher-level visual perception, by which we recog-

nize objects and people and what they’re doing.

• An important distinction to be made is between pre-attentive and at-

tentive vision. Most of the research on this was initially done by Anne Triesman. Suppose you have a visual task, like finding an object in your field of

• view. Suppose that object can be distinguished by one feature alone,

such as color or shape. The task might be “Find a red thing in this picture”, or “Find a letter X in this picture” (of any color). Then we are able to perform this task very quickly, and in constant time—that is the time taken does not depend on the number of other objects in the field of view nor on whether the object sought is present or not. This is called pre-attentive vision, because it can happen before we pay attention to any one of the objects we see. It is suggested that it can be done by parallel processing in the brain. However if our task involves evaluating a conjunction of several features, like “Find a red letter X in this picture”—that is, something that is both red and an X, then the task takes noticeably longer. What’s more, the time taken depends linearly on the number of objects seen, and on average takes twice as long if the sought object is not present. It seems 16

that to evaluate a conjunction of features, our visual system has to fall back onto sequential processing, scanning through the visual field (even if unconsciously), using essentially linear search. This is called attentive vision, since we have to attend to each object individually. Of course, for a simple conjunction, like color and shape, and a modest number of objects in the field of view, the time taken for attentive vision is quite short, but still longer than for pre-attentive vision. The time differences can be measured by reaction-time experiments. It’s a little bit more complicated than this. There are certain combi- nations of features for which we seem to have the neural circuitry to process them in parallel. For example, if you’re shown a 3D stereo dis- play showing objects at different depths away from you, then you can quickly perform task like “Find a nearby red object”, even though that seems to be a conjunction of two features, “nearness” and “redness”. But the distinction between pre-attentive and attentive vision is still an important one, with obvious implications for the design of user in- terfaces. Visual Illusions and Oddities M¨ uller-Lyer Illusion Visual Illusions and Oddities Kanizsa Figure Notes Figures 4 and 5 show two well known “optical illusions”—although I prefer to call them “visual illusions” since they have more to do with human visual processing than with optics. In the M¨ uller-Lyer Illusion (of which Figure 4 is one of several variants), both lines are objectively of identical length on the page. You can measure them with a ruler. However, to most people, the line with the outward di- rected Vs (like arrow tails) appears longer than the line with inward directed Vs (like arrow heads). Why do we have this apparently incorrect perception? There are a num- ber of theories to explain it. The most interesting, mainly due to the vision scientist Richard Gregory, is this: The line with the arrow heads looks like the corner of a box or building seen from the outside. In this case, the line is interpreted as being closer, and therefore as appearing bigger than it really

• is. Our vision system therefore unconsciously perceives this line as shorter,

as being an indicator of its true size in 3D space. Conversely, the line with arrow tails looks like the corner of a room seen from inside the room. In this case the line is interpreted as being further away, and therefore as appearing 17

Figure 4: M¨ uller-Lyer Illusion. Figure 5: Kanizsa Figure. 18

smaller than it really is, and our vision system unconsciously perceives this line as longer, again as being an indicator of its true size in 3D space. As a point of interest, the M¨ uller-Lyer illusion is to some degree cul- turally dependent. The effect seems to be stronger for people who live in urbanized cultures, where presumably there are lots of rectilinear structures and pictures of such structures. Also, the effect apparently becomes weaker if you’re exposed to the display a lot—presumably your visual system can learn in time to partly disregard the depth cues from the arrow heads and arrow tails. After all the display is just a pattern of lines on a flat page (or display screen). Figure 5 shows a Kanizsa Figure, illustrating the phenomenon of subjec- tive contours. Most people see a white triangle in front of, and obscuring,

• ther shapes in the figure, which seem to be three black discs and an outline
• triangle. The white triangle seems to be a slightly brighter white than the
• background. But “in reality” there is no white triangle—it is completely
• illusory. The figure is made up only of three V shapes and three “pacman”

shapes (each a disc with a wedge cut out). These shapes are just artfully arranged and aligned so as to suggest the illusory triangle. Most people can see definitely, if faintly, the straight edges between the whiter triangle and the white background—these make up the contour of the triangle, the subjective contour, since it doesn’t exist objectively. The paper (or screen) inside the white triangle is exactly the same brightness as the background. Nonetheless the perception is very real. The “non-existent” subjective contours can be used to construct other illusions (such as variations

• n the Ponzo Illusion). Also, when subjects are asked to adjust the brightness
• f another display to match their perceived brightness of the illusory triangle,

and then similarly to adjust that other display to match their perceived brightness of the background, they will consistently and objectively set that

• ther display brighter when matching the illusory triangle.

Why do we see the illusory triangle? It’s mostly our visual systems un- consciously making the best guess as to what’s out there, given the visual

• inputs. In the real world (that is, outside psychology textbooks), what is

the most likely explanation for that picture? One explanation is that there really is a triangle there—that best explains the aligned obscurations of the inferred disks and outline triangle. The other explanation is that the pacman shapes and V shapes have been exactly lined up in a very improbable way. Our visual system opts for the first explanation, as being most likely. Of course, this isn’t a conscious process of weighing up probabilities; rather it’s the way our vision systems have been tuned by evolution to work best for us in the real world. What Helmholtz called “unconscious inference”. What’s the point of studying such illusions? Well, for everyone, including 19

HCI practitioners, they emphatically make the point that our perception is a construct—it’s not just like a verbatim image made by a camera. Also, for vision scientists they provide clues about how the human vision system

• works. And for anyone creating visual displays, including HCI practitioners,

they serve as a warning that a badly designed display might mislead our users and lead them to make wrong judgements and decisions. Color Vision

• Light is electromagnetic radiation in the wavelength range of roughly

400nm (“blue”) to 700nm (“red”)

• Almost all light is a mixture of wavelengths, e.g., the rainbow spectrum
• f white light from the sun
• Tristimulus theory of color perception

Color Vision

• Three kinds of cones, nominally sensitive to “red”, “green” and “blue”

light – “Tuning” is quite broad – More accurately perhaps one short wavelength system and two long wavelength systems

• Tristimulus values: (R, G, B)—raw input
• Later processing into intensity and color opponents R − G and B − Y
• (Intensity, hue, saturation)
• Color constancy

Cone Responses 20

red light 400 500 600 700 Wavelength in nm. "blue" "green" "red" blue light

Relative responses of human cone photoreceptors (Approximate only) Characteristics of Human Vision. . .

• Acuity and hyperacuity
• Form and motion perception does not ‘see’ color

– Color “washes” in drawings – Encoding of luminance and chroma in color TV – Isoluminance contours – Chromatic aberration: color edges cannot be brought to sharp focus . . . Characteristics of Human Vision

• Color vision characteristics

– Color perception depends on color context – Blue alone tends to be perceived quite weakly – Greatest color discrimination is in the green to yellow range – Color blindness: red-green, blue-yellow, achromatic – Possible variation even amongst people with “normal” color vision – Implications for computer processing and display 21

2 Hearing

Human Auditory System Human Hearing

• Vibrations: 20Hz to 20,000Hz less with age
• Sound localization: ITD, IID

3 Memory

Complexities

• Human memory is very complex and little understood
• Be wary of any simplistic classifications
• Still, some useful knowledge for HCI

22

Divisions of Memory

• LTM, long-term memory
• STM, short-term memory

– (part of Working Memory, according to some)

• (probably also medium-term memory)

Long-term Memory

• effectively life-long memory
• effectively “infinite” capacity
• retrieval/accessibility
• probably mediated by growth of neural connections
• may take up to two years to form

Short-term Memory

• short-duration, task in hand, e.g. dialling a phone number
• probably mediated by neural activation patterns
• limited capacity, “7 plus or minus 2” items (Miller, 1956)
• chunking

– affected by recognition/experience Kinds of Memory

• Sensory memory

– sense impressions: sight, hearing, smell, taste, touch. . . – related: motor memory

• Episodic memory

– events: what happened at lunchtime today

• Semantic memory

23

– facts: What is the capital of Maryland? – (comment: We’re usually good at knowing what we don’t know.)

• Procedural memory

– “knowing how” versus “knowing that” – like swimming, riding a bike, typing your password Other Memory Phenomena

• primacy and recency
• closure
• psychological/emotional factors

– vividness, associations – blocked/suppressed memories – false memories, biases Some Implications for HCI

• Keep well within STM limitations

– (Leave room for user’s real goals)

• Much HCI depends on sensory, motor, procedural LTM

– (Re-arrange a familiar GUI at your peril!)

• . . .

24