Depth Perception and the Perception of Events

bundet

228 Depth Perception and the Perception of Events followers of this second proposal used more naturalistic set- tings, most studies on the perception of heading have used random-dot displays simulating the optical motion that would be produced by an observer moving relative to a ground plane, a three-dimensional cloud of dots, or one or more fronto-parallel surfaces at different depths. Overall, empirical investigations on heading show that the human visual system can indeed recover the heading direc- tion from velocity ﬁelds like those generated by the normal range of locomotion speeds. The psychophysical studies in particular have revealed the following about human percep- tion of heading: It is remarkably robust in noisy ﬂow ﬁelds (van den Berg, 1992); it is capable of making use of sparse clouds of motion features (Cutting et al., 1992) and of ex- traretinal information about eye rotation (Royden, Banks, & Crowell, 1992); and it improves in its performance when other three-dimensional cues are present in the scene (van den Berg & Brenner, 1994). Some of the proposed computa- tional models embody certain of these features, but so far no model has been capable of mimicking the whole range of ca- pabilities revealed by human observers. Vection Observers sometimes experience an illusory perception of self-motion while sitting in a stationary train and watching an adjacent train pulling out of the station.This trainillusion isthe best-known example of vection (Fisher & Kornmüller, 1930). Vection can be induced not only by visual, but also by auditory (Lackner, 1977), somatosensory (Lackner & DiZio, 1984), and combined somatosensory and kinesthetic (Bles, 1981) in- formation. The ﬁrst studies on visually induced vection can be dated back to Mach (1875) and were performed using a verti- cally striped optokinetic drum or an endless belt (Mach, 1922). Two kinds of vection can be distinguished: circular and linear. Circular vection typically refers to yaw motion about the verti- cal axis, whereas linear vection refers to translatory motion through a vertical or horizontal axis. Vection is called satu- rated when the inducing stimulus appears to be stationary and only self-motion is perceived (Wertheim, 1994). Linear vection is typically induced about 1–2 s after the onset of stimulation (Giannopulu & Lepecq, 1998), circular vection after about 2–3 s, and saturated vection after about 10 s (Brandt, Dichgans, & Koenig, 1973). A more compelling vection is induced by faster speeds of translation or rotation (Larish & Flach,1990), by low temporal frequencies (Berthoz, Lacour, Soechting, & Vidal, 1979), by more or larger elements (Brandt, Wist, & Dichgans, 1975); this is also the case when larger retinal areas are stimulated (Brandt et al., 1975) and when the inducing stimulus belongs to the background relative to the foreground (Nakamura & Shimojo, 1999). Visual Control of Posture Postural stability, or stance, is affected by visual, vestibular, and somatosensory information. Visual and somatosensory information are more effective in the low-frequency range of postural sway, whereas vestibular information is more effec- tive in the high-frequency range (Howard, 1986). A device known as the moving room has been used to demonstrate that visual information can be used to control posture. In their original study, Lee and Aronson (1974) required infants to stand within a room in which the walls were detached from the ﬂoor and could slide back and forth. They reported that when the walls moved, infants swayed or staggered in spite of the fact that the ﬂoor remained stationary, a ﬁnding later replicated by many other studies (Bertenthal & Bai, 1989; for adult, see Lee & Lishman, 1975). Two sources of visual information are available for pos- tural control: the radial and lamellar motions of front and side surfaces, respectively, and the motion parallax between ob- jects at different depths that is generated by the translation of the observer’s head (Warren, 1995). Evidence has shown that posture is regulated by compensatory movements that tend to minimize both of these patterns of optical motion (Lee & Lishman, 1975; Warren, Kay, & Yilmaz, 1996). Three hy- potheses have been proposed concerning the locus of retinal stimulation: (a) the peripheral dominance hypothesis, which states that the retinal periphery dominates both the perception of self-motion and the control of stance (Dichgans & Brandt, 1978); (b) the retinal invariance hypothesis, which states that self-motion and object motion are perceived independently of the part of the retina being stimulated (Crowell & Banks, 1993); and © the functional sensitivity hypothesis, which states that “central vision accurately extracts radia l…and lamellar ﬂow, whereas peripheral vision extracts lamellar ﬂow but it is less sensitive to radia l…ﬂow”(W arren & Kurtz, 1992, p. 451). Empirical ﬁndings have contradicted the pe- ripheral dominance hypothesis (Stoffregen, 1985) and the functional sensitivity hypothesis (Bardy, Warren, & Kay, 1999). Instead, they support the retinal invariance hypothesis by emphasizing the importance of the optic-ﬂow structure for postural control, regardless of the locus of retinal stimulation. Perceiving Approaching Objects Time to Contact and Time to Passage Coordinating actions within a dynamic environment often requires temporal information about events. For example, when catching a ball, we need to be able to initiate the grasp before the ball hits our hand. One might suspect that in order to plan for the ball’s time of arrival, the perceptual system Conclusion 229 would have to compute the ball’s speed and distance; how- ever, this turns out not to be the case. In determining time to contact, there exists an optical invariant, tau, which does not require that object speed or distance be taken into account. First derived by Hoyle (1957) in his science-ﬁction novel, The Black Cloud, and later introduced to the vision commu- nity by Lee (1974), tau relates the optical size of an object to its rate of expansion in a manner that speciﬁes time to contact. Tau is deﬁned as follows: tau 艐  兾  兾  t where  is the angular extend of the object in radians, and  兾  t is the rate of its expansion. Tau speciﬁes time to contact under the assumption that the object is moving with a con- stant velocity. In the case in which the object and observer are not on a collision course, a similar relationship speciﬁes time to pas- sage (Lee, 1974, 1980). Let  be the angular extent between the observer’s heading direction and the object, then time-to- passage, in terms of tau, is deﬁned by tau 艐  兾  兾  t Tresilian (1991) has reﬁned the deﬁnition of tau by distin- guishing between local and global tau. Local tau can be com- puted solely on the basis of information about angular extents and their rate of change. Global tau is only available to a moving observer and requires that the direction of heading be computed as well. Research on tau has taken two forms. First, researchers have investigated whether people and animals are sensitive to time to contact and time to passage. Second, they have stud- ied whether performance is actually based upon a perceptual derivation of tau. In summary, the literature suggests that time to contact and time to passage are accurately perceived; however, whether this perceptual feat is due to an apprecia- tion of tau is currently a point of contention. Optical expansion, or looming, evokes defensive postures in adults, animals (Schiff, 1965), and human infants (Bower, Broughton, & Moore, 1970). The assumption is that these de- fensive actions are motivated by a perception that the ex- panding object is on an imminent collision course. Although it is tempting to think of the link between looming and im- pending collision as being innate, human infants do not clearly show behaviors that can be deﬁned as defensive in these conditions until 9 months of age (Yonas et al., 1977). Adults are quite accurate in making time-to-contact judg- ments (Schiff & Oldak, 1990; Todd, 1981). Todd’s data show relative time-to-contact judgments to be sensitive to less than 100-ms time differences. Relative time-to-passage judgments are less accurate, requiring differences of about 500 ms (Kaiser & Mowafy, 1993). How people actually make time-to-contact judgments is currently a topic of debate. In a review of the literature, Wann (1996) found that empirical support for the tau proposal was weak and that other optical variables, such as indexes of rela- tive distance, could account for research ﬁndings as well as tau. Recently, Tresilian (1999) provided a revised tau hypoth- esis in which it is acknowledged that the effective informa- tion in time-to-contact situations is task- and context-speciﬁc, and moreover that it involves the utilization of multiple cues from diverse sources. Intercepting a Fly Ball In order to catch a ﬂy ball, players must achieve two goals: First, they must get themselves to the location where the ball will land; and second, they must catch it. It seems reasonable to assume that satisfying the ﬁrst goal of interception would require a determination of the ball’s landing location, but this is not necessarily so. If a player looks at the ball in ﬂight and runs so that the ball’s perceived trajectory follows a straight path, then the ball will intersect the player’s path on its de- scent (McBeath, Shaffer, & Kaiser, 1995). If players fol- lowed this simple control heuristic, then they would run along curved paths to the location of the ball’s landing. If, in- stead, they knew where the ball would land, then they would run to that place in a straight line. In fact, outﬁelders run on curved paths that closely follow the predictions of the control heuristic formulation (McBeath et al., 1995). (See the chapter by Heuer in this volume for a discussion of motor control.) This ﬁnal experimental ﬁnding clearly points to the difﬁ- culty of disentangling conscious visual perceptions from the visual control of actions. Ask baseball players what they do when they catch ﬂy balls, and they will tell you that they see the ball moving through space and run to where they can catch it. Without doubt, they perceive the ball to be moving in depth. On the other hand, the control heuristic that guides their run- ning does not entail a representation of three-dimensional space. The heuristic applies to a two-dimensional representa- tion of the ball’s trajectory in the virtual image plan deﬁned by their line of sight to the ball. CONCLUSION In perceiving depth and events, the relevant information is both limited and abundant. Viewed in isolation, visual infor- mation is almost always found lacking in its ability to uniquely specify those aspects of the environment to which it relates. However, combining different informational sources leads to 230 Depth Perception and the Perception of Events a more satisfactory state of affairs. In general, the more com- plex the visual scene, the more well speciﬁed it becomes. Movement- and goal-directed behaviors are complications that add considerably to the sufﬁciency of optical information for specifying environmental layout and events; however, their study has recently led to the following conundrum: Con- scious visual perceptions and visually guided actions do not always reﬂect a common underlying representation. For ex- ample, geographical slant is grossly overestimated; however, a visually guided adjustment of perceived slant is accurate. When catching a baseball, players perceive themselves to be moving in a three-dimensional environment even though the visual guidance of their running path is controlled by heuris- tics applied to a two-dimensional representation of the scene. The disparity between awareness and action in these cases may reﬂect the functioning of multiple perceptual systems. Looking to the future, we see at least three developments that should have a signiﬁcant impact on research on how peo- ple perceive depth and events. These developments include (a) improvements in research technology, (b) increased breadth in the interdisciplinary nature of research, and © in- creased sophistication in the theoretical approach. Perceptual research has beneﬁted enormously from com- puter technology. For example, Johansson (1950) used com- puters to create moving point-light displays on an oscilloscope thereby establishing the ﬁeld of event perception. Current computer systems allow researchers to create almost any imaginable scene. Over the last 10 years, immersive displays have become available. Immersive displays surround ob- servers and allow them to move and interact within a virtual environment. Head-mounted displays present images with small screens in front of the eyes and utilize tracking systems to register the position and orientation of the head and other tracked parts of the body.Another immersive display system is the Cave Automatic Virtual Environment, CAVE, which is a room having rear-projected images. The observer’s head is tracked and the projected images transform in a manner consistent with the observer’s movement through a three- dimensional environment. Such immersive display systems allow researchers to control optical variables that heretofore could only be manipulated within the conﬁnes of a computer terminal. Given the increased availability of immersive dis- play systems, we expect to see more investigations of per- ceptions in situations entailing the visual control of action. Understanding the perception of space and events is of in- terest to a wide variety of disciplines. The current chapter has emphasized the psychophysical perspective, which relates rel- evant optical information to perceptual sensitivities. However, within such ﬁelds as computer science and cognitive neuro- science, there is also considerable research on this topic. Computer scientists are often interested in automating per- ceptual feats, such as the recovery of three-dimensional struc- ture from optical motion information, and comparisons of digital and biological algorithms have proven to be useful (Marr, 1982). Another area of computer science that is ripe for interdisciplinary collaboration is in the computer-graphics animation of events. Interestingly, many movies today em- ploy methods of motion capture to create computer-animated actors. These methods entail placing sensors on the head and joints of real actors and recovering an animation of a stick ﬁgure that can be ﬂeshed out in graphics. One cannot help but think of Johansson’s point-light walker displays when view- ing such a motion capture system in use. Currently, there is considerable work attempting to create synthetic actors di- rectly with algorithms. Perceptual scientists should be able to learn a lot by studying what works and what does not in this attempt to create synthetic thespians. Just as the pictorial depth cues were ﬁrst discovered by artists and then articu- lated by psychologists, the study of computer-simulated events should help us better understand what information is needed to evoke the perceptions of such natural motions as a person walking, and perhaps more generally, perceptions of animacy and purpose. Research in cognitive neuroscience has had an increasing impact on perceptual theory, and this trend is likely to con- tinue. Advances in clinical case studies, functional brain imaging, and animal research have greatly shaped our current conceptions of perceptual processing. For example, the anatomical signiﬁcance of the dorsal and ventral cortical pathways is currently receiving a lot of attention (Creem & Profﬁtt, 2001). These two pathways are the dominant visual processing streams in the cortex; however, there are many others visual streams in the brain. We have much to learn from functional anatomy that will help us constrain and de- velop our theoretical conceptions. Finally, there have been a number of recent advances in the sophistication of our theoretical approach. One of the most notable of these was made recently by Cutting and Vishton (1995). Every text on depth perception provides a list of depth cues, as does the current chapter. How these cues are combined is still much debated. Given the huge number of cues, however, an account of how depth is perceived in the context of all possible combinations of these variables is probably unattainable. On the other hand, Cutting and Vishton showed that there is much to be gained by investi- gating the range of efﬁcacy of different cues. For example, binocular disparity is useful at near distances but not far ones, whereas occlusion is equally useful at all distances. Looking at the problem of depth perception from this perspective mo- tivates a search for the conditions under which information is References 231 useful. A related theoretical approach is seen in the search for perceptual heuristics. Heuristics are simple processing rules having a precision that is no better than what is needed to ef- fectively guide behavior. The control heuristics engaged when catching baseballs are examples (McBeath et al., 1995). From a pragmatic perspective, optical information is useful to the degree that it helps inform the requirements de- ﬁned by the demands of the task at hand. REFERENCES Amorim, M.-A., Loomis, J. M., & Fukusima, S. S. (1998). Repro- duction of object shape is more accurate without the continued availability of visual information. Perception, 27, 69–86. Andersen, G. J., & Braunstein, M. L. (1983). Dynamic occlusion in the perception of rotation in depth. Perception & Psychophysics, 34, 356–362. Baird, J. C., & Wagner, M. (1991). Transformation theory of size judgment. Journal of Experimental Psychology: Human Percep- tion and Performance, 17, 852–864. Barclay, C. D., Cutting, J. E., & Kozlowski, L. T. (1978). Temporal and spatial factors in gait perception that inﬂuence gender recog- nition. Perception & Psychophysics, 23, 145–152. Bardy, B. G., Warren W. H., & Kay, B. A. (1999). The role of central and peripheral vision in postural control during walking. Percep- tion & Psychophysics, 61, 1356–1368. Berkeley, G. (1709). An essay towards a new theory of vision. London: J. M. Dent. Bertamini, M., & Profﬁtt, D. R. (2000). Hierarchical motion organi- zation inrandomdot conﬁgurations. Journal of Experimental Psy- chology: Human Perception and Performance, 26, 1371–1386. Bertamini, M., Yang, T. L., & Profﬁtt, D. R. (1998). Relative size perception at a distance is best at eye level. Perception & Psy- chophysics, 60, 673–682. Bertenthal, B. I., & Bai, D. L. (1989). Infants’ sensitivity to optical ﬂow for controlling posture. Developmental Psychology, 25, 936–945. Bertenthal, B. I., & Pinto, J. (1993). Complementary processes in the perception and production of human movements. In E. Thelan & L. Smith (Eds.), A dynamic systems approach to development: Applications (pp. 209–239). Cambridge, MA: MIT Press. Bertenthal, B. I., Profﬁtt, D. R., & Cutting, J. E. (1984). Infant sen- sitivity to ﬁgural coherence in biomechanical motions. Journal of Experimental Child Psychology, 37, 171–178. Berthoz, A., Istraël, I., George-François, P., Grasso, R., & Tsuzuku, T. (1995). Spatial memory of body linear displacement: What is being stored? Science, 269, 95–98. Berthoz, A., Lacour, M., Soechting, J. F., & Vidal, P. P. (1979). The role of vision in the control of posture during linear motion. Progress in Brain Research, 50, 197–209. Bhalla, M., & Profﬁtt, D. R. (1999). Visual-motor recalibration in geographical slant perception. Journal of Experimental Psychol- ogy: Human Perception and Performance, 25, 1076–1096. Bingham, G. P. (1987). Kinematic form and scaling: Further inves- tigations on the visual perception of lifted weight. Journal of Experimental Psychology: Human Perception and Performance, 13, 155–177. Bingham, G. P. (1993). Perceiving the size of trees: Form as infor- mation about scale. Journal of Experimental Psychology: Human Perception and Performance, 19, 1139–1161. Bingham, G. P., & Pagano, C. C. (1998). The necessity of a perception-action approach to deﬁnite distance perception: Monocular distance perception to guide reaching. Journal of Ex- perimental Psychology: Human Perception and Performance, 24, 145–168. Bingham, G. P., Schmidt, R. C., & Rosenblum, L. D. (1995). Hefting for a maximum distance throw: A smart perceptual mechanism. Journal of Experimental Psychology: Human Perception and Performance, 15, 507–528. Bingham, G. P., Schmidt, R. C., & Zaal, F. T. J. M. (1999). Visual perception of the relative phasing of human limb movements. Perception & Psychophysics, 2, 246–258. Bingham, G. P., Zaal, F., Robin, D., & Shull, J. A. (2000). Distor- tions in deﬁnite distance and shape perception as measured by reaching without and with haptic feedback. Journal of Experi- mental Psychology: Human Perception and Performance, 26, 1436–1460. Blake, A., & Marinos, C. (1990). Shape from texture: Estimation, isotropy and moments. Journal of Artiﬁcial Intelligence, 45, 323–380. Blake, R. (1993). Cats perceive biological motion. Psychological Science, 4, 54–57. Bles, W. (1981). Stepping around: Circular vection and Coriolis ef- fects. In J. B. Long & A. D. Baddeley (Eds.), Attention and per- formance (Vol. 9, pp. 47–61). Hillsdale, NJ: Erlbaum. Borjesson, E., & von Hofsten, C. (1975). A vector model for per- ceived object rotation and translation in space. Psychological Research, 38, 209–230. Bower, T. G. R., Broughton, J. M., & Moore, M. K. (1970). The co- ordination of visual and tactile input in infants. Perception & Psychophysics, 8, 51–53. Brandt, T., Dichgans, J., & Koenig, E. (1973). Differential effects of central versus peripheral vision on egocentric and exocentric motion perception. Experimental Brian Research, 16, 476–491. Brandt, T., Wist, E. R., & Dichgans, J. (1975). Foreground and background in dynamic spatial orientation. Perception & Psy- chophysics, 17, 497–503. Braunstein, M. L. (1976). Depth perception through motion . New York: Academic Press. Braunstein, M. L. (1994). Decoding principles, heuristics and infer- ence in visual perception. In G. Johansson, S. S. Bergstrom, & 232 Depth Perception and the Perception of Events W. Epstein (Eds.), Perceiving events and objects (pp. 436–446). Hillsdale, NJ: Erlbaum. Bruno, N., & Cutting, J. E. (1988). Minimodularity and the percep- tion of layout. Journal of Experimental Psychology: General, 117, 161–170. Carello, C., Grosofsky, A., Reichel, F., Solomon, H. Y., & Turvey, M. T. (1989). Perceiving what is reachable. Ecological Psychol- ogy, 1, 27–54. Caudek, C., & Domini, F. (1998). Perceived orientation of axis of rotation in structure-from-motion. Journal of Experimental Psy- chology: Human Perception and Performance, 24, 609–621. Caudek, C., Domini, F., & Di Luca, M. (in press). Illusory 3D rota- tion induced by dynamic image shading. Perception & Psy- chophysics . Chapanis, A., & Mankin, D. A. (1967). The vertical-horizontal illusion in a visually rich environment. Perception & Psy- chophysics, 2, 249–255. Clark, J., & Yuille. A. (1990). Data fusion for sensory information processing systems . Boston: Kluwer. Creem, S. H., & Profﬁtt, D. R. (2001). Deﬁning the cortical visual systems: “What,” “where,” and “how.” Acta Psychologica, 107, 43–68. Crowell, J. A., & Banks, M. S. (1993). Perceiving heading with dif- ferent retinal regions and types of optic ﬂow. Perception & Psy- chophysics, 53, 325–337. Cumming, B. G., Johnston, E. B., & Parker, A. J. (1991). Vertical disparities and perception of three-dimensional shape. Nature, 349, 411–413. Cutting, J. E., & Kozlowski, L. T. (1977). Recognizing friends by their walk: Gait perception without familiarity cues. Bulletin of the Psychonomic Society, 9, 353–356. Cutting, J. E., & Millard, R. T. (1984). Three gradients and the per- ception of ﬂat and curved surfaces. Journal of Experimental Psy- chology: General, 113, 198–216. Cutting, J. E., & Profﬁtt, D. R. (1982). The minimum principle and the perception of absolute, common, and relative motions. Cog- nitive Psychology, 14, 211–246. Cutting, J. E., Springer, K., Braren, P. A., & Johnson, S. H. (1992). Wayﬁnding on foot from information in retinal, not optical, ﬂow. Journal of Experimental Psychology: General, 102, 41–72, 129. Cutting, J. E., & Vishton, P. M. (1995). Perceiving layout and know- ing distances: The integration, relative potency, and contextual use of different information about depth. In W. Epstein & S. Rogers (Eds.), Perception of space and motion (pp. 69–117). San Diego, CA: Academic Press. Dichgans, J., & Brandt, T. (1978). Visual-vestibular interaction: Effects on self-motion perception and postural control. In H. Leibowitz & H.-L. Teuber (Eds.), Handbook of sensory physiol- ogy (pp. 755–804). New York: Springer-Verlag. Domini, F., & Braunstein, M. L. (1998). Recovery of 3-D structure from motion is neither euclidean nor afﬁne. Journal of Experi- mental Psychology: Human Perception and Performance, 24, 1273–1295. Domini, F., & Caudek, C. (1999). Perceiving surface slant from de- formation of optic ﬂow. Journal of Experimental Psychology: Human Perception and Performance, 25, 426–444. Domini, F., Caudek, C., & Profﬁtt, D. R. (1997). Misperceptions of angular velocities inﬂuence the perception of rigidity in the ki- netic depth effect. Journal of Experimental Psychology: Human Perception and Performance, 23, 1111–1129. Domini, F., Caudek, C., & Richman, S. (1998). Distortions of depth- order relations and parallelism in structure from motion. Percep- tion & Psychophysics, 60, 1164–1174. Domini, F., Caudek, C., Turner, J., & Favretto, A. (1998). Discrimi- nating constant from variable angular velocities in structure from motion. Perception & Psychophysics, 60, 747–760. Duncker, K. (1937). Induced motion. In W. D. Ellis (Ed. and Trans.), A source-book in Gestalt psychology . London: Routledge. (Original work published 1929) Epstein, W. (1977). Stability and constancy in visual perception . New York: Wiley. Epstein,W.(1982).Percept-perceptcouplings. Perception,11, 75–83. Erkelens, C. J., & Collewijn, H. (1985). Eye movements and stereopsis during dichoptic viewing of moving random-dot stereograms. Vision Research, 25, 1689–1700. Ernst, M. O., Banks, M. S., & Bülthoff, H. H. (2000). Touch can change visual slant perception. Nature Neuroscience, 3, 69–73. Fisher, S. K., & Ciuffreda, K. J. (1988). Accommodation and appar- ent distance. Perception, 17, 609–621. Fisher, M. H., & Kornmüller, A. E. (1930). Optokinetisch ausgelöste Bewegungswahrnehmungen und optokinetischer Nystagmus [The perception of movement and optokinetic nystagmus initi- ated by optokinesis]. Journal für Psychologie und Neurologie, 41, 273–308. Foley, J. M. (1968). Depth, size and distance in stereoscopic vision. Perception & Psychophysics, 3, 265–274. Foley, J. M. (1985). Binocular distance perception: Egocentric dis- tance tasks. Journal of Experimental Psychology: Human Per- ception and Performance, 11, 133–149. Fox, R., & McDaniels, C. (1982). The perception of biological mo- tion by human infants. Science, 218, 486–487. Fry, G. A., Bridgeman, C. S., & Ellerbrock, V. J. (1949). The effect of atmospheric scattering on binocular depth perception. American Journal of Optometry, 26, 8–15. Gårding, J. (1992). Shape from texture for smooth curved surfaces in perspective projection. Journal of Mathematical Imaging and Vision, 2, 327–350. Giannopulu, I., & Lepecq, J. C. (1998). Linear-vection chronometry along spinal and sagittal axes in erect man. Perception, 27, 363–372. Gibson, J. J. (1950). The perception of the visual world . Boston: Houghton Mifﬂin. References 233 Gibson, J. J. (1979). The ecological approach to visual perception . Boston: Houghton Mifﬂin. Gibson, J. J., Kaplan, G. A., Reynolds, H. N., & Wheeler, K. (1969). The change form visible to invisible: A study of optical transi- tion. Perception & Psychophysics, 5, 113–116. Gilden, D. L., & Profﬁtt, D. R. (1989). Understanding collision dy- namics. Journal of Experimental Psychology: Human Percep- tion and Performance, 15, 372–383. Gogel, W. C., & Da Silva, J.A. (1987). Familiar size and the theory of off-sized perceptions. Perception & Psychophysics, 41, 318–328. Hecht, H., & Profﬁtt, D. R. (1995). The price of expertise: Effects of experience on the water-level task. Psychological Science, 6, 90–95. Hershenson, M. (1989). Duration, time constant, and decay of the linear motion aftereffect as a function of inspection duration. Perception & Psychophysics, 45, 251–257. Hershenson, M., & Samuels, S. M. (1999). An airplane illusion: Apparent velocity determined by apparent distance. Perception, 28, 433–436. Higashiyama,A. (1996). Horizontal and vertical distance perception: The discorded-orientation theory. Perception & Psychophysics, 58, 259–270. Hildreth, E. C. (1984). The measurement of visual motion . Cambridge, MA: MIT Press. Hildreth, E. C., & Royden, C. S. (1998). Computing observer mo- tion from optical ﬂow. In T. Watanabe (Ed.), High-level motion processing: Computational, neurobiological, and psychophysi- cal perspectives (pp. 269–293). Cambridge, MA: MIT Press. Hoffman, D. D. (1982). Inferring local surface orientation from mo- tion ﬁelds. Journal of the Optical Society of America, 72, 888–892. Hoffman, D. D., & Flinchbaugh, B. E. (1982). The interpretation of biological motion. Biological Cybernetics, 42, 195–204. Hoffman, W. C. (1966). The Lie algebra of visual perception. Jour- nal of Mathematical Psychology, 3, 65–98. Howard, I. P. (1978). Recognition and knowledge of the water-level problem. Perception, 7, 151–160. Howard, I. P. (1986). The perception of posture, self-motion, and the visual vertical. In K. R. Boff, L. Kaufman, & J. P. Thomas (Eds.), Handbook of perception and human performance (pp. 18-1– 18-62). New York: Wiley. Hoyle, F. (1957). The black cloud . London: Heineman. Hume, D. (1978). A treatise on human nature . Oxford, UK: Oxford University Press. (Original work published 1739) Ittelson, W. H. (1960). Visual space perception . Berlin: Springer. Johansson, G. (1950). Conﬁguration in event perception . Uppsala, Sweden: Almqvist & Wiksell. Johansson, G. (1973). Visual perception of biological motion and a model for its analysis. Perception & Psychophysics, 14, 210– 211. Johnston, E. B. (1991). Systematic distortions of shape from stere- opsis. Vision Research, 31, 1351–1360. Kaiser, M. K., Jonides, J., & Alexander, J. (1986). Intuitive reason- ing about abstract and familiar physics problems. Memory & Cognition, 14, 308–312. Kaiser, M. K., & Mowafy, L. (1993). Optical speciﬁcation of time- to-passage: Observers’ sensitivity to global tau. Journal of Ex- perimental Psychology: Human Perception and Performance, 19, 1028–1040. Kaiser, M. K., Profﬁtt, D. R., Whelan, S. M., & Hecht, H. (1992). In- ﬂuence of animation on dynamical judgments. Journal of Exper- imental Psychology: Human Perception and Performance, 18, 669–690. Kammann, R. (1967). The overestimation of vertical distance and slope and its role in the moon illusion. Perception & Psy- chophysics, 2, 585–589. Kaplan, G. A. (1969). Kinetic disruption of optical texture: The per- ception of depth at an edge. Perception & Psychophysics, 6, 193–198. Kilpatrick, F. P., & Ittelson, W. H. (1953). The size-distance invari- ance hypothesis. Psychological Review, 60, 223–231. Koenderink, J. J. (1986). Optic ﬂow. Vision Research , 26, 161–179. Koenderink, J. J., & van Doorn, A. J. (1976). Geometry of binocular vision and a model for stereopsis. Biological Cybernetics, 21, 29–35. Koenderink, J. J., & van Doorn, A. J. (1991). Afﬁne structure from motion. Journal of the Optical Society of America, 8A, 377–385. Kubovy, M. (1986). The psychology of perspective and Renaissance art . Cambridge, UK: Cambridge University Press. Künnapas, T. (1968). Distance perception as a function of available visual cues. Journal of Experimental Psychology, 77, 523–529. Lackner, J. R. (1977). Induction of illusory self-rotation and nystag- mus by a rotating sound-ﬁeld. Aviation, Space, & Environmental Medicine, 48, 129–131. Lackner, J. R., & DiZio, P. (1984). Some efferent and somatosen- sory inﬂuences on body orientation and oculomotor control. In L. Spillmann & B. R. Wooten (Eds.), Sensory experience, adap- tation, and perception . Hillsdale, NJ: Erlbaum. Landy, M. S., Maloney, L. T., Johnston, E. B., & Young, M. (1995). Measurement and modeling of depth cue combination: In de- fense of weak fusion. Vision Research, 35, 389–412. Larish, J. F., & Flach, J. M. (1990). Sources of optical information useful for perception of speed of rectilinear self-motion. Journal of Experimental Psychology: Human Perception and Perfor- mance, 16, 295–302. Lee, D. N. (1974). Visual information during locomotion. In R. B. McLeod & H. Pick (Eds.), Perception: Essays in honor of J. J. Gibson (pp. 250–267). Ithaca, NY: Cornell University Press. Lee, D. N. (1980). Visuo-motor coordination in space-time. In G. E. Stelmach & J. Requin (Eds.), Tutorials in motor behavior (pp. 281–295). Amsterdam: North-Holland. Lee, D. N., & Aronson, E. (1974). Visual proprioceptive control of standing in human infants. Perception & Psychophysics, 15, 529–532. 234 Depth Perception and the Perception of Events Lee, D. N., & Lishman, J. R. (1975). Visual proprioceptive control of stance. Journal of Human Movement Studies, 1, 224–230. Leeuwenberg, E. L. J. (1971). A perceptual coding language for vi- sual and auditory patterns. American Journal of Psychology, 84, 307–349. Leeuwenberg, E. L. J. (1978). Quantiﬁcation of certain visual pat- tern similarities: Salience, transparency, similarity. In E. L. J. Leeuwenberg & H. F. J. M. Buffart (Eds.), Formal theories of visual perception (pp. 277–298). New York: Wiley. Loomis, J. M., Da Silva, J. A., Fujita, N., & Fukusima, S. S. (1992). Visual space perception and visually directed action. Journal of Experimental Psychology: Human Perception and Performance, 18, 906–921. Loomis, J. M., & Philbeck, J. W. (1999). Is the anisotropy of per- ceived 3-D shape invariant across scale? Perception & Psy- chophysics, 61, 397–402. Lunenburg, R. K. (1947). Mathematical analysis of binocular vi- sion . Princeton, NJ: Princeton University Press. Mach, E. (1875). Grundlinien der Lehre von der Bewegungsempﬁnd- ungen [Basic principles for the study of motion perception]. Leipzig, Germany: Engelmann. Mach, E. (1922). Die Analyse der Empﬁndungen [The analysis of sensations]. Jena, Germany: Gustav Fischer. Marr, D. (1982). Vision: A computational investigation into the human representation and processing of visual information . San Francisco: W. F. Freeman. Mass, J. B., Johansson, G., Janson, G., & Runeson, S. (1971). Motion perception I and II [Film]. Boston: Houghton Mifﬂin. Massaro, D. W. (1988). Ambiguity in perception and experimenta- tion. Journal of Experimental Psychology: General, 117, 417– 421. Mayhew, J. E., & Longuet-Higgins, H. C. (1982). A computational model of binocular depth perception. Nature, 297, 376–378. McAfee, E. A., & Profﬁtt, D. R. (1991). Understanding the surface orientation of liquids. Cognitive Psychology, 23, 483–514. McBeath, M. K., Shaffer, D. M., & Kaiser, M. K. (1995). How base- ball outﬁelders determine where to run to catch ﬂy balls. Science, 268, 569–573. McCloskey, M. (1983). Intuitive physics. Scientiﬁc American, 248, 122–130. McCloskey, M., Caramazza, A., & Green, B. (1980). Curvilinear motion in the absence of external forces: Naive beliefs about the motion of objects. Science, 210, 1139–1141. McCloskey, M., Washburn, A., & Felch, L. (1983). Intuitive physics: The straight-down belief and its origin. Journal of Experimental Psychology: Learning, Memory, & Cognition, 9, 636–649. McCready, D. (1985). On size, distance, and visual angle percep- tion. Perception & Psychophysics, 37, 323–334. Michotte, A. (1963). The perception of causality (T. R. Miles & E. Miles, Trans.). London: Methuen. Milner, A. D., & Goodale, M. A. (1995). The visual brain in action. Oxford, UK: Oxford University Press. Mon-Williams, M., & Tresilian, J. R. (1999). Some recent studies on the extraretinal contribution to distance perception. Perception, 28, 167–181. Nakamura, S., & Shimojo, S. (1999). Critical role of foreground stimuli in perceiving visually induced self-motion (vection). Perception, 28, 893–902. Nakayama, K., & Shimojo, S. (1992). Experiencing and perceiving visual surfaces. Science, 257, 1357–1363. Norman, J. F., & Todd, J. T. (1992). The visual perception of 3-dimensional form. In G. A. Carpenter & S. Grossberg (Eds.), Neural networks for vision and image processing (pp. 93–110). Cambridge, MA: MIT Press. Norman, J. F., Todd, J. T., Perotti, V. J., & Tittle, J. S. (1996). The visual perception of three-dimensional length. Journal of Experi- mental Psychology: Human Perception and Performance, 22, 173–186. Ogle, K. O. (1952). On the limits of stereoscopic vision. Journal of Experimental Psychology, 48, 50–60. Ono, H., & Comerford, T. (1977). Stereoscopic depth constancy. In W. Epstein (Ed.), Stability and constancy in visual perception: Mechanisms and processes (pp. 91–128). New York: Wiley. Ono, H., Rogers, B. J., Ohmi, M., & Ono, M. E. (1988). Dynamic occlusion and motion parallax in depth perception. Perception, 17, 255–266. Oyama, T. (1974). Perceived size and perceived distance in stereo- scopic vision and an analysis of their causal relations. Perception & Psychophysics, 16, 175–182. Pagano, C. C., & Bingham, G. P. (1998). Comparing measures of monocular distance perception: Verbal and reaching errors are not correlated. Journal of Experimental Psychology: Human Perception and Performance, 24, 1037–1051. Pentland, A. P. (1984). Local shading analysis. IEEE, Transactions on Analysis & Machine Intelligence, 6, 170–187. Philbeck, J. W., & Loomis, J. M. (1997). Comparison of two indica- tors of perceived egocentric distance under full-cue and reduced- cue conditions. Journal of Experimental Psychology: Human Perception and Performance, 23, 72–85. Philbeck, J. W., Loomis, J. M., & Beall, A. C. (1997). Visually per- ceived location is an invariant in the control of action. Percep- tion & Psychophysics, 59, 601–612. Predebon, J. (1991). Spatial judgments of exocentric extents in an open-ﬁeld situation: Familiar versus unfamiliar size. Percep- tion & Psychophysics, 50, 361–366. Profﬁtt, D. R., Bhalla, M., Gossweiler, R., & Midgett, J. (1995). Per- ceiving geographical slant. Psychonomic Bulletin & Review, 2, 409–428. Profﬁtt, D. R., Creem, S. H., & Zosh, W. (2001). Seeing mountains in mole hills: Geographical slant perception. Psychological Sci- ence, 12, 418–423. References 235 Profﬁtt, D. R., Cutting, J. E., & Stier, D. M. (1979). Perception of wheel-generated motions. Journal of Experimental Psychology: Human Perception and Performance, 5, 289–302. Profﬁtt, D. R., & Gilden, D. L. (1989). Understanding natural dy- namics. Journal of Experimental Psychology: Human Percep- tion and Performance, 15, 384–393. Ramachandran, V. S. (1988). Perceiving shape from shading. Scien- tiﬁc American, 259, 76–83. Regan, D., & Beverly, K. I. (1982). How do we avoid confounding the direction we are looking and the direction we are moving? Science, 215, 194–196. Restle, F. (1979). Coding theory of the perception of motion conﬁg- urations. Psychological Review, 86, 1–24. Rieser, J. J., Ashmead, D. H., Talor, C. R., & Youngquist, G. A. (1990). Visual perception and the guidance of locomo- tion without vision to previously seen targets. Perception, 19, 675–689. Rogers, B. J., & Bradshaw, M. F. (1993). Vertical disparities, differ- ential perspective and binocular stereopsis. Nature , 361, 253– 255. Rogers, S. (1996). The horizon-ratio relation as information for relative size in pictures. Perception & Psychophysics, 58, 142– 152. Rosenholtz, R., & Malik, J. (1997). Surface orientation from tex- ture: Isotropy or homogeneity (or both)? Vision Research, 37, 2283–2293. Ross, H. E. (1971). Do judgements of distance and greyness follow the physics of aerial perspective? In A. Garriga-Trillo, P. R. Minon, C. Garcia-Gallego, P. Lubin, A. Merino, & A. Villarino (Eds.), Fechner Day ’93: Proceedings of the International Soci- ety for Psychonomics (pp. 233–238). Palma de Mallorca, Spain: International Society for Psychophysics. Ross, H. E. (1974). Behavior and perception in strange environ- ments . London: George Allen & Unwin. Royden, C. S., Banks, M. S., & Crowell, J. A. (1992). The percep- tion of heading during eye movements. Nature , 360 , 583–585. Rubin, E. (1927). Visuell whrgenommene wirkliche Bewegungen [Visually perceived actual motion]. Zeitschrift fur Psychologie, 103, 384–392. Runeson, S. (1983). On visual perception of dynamic events. Acta Universitatis Upsaliensis: Studia Psychologica Upsaliensia (Series 9) . (Original work published 1977) Runeson, S., & Frykholm, G. (1981). Visual perception of lifted weight. Journal of Experimental Psychology: Human Percep- tion and Performance, 7, 733–740. Runeson, S., & Frykholm, G. (1983). Kinematic speciﬁcation of dynamics as an informational basis for person and action per- ception: Expectation, gender recognition, and deceptive inten- tion. Journal of Experimental Psychology: General, 112, 585–615. Runeson, S., Juslin, P., & Olsson, H. (2000). Visual perception of dynamic properties: Cue heuristics versus direct-perceptual competence. Psychological Review, 107, 525–555. Schiff, W. (1965). Perception of an impending collision: A study of visually directed avoidant behavior. Psychological Monograph: General and Applied, 79, (11, Whole No. 604). Schiff, W., & Oldak, R. (1990). Accuracy of judging time-to-arrival: Effects of modality, trajectory, and gender. Journal of Experi- mental Psychology: Human Perception and Performance, 16, 303–316. Sedgwick, H. (1973). The visible horizon: A potential source of visual information for the perception of size and distance. Dissertation Abstracts International, 34, 1301–1302B. (UMI No. 73-22, No. 03B 530) Sedgwick, H. (1986). Space perception. In K. R. Boff, L. Kaufman, & J. P. Thomas (Eds.), Handbook of perception and human perfor- mance (Vol. 1, pp. 1–57). New York: Wiley. Shimojo, S., Silverman, G. H., & Nakayama, K. (1989). Occlusion and the solution to the aperture problem for motion. Vision Re- search, 29, 619–626. Sinai, M. J., Ooi, T. L., & He, Z. J. (1998). Terrain inﬂuences the ac- curate judgement of distance. Nature, 395, 497–500. Stevens, K. A. (1981). The information content of texture gradients. Biological Cybernetics, 42, 95–105. Stoffregen, T. A. (1985). Flow structure versus retinal location in the optical control of stance. Journal of Experimental Psychology: Human Perception and Performance, 11, 554–565. Sumi, S. (1984). Upside down presentation of the Johansson mov- ing light spot pattern. Perception, 13, 283–286. Thomson, J. A. (1983). Is continuous visual monitoring necessary in visually guided locomotion? Journal of Experimental Psychol- ogy: Human Perception and Performance, 9, 427–443. Tittle, J. S., & Perotti, V. J. (1997). The perception of shape and curvedness from binocular stereopsis and structure from motion. Perception & Psychophysics, 59, 1167–1179. Todd, J. T. (1981). Visual information about moving objects. Jour- nal of Experimental Psychology: Human Perception and Perfor- mance, 7, 795–810. Todd, J. T., & Bressan, P. (1990). The perception of 3-dimensional afﬁne structure from minimal apparent motion sequences. Per- ception & Psychophysics, 48, 419–430. Todd, J. T., Koenderink, J. J., van Doorn, A. J., & Kappers, A. M. (1996). Effects of changing viewing conditions on the per- ceived structure of smoothly curved surfaces. Journal of Exper- imental Psychology: Human Perception and Performance, 22, 695–706. Todd, J. T., & Mingolla, E. (1983). Perception of surface curvature and direction of illumination from patterns of shading. Journal of Experimental Psychology: Human Perception and Perfor- mance, 9, 583–595. 236 Depth Perception and the Perception of Events Todd, J. T., Reichel, F. D. (1989). Ordinal structure in the visual per- ception and cognition of smoothly curved surfaces. Psychologi- cal Review, 96, 643–657. Todd, J. T., & Warren, W. H., Jr. (1982). Visual perception of rela- tive mass in dynamic events. Perception, 11, 325–335. Tresilian, J. R. (1991). Approximate information sources and perceptual variables in interceptive timing. Journal of Experi- mental Psychology: Human Perception and Performance, 20, 154–173. Tresilian, J. R. (1999). Analysis of recent empirical challenges to an account of interceptive timing. Perception & Psychophysics, 61, 515–528. Ullman, S. (1979). The interpretation of visual motion . Cambridge, MA: MIT Press. van den Berg, A. V. (1992). Robustness of perception of heading from optic ﬂow. Vision Research, 32, 1285–1296. van den Berg, A. V., & Brenner, E. (1994). Humans combine the optic ﬂow with static depth cues for robust perception of head- ing. Vision Research, 34, 2153–2167. Vicario, G. B., & Bressan, P. (1990). Wheels: A new illusion in the perception of rolling objects. Perception, 19, 57–61. Wallach, H., & O’Connell, D. N. (1953). The kinetic depth effect. Journal of Experimental Psychology, 45, 205–217. Wann, J. P. (1996). Anticipating arrival: Is the tau-margin a specious theory? Journal of Experimental Psychology: Human Percep- tion and Performance, 22, 1031–1048. Warren, W. H. (1995). Self-motion: Visual perception and visual control. In W. Epstein & S. Rogers (Eds.), Perception of space and motion (pp. 263–325). San Diego, CA: Academic Press. Warren, W. H. (1998). The state of ﬂow. In T. Watanabe (Ed.), High- level motion processing: Computational, neurobiological, and psychophysical perspectives (pp. 315–358). Cambridge, MA: MIT Press. Warren, W. H., Kay, B. A., & Yilmaz, E. H. (1996). Visual control of posture during walking: Functional speciﬁcity. Journal of Experimental Psychology: Human Perception and Performance, 22, 818–838. Warren, W. H., & Kurtz, K. J. (1992). The role of central and pe- ripheral vision in perceiving the direction of self-motion. Per- ception & Psychophysics, 51 , 443–454. Webb, J. A., & Aggarwal, J. K. (1982). Structure from motion of rigid and jointed objects. Artiﬁcial Intelligence, 19, 107–130. Wertheim, A. H. (1994). Motion perception during self-motion: The direct versus inferential controversy revisited. Behavioral & Brain Sciences, 17, 293–355. Wertheimer, M. (1937). Laws of organization in perceptual forms. In W. D. Ellis (Ed. & Trans.), A source book of Gestalt psychology (pp. 71–88). London: Routledge. (Original work published 1923) Wheatstone, C. (1938). Contributions to the physiology of vision: I. On some remarkable and hitherto unobserved phenomena of binocular vision. Philosophical Transactions of the Royal Soci- ety of London, 128, 371–394. Wraga, M. (1999). The role of eye height in perceiving affordances and object dimensions. Perception & Psychophysics, 61, 490–507. Wraga, M., Creem, S. H., & Profﬁtt, D. R. (2000). Perception-action dissociations of a walkable Müller-Lyer conﬁguration. Psycho- logical Science, 11, 239–243. Yang, T. L., Dixon, M. W., & Profﬁtt, D. R. (1999). Seeing big things: Overestimation of heights is greater for real objects than for objects in pictures. Perception, 28, 445–467. Yonas, A., Bechtold, A. G., Frankel, D., Gordon, F. R., McRoberts, G., Norcia, A., & Sternfels, S. (1977). Development of sensitiv- ity to information for impending collisions. Perception & Psy- chophysics, 21, 97–104. Yonas, A., Craton, L. G., & Thompson, W. B. (1987). Relative motion: Kinetic information for the order of depth at an edge. Perception & Psychophysics, 41, 53–59. Young, M. J., Landy, M. S., & Maloney, L. T. (1993). A perturbation analysis of depth perception from combinations of texture and motion cues. Vision Research, 33, 2685–2696. CHAPTER 9 Speech Production and Perception CAROL A. FOWLER 237 PHONOLOGICAL COMPETENCE 238 Phonetics 239 Phonology 241 Another Abstractness Issue: Exemplar Theories of the Lexicon 242 PHONOLOGICAL PLANNING 243 Speech Errors 243 Experimental Evidence About Phonological Planning 245 Disagreements Between the Theories of Dell, 1986, and Levelt et al., 1999 246 SPEECH PRODUCTION 247 How Acoustic Speech Signals Are Produced 247 Some Properties of Speech That a Production Theory Needs to Explain 248 Acoustic Targets of Speech Production 249 Gestural Targets of Speech Production 250 Evidence for Both Models: The Case of /r/ 251 SPEECH PERCEPTION 252 Phonetic Perception 252 Learning and Speech Perception 257 SUMMARY 261 REFERENCES 261 In order to convey linguistic messages that are accessible to listeners, speakers have to engage in activities that count in their language community as encodings of the messages in the public domain. Accordingly, spoken languages consist of forms that express meanings; the forms are (or, by other accounts, give rise to) the actions that make messages public and perceivable. Psycholinguistic theories of speech are con- cerned with those forms and their roles in communicative events. The focus of attention in this chapter will be on the phonological forms that compose words and, more speciﬁ- cally, on consonants and vowels. As for the roles of phonological forms in communicative events, four are central to the psycholinguistic study of speech. First, phonological forms may be the atoms of word forms as language users store them in the mental lexicon. To study this is to study phonological competence (that is, knowledge). Second, phonological forms retrieved from lexical entries may specify words in a mental plan for an utterance. This is phonological planning. Third, phonologi- cal forms are implemented as vocal tract activity, and to study this is to study speech production. Fourth, phonologi- cal forms may be the ﬁnest-grained linguistic forms that lis- teners extract from acoustic speech signals during speech perception. The main body of the chapter will constitute a review of research ﬁndings and theories in these four domains. Before proceeding to those reviews, however, I provide a caveat and then a setting for the reviews. The caveat is about the psycholinguistic study of speech. Research and theoriz- ing in the domains under review generally proceed indepen- dently and therefore are largely unconstrained by ﬁndings in the other domains (cf. Kent & Tjaden, 1997, and Browman & Goldstein, 1995a, who make a similar comment). As my review will reveal, many theorists have concluded that the relevant parts of a communicative exchange (phonological competence, planning, production, and perception) ﬁt to- gether poorly. For example, many believe that the forms of phonological competence have properties that cannot be implemented as vocal tract activity, so that the forms of lan- guage cannot literally be made public. My caveat is that this kind of conclusion may be premature; it may be a conse- quence of the independence of research conducted in the four domains. The stage-setting remarks just below will suggest why we should expect the ﬁt to be good. Preparation of this chapter was supported by NICHD grant HD-01994 and NIH grants DC-02717 and DC-03782 to Haskins Laboratories. 238 Speech Production and Perception In the psycholinguistic study of speech, as in psycholin- guistics generally (see chapter by Treiman, Clifton, Meyer, & Wurm in this volume), the focus of attention is almost solely on the individual speaker/hearer and speciﬁcally on the mem- ory systems and mental processing that underlie speaking or listening. It is perhaps this sole focus of attention that has fos- tered the near autonomy of investigations into the various components of a communicative exchange just described. Outside of the laboratory, speaking almost always occurs in the context of social activity; indeed, it is, itself, prototypi- cally a social activity. This observation matters, and it can help to shape our thinking about the psycholinguistics of speech. Although speaker/hearers can act autonomously, and sometimes do, often they participate in cooperative activities with others; jointly the group constitutes a special purpose sys- tem organized to achieve certain goals. Cooperation requires coordination, and speaking helps to achieve the social coordi- nations that get conjoint goals accomplished (Clark, 1996). How, at the phonological level of description, can speech serve this role? Speakers speak intending that their utterance communicate to relevant listeners. Listeners actively seek to identify what a talker said as a way to discover what the talker intended to achieve by saying what he or she said. Required for successful communication is achievement of a relation of sufﬁcient equivalence between messages sent and received. I will refer to this relation, at the phonological level of de- scription, as parity (Fowler & Levy, 1995; cf. Liberman & Whalen, 2000). That parity achievement has to be a typical outcome of speech is one conclusion that emerges from a shift in per- spective on language users, a shift from inside the mind or brain of an individual speaker/listener to the cooperative activities in which speech prototypically occurs. Humans would not use speech to communicate if it characteristically did not. This conclusion implies that the parts of a commu- nicative exchange (competence, planning, production, per- ception) have to ﬁt together pretty well. A second observation suggests that languages should have parity-fostering properties. The observation is that language is an evolved, not an invented, capability of humans. This is true of speech as well as of the rest of language. There are adaptations of the brain and the vocal tract to speech (e.g., Lieberman, 1991), suggesting that selective pressures for ef- ﬁcacious use of speech shaped the evolutionary development of humans. Following are two properties that, if they were character- istic of the phonological component of language, would be parity fostering. The ﬁrst is that phonological forms, here consonants and vowels, should be able to be made public and therefore accessible to listeners. Languages have forms as well as meanings exactly because messages need to be made public to be communicated. The second parity-fostering characteristic is that the elements of a phonological message should be preserved throughout a communicative exchange. That is, the phonological elements of words that speakers know in their lexicons should be the phonological ele- ments of words that they intend to communicate, they should be units of action in speech production, and they should be objects of speech perception. If the elements are not pre- served—if, say, vocal tract actions are not phonological things and so acoustic signals cannot specify phonological things—then listeners have to reconstruct the talker’s phono- logical message from whatever they can perceive. This would not foster achievement of parity. The next four sections of the chapter review the literature on phonological competence, planning, production, and per- ception. The reviews will accurately reﬂect the near inde- pendence of the research and theorizing that goes on in each domain. However, I will suggest appropriate links between domains that reﬂect the foregoing considerations. PHONOLOGICAL COMPETENCE The focus here is on how language users know the spoken word forms of their language, concentrating on the phono- logical primitives, consonants and vowels ( phonetic or phonological segments). Much of what we know about this has been worked out by linguists with expertise in phonetics or phonology. However, the reader will need to keep in mind that the goals of a phonetician or phonologist are not neces- sarily those of a psycholinguist. Psycholinguists want to know how language users store spoken words. Phoneticians seek realistic descriptions of the sound inventories of lan- guages that permit insightful generalizations about universal tendencies and ranges of variation cross-linguistically. Pho- nologists seek informative descriptions of the phonological systematicities that languages evidence in their lexicons. These goals are not psychologically irrelevant, as we will see. However, for example, descriptions of phonological word forms that are most transparent to phonological regularities may or may not reﬂect the way that people store word forms. This contrast will become apparent below when theories of linguistic phonology are compared speciﬁcally to a recent hypothesis raised by some speech researchers that lexical memory is a memory of word tokens (exemplars), not of abstract word types. Word forms have an internal structure, the component parts of which are meaningless. The consonants and vowels are also discrete and permutable. This is one of the ways Phonological Competence 239 in which language makes “inﬁnite use of ﬁnite means” (Von Humbolt, 1936/1972; see Studdert-Kennedy, 1998). There is no principled limit on the size of a lexicon having to do with the number of forms that can serve as words. And we do know a great many words; Pinker (1994) estimates about 60,000 in the lexicon of an average high school graduate. This is despite the fact that languages have quite limited numbers of consonants and vowels (between 11 and 141 in Maddieson’s (1984) survey of 317 representative languages of the world). In this regard, as Abler (1989) and Studdert-Kennedy (1998) observe, languages make use of a “particulate princi- ple” also at work in biological inheritance and chemical com- pounding, two other domains in which inﬁnite use is made of ﬁnite means. All three of these systems are self-diversifying in that, when the discrete particulate units of the domain (phonological segments, genes, chemicals) combine to form larger units, their effects do not blend but, rather, remain distinct. (Accordingly, words that are composed of the same phonological segments, such as “cat,” “act,” and “tack,” remain distinct.). In language, this in part underlies the un- boundedness of the lexicon and the unboundedness of what we can use language to achieve. Although some writing about speech production suggests that, when talkers coarticu- late, that is, when they temporally overlap the production of consonants and vowels in words, the result is a blending of the properties of the consonants and vowels (as in Hockett’s, 1955, famous metaphor of coarticulated consonants and vowels as smashed Easter eggs), this is a mistaken under- standing of coarticulation. Certainly, the acoustic speech sig- nal at any point in time is jointly caused by the production of more than one consonant or vowel. However, the information in its structure must be about discrete consonants and vowels for the particulate principle to survive at the level of lexical knowledge. Phonetics Feature Systems From phonetics we learn that consonants and vowels can be described by their featural attributes, and, when they are, some interesting cross-linguistic tendencies are revealed. Feature systems may describe consonants and vowels largely in terms of their articulatory correlates, their acoustic corre- lates, or both. A feature system that focuses on articulation might distinguish consonants primarily by their place and manner of articulation and by whether they are voiced or unvoiced. Consider the stop consonants in English. Stop is a manner class that includes oral and nasal stops. Production of these consonants involves transiently stopping the ﬂow of air through the oral cavity. The stops of English are conﬁgured as shown. Bilabial Alveolar Velar oral stops: voiced b d g unvoiced p t k nasal stops: voiced m n N The oral and nasal voiced stops are produced with the vocal folds of the larynx approximated (adducted); the oral voice- less stops are produced with the vocal folds apart (abducted). When the vocal folds are adducted and speakers exhale as they speak, the vocal folds cyclically open and close releas- ing successive puffs of air into the oral cavity. We hear a voic- ing buzz in consonants produced this way. When the vocal folds are abducted, air ﬂows more or less unchecked by the larynx into the oral cavity, and we hear such consonants as unvoiced. Compatible descriptions of vowels are in terms of height, backing, and rounding. Height refers to the height of the tongue in the oral cavity, and backing refers to whether the tongue’s point of closest contact with the palate is in the back of the mouth or the front. Rounding (and unroundedness) refers to whether the lips are protruded during production of the vowel as they are, for example, in the vowel in shoe . Some feature systems focus more on the acoustic realiza- tions of the features than on the articulatory realizations. One example of such a system is that

bundet

of Jakobson, Fant, and Halle (1962), who, nonetheless, also provide articulatory correlates of the features they propose. An example of a feature contrast of theirs that is more obviously captured in acoustic than articulatory terms is the feature [ grave]. Segments denoted as [ + grave] are described as having acoustic ene r gy that pre- dominates in the lower region of the spectrum. Examples of [ + grave] consonants are bilabials with extreme front articu- lations and uvulars with extreme back places of articulation. Consonants with intermediate places of articulation are [ – grave]. Despite the possible articulatory oddity of the fea- ture contrast [ grave], Jakobson, Fant, and Halle had reason to identify it as a meaningful contrast (see Ohala, 1996, for some reasons). Before turning to what one can learn by describing conso- nants and vowels in terms of their features, consider two addi- tional points that relate back to the stage-setting discussion above. First, many different feature systems have been pro- posed. Generally they are successful in describing the range of consonants and vowels in the world’s languages and in captur- ing the nature of phonological slips of the tongue that speakers make (see section titled “Speech Errors”). Both of these 240 Speech Production and Perception observations are relevant to a determination of how language users know the phonological forms of words. Nonetheless, there are differences among the systems that may have psy- chological signiﬁcance. One relates back to the earlier discus- sion of parity. I suggested there that a parity-fostering property of languages would be a common currency in which messages are stored, formulated, sent, and received so that the phonolog- ical form of a message is preserved throughout a communica- tive exchange.Withinthe contextof that discussion,a proposal that the features of consonants and vowels as language users know them are articulatory implies that the common currency is articulatory. A proposal that featural correlates are acoustic suggests that the common currency is acoustic. A second point is that there is a proposal in the literature that the properties of consonants and vowels on which lan- guage knowledge and use depends are not featural. Rather, the phonological forms of words as we know them consist of “gestures” (e.g., Browman & Goldstein, 1990). Gestures are linguistically signiﬁcant actions of the vocal tract. An exam- ple is the bilabial closing gesture that occurs when speakers of English produce /b/, /p/, or /m/. Gestures do not map 1:1 onto either phonological segments or features. For example, /p/ is produced by appropriately phasing two gestures, a bil- abial constriction gesture and a devoicing gesture. Because Browman and Goldstein (1986) propose that voicing is the default state of the vocal tract producing speech, /b/ is achieved by just one gesture, bilabial constriction. As for the sequences /sp/, /st/, and /sk/, they are produced by appropri- ately phasing a tongue tip (alveolar) constriction gesture for /s/ and another constriction gesture for /p/, /t/, or /k/ with a single devoicing gesture that, in a sense, applies to both con- sonants in the sequence. Browman and Goldstein (e.g., 1986) have proposed that words in the lexicon are speciﬁed as sequences of appropri- ately phased gestures (that is, as gestural scores ). In a parity- fostering system in which these are primitives, the common currency is gestural. This is a notable shift in perspective be- cause the theory gives primacy to public phonological forms (gestures) rather than to mental representations (features) with articulatory or acoustic correlates. Featural Descriptions and the Sound Inventories of Languages Featural descriptions of the sound inventories of languages have proven quite illuminating about the psychological factors that shape sound inventories. Relevant to our theme of languages’ developing parity-fostering characteristics, re- searchers have shown that two factors, perceptual distinctive- ness and articulatory simpliﬁcation (Lindblom, 1990), are major factors shaping the consonants and vowels that lan- guages use to form words. Perceptual distinctiveness is par- ticularly important in shaping vowel inventories. Consider two examples. One is that, as noted earlier, vowels may be rounded (with protruded lips) or unrounded. In Maddieson’s (1984) survey of languages, 6% of front vowels were rounded, whereas 93.5% of back vowels were rounded. The evident reason for the correlation between backing and rounding is perceptual distinctiveness. Back vowels are produced with the tongue’s constriction location toward the back of the oral cavity. This makes the cavity in front of the constriction very long. Rounding the lips makes it even longer. Front vowels are pro- duced with the tongue constriction toward the front of the oral cavity so that the cavity in front of the constriction is short. An acoustic consequence of backing/fronting is the fre- quency of the vowel’s second formant (i.e., the resonance as- sociated with the acoustic signal for the vowel that is second lowest in frequency [F2]). F2 is low for back vowels and high for front vowels. Rounding back vowels lowers their F2 even more, enhancing the acoustic distinction between front and back vowels (e.g., Diehl & Kluender, 1989; Kluender, 1994). A second example also derives from the study of vowel inventories. The most frequently occurring vowels in Maddieson’s (1984) survey were /i/ (a high front unrounded vowel as in heat), /a/ (a low central unrounded vowel as in hot ) and /u/ (a high back rounded vowel as in hoot ), occur- ring in 83.9% (/u/) to 91.5% (/i/) of the language sample. Moreover, of the 18 languages in the survey that have just three vowels, 10 have those three vowels. Remarkably, most of the remaining 8 languages have minor variations on the same theme. Notice that these vowels, sometimes called the point vowels, form a triangle in vowel space if the horizontal dimension represents front-to-back and the vertical dimen- sion vowel height: iu a Accordingly, they are as distinct as they can be articulatorily and acoustically. Lindblom (1986) has shown that a principle of perceptual distinctiveness accurately predicts the location of vowels in languages with more than three vowels. For example, it accurately predicts the position of the fourth and ﬁfth vowels of ﬁve-vowel inventories, the modal vowel in- ventory size in Maddieson’s survey. Consonants do not directly reﬂect a principle of perceptual dispersion as the foregoing conﬁguration of English stop consonants suggests. Very tidy patterns of consonants in voicing, manner, and place space are common, yet such patterns mean that phonetic space is being densely packed. An important consideration for consonants appears to be Phonological Competence 241 articulatory complexity. Lindblom and Maddieson (1988) classiﬁed consonants of the languages of the world into basic, elaborated, and complex categories according to the com- plexity of the articulatory actions required to produce them. They found that languages with small consonant inventories tend to restrict themselves to basic consonants. Further, lan- guages with elaborated consonants always have basic conso- nants as well. Likewise, languages with complex consonants (for example, the click consonants of some languages of Africa) always also have both basic and elaborated conso- nants as well. In short, language communities prefer con- sonants that are easy to produce. Does the foregoing set of observations mean that language communities value perceptual distinctiveness in vowels but articulatory simplicity in consonants? This is not likely. Lindblom (1990) suggests that the proper concept for under- standing popular inventories both of vowels and of conso- nants is that of “sufﬁcient contrast.” Sufﬁcient contrast is the equilibrium point in a tug-of-war between goals of perceptual distinctiveness and articulatory simplicity. The balance shifts toward perceptual distinctiveness in the case of vowel sys- tems, probably because vowels are generally fairly simple articulatorily. Consonants vary more in that dimension, and the balance point shifts accordingly. The major global observation here, however, is that the requirements of efﬁcacious public language use clearly shape the sound inventories of language. Achievement of parity matters. Features and Contrast: Onward to Phonology An important concept in discussions of feature systems is contrast. A given consonant or vowel can, in principle, be exhaustively described by its featural attributes. However, only some of those attributes are used by a language commu- nity to distinguish words. For example, in the English till, the ﬁrst consonant is /t/, an unvoiced, alveolar stop. It is also “aspirated” in that there is a longish unvoiced and breathy in- terval from the time that the alveolar constriction by the tongue tip is released until voicing for the following vowel begins. The /t/ in still is also an unvoiced, alveolar stop, but it is unaspirated. This is because, in the sequence /st/, although both the /s/ and the /t/ are unvoiced, there is just one devoic- ing gesture for the two segments, and it is phased earlier with respect to the tongue constriction gesture for /t/ than it is phased in till . Whereas a change in any of the voicing, manner, or place features can create a new word of English (voicing: dill; manner: sill; place: pill), a change in aspiration does not. Indeed, aspiration will vary due to rate of speaking and emphasis, but the /t/ in till will remain a /t/. Making a distinction between contrastive and noncon- trastive features historically allowed a distinction to be made also in how consonants and vowels were characterized. Char- acterizing them as phonological segments (or phonemes) involved specifying only their contrastive features. Charac- terizing them as phonetic segments (or phones) involved spec- ifying fairly exactly how they were to be pronounced. To a ﬁrst approximation, the contrastive/noncontrastive distinc- tion evolved into another relating to predictability that has had a signiﬁcant impact on how modern phonologists have char- acterized lexical word forms. Minimally, lexical word forms have to specify unpredictable features of words. These are ap- proximately contrastive features. That is, that the word mean- ing “medicine in a small rounded mass to be swallowed whole” (Mish, 1990) is pill, not, say, till, is just a fact about English language use. It is not predictable from any general phonological or phonetic properties of English. Language users have to know the sequence of phonological segments that compose the word. However, the fact that the /p/ is aspi- rated is predictable. Stressed-syllable initial unvoiced stops are aspirated in English. An issue for phonologists has been whether lexical word forms are abstract, specifying only un- predictable features (and so giving rise to differences between lexical and pronounced forms of words), or whether they are fully speciﬁed. The mapping of contrastive/noncontrastive onto pre- dictable/unpredictable is not exact. In context, some con- trastive features of words can be predictable. For example, if a consonant of English is labiodental (i.e., produced with teeth against lower lip as in /f/ or /v/), it must be a fricative. And if a word begins /skr/, the next segment must be [ + vocalic]. An issue in phonology has been to determine what should count as predictable and lexically unspeciﬁed. Deciding that determines how abstract in relation to their pro- nounced forms lexical entries are proposed to be. Phonology Most phonologists argue that lexical forms must be abstract with respect to their pronunciations. One reason that has loomed large in only one phonology (Browman & Goldstein’s, e.g., 1986, Articulatory Phonology) is that we do not pronounce the same word the same way on all occasions. Particularly, variations in speaking style (e.g., from formal to casual) can affect how a word is pronounced. Lexical forms, it seems (but see section titled “Another Abstractness Issue”), have to be abstracted away from detail that distinguishes those different pronunciations. A second reason given for ab- stract word forms is, as noted above, that some properties of word forms are predictable. Some linguists have argued that 242 Speech Production and Perception lexical entries should include just what is phonologically un- predictable about a word. Predictable properties can be ﬁlled in another way, by rule application, for example. A ﬁnal rea- son that words in the lexicon may be phonologically abstract is that the same morpheme may be pronounced differently in different words. For example, the preﬁxes on inelegant and imprecise are etymologically the same preﬁx, but the alveolar /n/ becomes labial /m/ before labial /p/ in imprecise . To cap- ture in the lexicon that the morpheme is the same in the two words, some phonologists have proposed that they be given a common form there. An early theory of phonology that focused on the second and third reasons was Chomsky and Halle’s (1968) genera- tive phonology. An aim there was to provide in the lexicon only the unpredictable phonological properties of words and to generate surface pronunciations by applying rules that pro- vided the predictable properties. In this phonology, the threshold was rather low for identifying properties as pre- dictable, and underlying forms were highly abstract. A recent theory of phonology that appears to have super- seded generative phonology and its descendents is optimality theory, ﬁrst developed by Prince and Smolensky (1993). This theory accepts the idea that lexical forms and spoken forms are different, but it differs markedly from generative phonol- ogy in how it gets from the one to the other. In optimality theory, there are no rules mediating lexical and surface forms. Rather, from a lexical form, a large num- ber of candidate surface forms are generated. These are eval- uated relative to a set of universal constraints. The constraints are ranked in language-particular ways, and they are violable. The surface form that emerges from the competition is the one that violates the fewest and the lowest ranked constraints. One kind of constraint that limits the abstractness of underly- ing forms is called a faithfulness constraint . One of these speciﬁes that lexical and surface forms must be the same. (More precisely, every segment or feature in the lexical entry must have an identical correspondent in the surface form, and vice versa.) This constraint is violated in imprecise, the lexi- cal form of which will have an /n/ in place of the /m/. A sec- ond constraint (the identical cluster constraint in Pulleyblank, 1997) requires that consonant clusters share place of articula- tion. It is responsible for the surface /m/. On the surface, this model is not plausible as a psycholog- ical one. That is, no one supposes that, given a word to say, the speaker generates lots of possible surface forms and then evaluates them and ends up saying the optimal one. But there are models that have this ﬂavor and are considered to have psychological plausibility. These are network models. In those models (e.g., van Orden, Pennington, & Stone, 1990), something input to the network (say, a written word) activates far more in the phonological component of the model than just the word’s pronunciation. Research suggests that this happens in humans as well (e.g., Stone, Vanhoy, & Van Orden, 1997). The activation then settles into a state reﬂect- ing the optimal output, that is, the word’s actual pronun- ciation. From this perspective, optimality theory may be a candidate psychological model of the lexicon. Another theory of phonology, articulatory phonology (Browman & Goldstein, 1986), is markedly different from both of those described above. It does not argue from pre- dictability or from a need to preserve a common form for the same morpheme in the lexicon that lexical entries are abstract. Indeed, in the theory, they are not very abstract. As noted earlier, primitive phonological forms in the theory are gestures. Lexical entries specify gestural scores. The lexical entries are abstract only with respect to variation due to speaking style. An attractive feature of their theory, as Browman and Goldstein (1995a) comment, is that phonol- ogy and phonetics are respectively macroscopic and micro- scopic descriptions of the same system. In contrast to this, in most accounts, phonology is an abstract, cognitive represen- tation, whereas phonetics is its physical implementation. In an account of language production incorporating articula- tory phonology, therefore, there need be no (quite mysteri- ous) translation from a mental to a physical domain (cf. Fowler, Rubin, Remez, & Turvey, 1980); rather, the same domain is at once physical and cognitive (cf. Ryle, 1949). Articulatory phonology is a candidate for a psychological model. Another Abstractness Issue: Exemplar Theories of the Lexicon Psychologists have recently focused on a different aspect of the abstractness issue. The assumption has been until recently that language users store word types, not word tokens, in the lexicon. That is, even though listeners may have heard the word boy a few million times, they have not stored memories of those few million occurrences. Rather, listeners have just one word boy in their lexicon. In recent years, this idea has been questioned, and some evidence has accrued in favor of a token or exemplar memory (see chapter by Goldstone & Kersten in this volume). The idea comes from theories of memory in cognitive psychol- ogy. Clearly, not all of memory is a type memory. We can recall particular events in our lives. Some researchers have suggested that exemplar memory systems may be quite pervasive. An example theory that has drawn the attention of Phonological Planning 243 speech researchers is Hintzman’s (e.g., 1986) memory model, MINERVA. In the model, input is stored as a trace, which consists of feature values along an array of dimensions. When an input is presented to the model, it not only lays down its own trace, but it activates existing traces to the ex- tent that they are featurally similar to it. The set of activated traces forms a composite, called the echo, which bears great resemblance to a type (often called a prototype in this litera- ture). Accordingly, the model can behave as if it stores types when it does not. In the speech literature, researchers have tested for an ex- emplar lexicon by asking whether listeners show evidence of retaining information idiosyncratic to particular occurrences of words, typically, the voice characteristics of the speaker. Goldinger (1996) provided an interesting test in which listen- ers identiﬁed words in noise. The words were spoken in 2, 6, or 10 different voices. In a second half of the test (after a delay that varied across subjects), he presented some words that had occurred in the ﬁrst half of the test. The tokens in the second half were produced by the same speaker who produced them in the ﬁrst half (and typically they were the same token) or were productions by a different speaker. The general ﬁnding was that performance identifying words was better if the words were repeated by the speaker who had pro- duced them in the ﬁrst half of the test. This across–test-half priming persisted across delays between test halves as long as one week. This study shows that listeners retain token- level memories of words (see also Goldinger, 1998). Does it show that these token-level memories constitute word forms in the mental lexicon? Not deﬁnitively. However, it is now incumbent on theorists who retain the claim that the lex- icon is a type memory to provide distinctively positive evi- dence for it. PHONOLOGICAL PLANNING Speakers are creators of linguistic messages, and creation requires planning. This is in part because utterances are syn- tactically structured so that the meaning of a sentence is dif- ferent from the summed meanings of its component words. Syntactic structure can link words that are distant in a sen- tence. Accordingly, producing a syntactically structured ut- terance that conveys an intended message requires planning units larger than a word. Planning may also be required to get the phonetic, including the prosodic, form of an utterance right. For many years, the primary source of evidence about planning for language production was the occurrence of spontaneous errors of speech production. In approximately the last decade other, experimentally generated, behavioral evidence has augmented that information source. Speech Errors Speakers sometimes make mistakes that they recognize as er- rors and are capable of correcting. For example, intending to say This seat has a spring in it, a speaker said This spring has a seat in it (Garrett, 1980), exchanging two nouns in the in- tended utterance. Or intending to say It’s the jolly green giant, a speaker said It’s the golly green giant (Garrett, 1980), antic- ipating the /g/ from green . In error corpora that researchers have collected (e.g., Dell, 1986; Fromkin, 1973; Garrett, 1980; Shattuck-Hufnagel, 1979), errors are remarkably sys- tematic and, apparently, informative about planning for speech production. One kind of information provided by these error corpora concerns the nature of planning units. Happily, they appear to be units that linguists have identiﬁed as linguistically coher- ent elements of languages. However, they do not include every kind of unit identiﬁed as signiﬁcant in linguistic theory. In the two examples above, errors occurred on whole words and on phonological segments. Errors involving these units are common, as are errors involving individual mor- phemes (e.g., point outed; Garrett, 1980). In contrast, sylla- ble errors are rare and so are feature errors (as in Fromkin’s, 1973, glear plue sky ). Rime (that is, the vowel and any postvocalic consonants of a syllable) errors occur, but conso- nant-vowel (CV) errors are rare (Shattuck-Hufnagel, 1983). This is not to say that syllables and features are irrelevant in speech planning. They are relevant, but in a different way from words and phonemes. Not only are the units that participate in errors tidy, but the kinds of errors that occur are systematic too. In the word error above, quite remarkably, two words exchanged places. Sometimes, instead, one word is anticipated, but it also oc- curs in its intended slot ( This spring has a spring in it ) or a word is perseverated ( This seat has a seat in it ). Sometimes, noncontextual substitutions occur in which a word appears that the speaker did not intend to say at all ( This sheep has a spring in it ). Additions and deletions occur as well. To a close approximation, the same kinds of errors occur on words and phonological segments. Errors have properties that have allowed inferences to be drawn about planning for speech production. Words exchange, anticipate, and perseverate over longer distances than do phonological segments. Moreover, word substitu- tions appear to occur in two varieties: semantic (e.g., saying 244 Speech Production and Perception summer when meaning to say winter ) and form-based (saying equivocal when meaning to say equivalent ). These observations suggested to Garrett (1980) that two broad phases of planning occur. At a functional level, lemmas (that is, words as semantic and syntactic entities) are slotted into a phrasal structure. When movement errors occur, lemmas might be put into the wrong phrasal slot, but because their syntactic form class determines the slots they are eligible for, when words anticipate, perseverate, or exchange, they are members of the same syntactic category. Semantic substitu- tion errors occur when a semantic neighbor of an intended word is mistakenly selected. At a positional level, planning concerns word forms rather than their meanings. This is where sound-based word substitutions may occur. For their part, phonological segment errors also have highly systematic properties. They are not sensitive, as word movement errors are, to the syntactic form class of the words involved in the errors. Rather, they are sensitive to phonolog- ical variables. Intended and erroneous segments in errors tend to be featurally similar, and their intended and actual slots are similar in two ways. They tend to have featurally similar segments surrounding them, and they come from the same syllable position. That is, onset (prevocalic) conso- nants move to other onset positions, and codas (postvocalic consonants) move to coda positions. These observations led theorists (e.g., Dell, 1986; Shattuck- Hufnagel, 1979) to propose that, in phonological planning, the phonemes that compose words to be said are slotted into syl- labic frames. Onsets exchange with onsets, because, when an onset position is to be ﬁlled, only onset consonants are candi- dates for that slot. There is something intuitively displeasing about this idea, but there is evidence for it, theorists have of- fered justiﬁcations for it, and there is at least one failed attempt to avoid proposing a frame (Dell, Juliano, & Govindjee, 1993). The idea of slotting the phones of a word into a structural frame is displeasing, because it provides the opportunity for speakers to make errors, but seems to accomplish little else. The phones of words must be serially ordered in the lexical entry. Why reselect and reorder them in the frame? One justiﬁ- cation has to do with productivity (e.g., Dell, 1986; Dell, Burger, & Svec, 1997). The linguistic units that most fre- quently participate in movement errors are those that we use productively. That is, words move, and we create novel sen- tences by selecting words and ordering them in new ways. Morphemes move, and we coin some words (e.g., videocas- sette ) by putting morphemes together into novel combinations. Phonemes move, and we coin words by selecting consonants and vowels and ordering them in new ways (e.g., smurf ). The frames for sentences (that is, syntactic structure) and for sylla- bles permit the coining of novel sentences and words that ﬁt the language’s constraints on possible sentences and possible words. Dell et al. (1993; see also Dell & Juliano, 1996) developed a parallel-distributed network model that allowed accurate sequences of phones to be produced without a frame-content distinction. The model nonetheless produced errors hitherto identiﬁed as evidence for a frame. (For example, errors were phonotactically legal the vast majority of the time, and con- sonants substituted for consonants and vowels for vowels.) However, the model did not produce anticipations, perse- verations, or exchanges, and, even with modiﬁcations that would give rise to anticipations and perseverations, it would not make exchange errors. So far, theories and models that make the frame-content distinction have the edge over any that lack it. Dell (1986) more or less accepted Garrett’s (1980) two- tiered system for speech planning. However, he proposed that the lexical system in which planning occurs has both feedforward (word to morpheme to syllable constituent to phone) links and feedback links, with activation of planned lexical units spreading bidirectionally. The basis for this idea was a set of ﬁndings in speech error corpora. One is that, al- though phonological errors do create nonwords, they create words at a greater than chance rate. Moreover, in experi- mental settings, meaning variables can affect phonological error rates (see, e.g., Motley, 1980). Accordingly, when planning occurs at the positional level, word meanings are not irrelevant, as Garrett had supposed. The feedforward links in Dell’s network provide the basis for this inﬂuence. A second ﬁnding is that semantic substitutions (e.g., the summer/winter error above) tend to be phonologically more related than are randomly re-paired intended and error words. This implies activation that spreads along feedback links. In the last decade, researchers developed new ways to study phonological planning. One reason for these develop- ments is concern about the representativeness of error cor- pora. Error collectors can only transcribe errors that they hear. They may fail to hear errors or mistranscribe them for a variety of reasons. Some errors occur that are inaudible. This has been shown by Mowrey and MacKay (1990), who mea- sured activity in muscles of the vocal tract as speakers pro- duced tongue twisters (e.g., Bob ﬂew by Bligh Bay ). In some utterances, Mowrey and MacKay observed tongue muscle activity for /l/ during production of Bay even though the word sounded error free to listeners. The ﬁndings show that errors occur that transcribers will miss. Mowrey and MacKay also suggest that their data show that subphonemic errors occur, in particular, in activation of single muscles. This conclusion is not yet warranted by their data, because other, unmonitored Phonological Planning 245 muscles for production of an intruding phoneme might also have been active. However, it is also possible that errors may appear to the listener tidier than they are. We know, too, that listeners tend to “ﬂuently restore” (Marslen-Wilson & Welsh, 1978) speech errors. They may not hear errors that are, in principle, audible, because they are focusing on the content of the speaker’s utterance, not its form. These are not reasons to ignore the literature on speech errors; it has provided much very useful information. How- ever, it is a reason to look for converging measures, and that is the next topic. Experimental Evidence About Phonological Planning Some of the experimental evidence on phonological planning has been obtained from procedures that induce speech errors (e.g., Baars, Motley, & MacKay, 1975; Dell, 1986). Here, however, the focus is on ﬁndings from other procedures in which production response latencies constitute the main dependent measure. This research, pioneered by investigators at the Max Planck Institute for Psycholinguistics in the Netherlands, has led to a theory of lexical access in speech production (Levelt, Roelofs, & Meyer, 1999) that will serve to organize presen- tation of relevant research ﬁndings. The theory has been partially implemented as a computational model, WEAVER (e.g., Roelofs & Meyer, 1998). However, I will focus on the theory itself. It begins by representing the concepts that a speaker might choose to talk about, and it describes processes that achieve selection of relevant linguistic units and ulti- mately speech motor programs. Discussion here is restricted to events beginning with word form selection. In the theory, selection of a word form provides access to the word’s component phonological segments, which are ab- stract, featurally underspeciﬁed segments (see section titled “Features and Contrast: Onward to Phonology”). If the word does not have the default stress pattern (with stress on the syl- lable with the ﬁrst full vowel for both Dutch and English speakers), planners also access a metrical frame, which spec- iﬁes the word’s number of syllables and its stress pattern. For words with the default pattern, the metrical frame is con- structed online. In this theory, as in Dell’s, the segments are types, not tokens, so that the /t/ in touch is the very /t/ in tiny . This allows for the possibility of form priming. That is, preparing to say a word that shares its initial consonant with a prime word can facilitate latency to produce the target word. In contrast to Dell’s (1986) model, however, conso- nants are not exclusively designated either onset consonants or coda consonants. That is, the /t/ in touch is also the very /t/ in date . Accessed phonological segments are spelled out into phonological word frames. This reﬂects an association of the phonological segments of a word with the metrical frame, if there is an explicit one in the lexical entry, or with a frame computed on line. This process, called prosodiﬁcation, is pro- posed to be sequential; that is, segments are slotted into the frame in an early-to-late (left-to-right) order. Meyer and Shriefers (1991) found evidence of form prim- ing and a left-to-right process in a picture-naming task. In one experiment, at some stimulus onset asynchrony (SOA) be- fore or after presentation of a picture, participants heard a monosyllabic word that overlapped with the monosyllabic picture name at the beginning (the initial CV), at the end (the VC), or not at all. On end-related trials, the SOA between word and picture was adjusted so that the VC’s temporal relation to the picture was the same as that of the CV of begin-related words. On some trials no priming word was presented. The priming stimulus generally slowed re- sponses to the picture, but, at some SOAs, it did so less if it was related to the target. For words that overlapped with the picture name in the initial CV, the response time advantage (over response times to pictures presented with unrelated primes) was signiﬁcant when words were presented 150 ms before the pictures (but not 300 ms before) and continued through the longest lagging SOA tested, when words were presented 150 ms after the picture. For words overlapping with the picture name in the ﬁnal VC, priming began to have an effect at 0 ms SOA and continued through the 150-ms lag condition. The investigators infer that priming occurs during phonological encoding, that is, as speakers access the phono- logical segments of the picture name. Perhaps at a 300-ms lead the activations of phonological segments shared be- tween prime and picture name have decayed by the time the picture is processed. However, by a 150-ms lead, the prime facilitates naming the picture, because phonemes activated by its presentation are still active and appropriate to the pic- ture. The ﬁnding that end-related primes begin facilitating later than begin-related items, even though the overlapping phonemes in the prime bore the same temporal relation to the picture’s presentation as did the overlapping CVs or initial syllables, suggests an early-to-late process. Using another procedure, Meyer (1990, 1991) also found form priming and evidence of a left-to-right process. Meyer (1990) had participants learn word pairs. Then, prompted by the ﬁrst word of the pair, they produced the second. In homo- geneous sets of word pairs, disyllabic response words of each pair shared either their ﬁrst or their second syllable. In het- erogeneous sets, response words were unrelated. The ques- tion was whether, across productions of response words in homogeneous sets, latencies would be faster than to response 246 Speech Production and Perception words in heterogeneous sets, because segments in the over- lapping syllables would remain prepared for production. Meyer found shorter response latencies only in the homoge- neous sets in which the ﬁrst syllable was shared across re- sponse words. In a follow-up study, Meyer (1991) showed savings when word onsets were shared but not when rimes were shared. On the one hand, these studies provide evidence converging with that of Meyer and Shriefers (1991) for form priming and left-to-right preparation. However, the evidence appears to conﬂict in that Meyer (1990, 1991) found no end- overlap priming, whereas Meyer and Shriefers did. Levelt et al. (1999) suggested, as a resolution, that the latter results occur as the segments of a lexical item are activated, whereas the results of Meyer reﬂect prosodiﬁcation (that is, merging of those segments with the metrical frame). The theory of Levelt et al. (1999) makes a variety of predictions about the prosodiﬁcation process. First, the phonological segments and the metrical frame are retrieved as separate entities. Second, the metrical frame speciﬁes only the number of syllables in the word and the word’s stress pat- tern; it does not specify the CV pattern of the syllables. Third, for words with the default stress pattern, no metrical frame is retrieved; rather, it is computed online. Roelofs and Meyer (1998) tested these predictions using the implicit priming procedure. In the ﬁrst experiment, in homogeneous sets, response words were disyllables with second-syllable stress that shared their ﬁrst syllables; het- erogeneous sets had unrelated ﬁrst syllables. Alternatively, homogeneous (same ﬁrst syllables) and heterogeneous (unrelated ﬁrst syllables) response words had a variable num- ber of syllables (2–4) with second-syllable stress. None of the words in this and the following experiments had the default stress pattern, so that, according to the theory, a metrical frame had to be retrieved. Priming (that is, an advantage in response latency for the homogeneous as compared to the heterogeneous sets) occurred only if the number of syllables was the same across response words. This is consistent with the prediction that the metrical frame speciﬁes the number of syllables. A second experiment conﬁrmed that, with the num- ber of syllables per response word held constant, the stress pattern had to be shared for priming to occur. A third experi- ment tested the prediction that shared CV structure did not in- crease priming. In this experiment, response words were monosyllables that, in homogeneous sets, shared their initial consonant clusters (e.g., br). In one kind of homogeneous set, the words shared their CV structure (e.g., all were CCVCs); in another kind of homogeneous set, they had different CV structures. The two homogeneous sets produced equivalent priming relative to latencies to produce heterogeneous re- sponses. This is consistent with the claim of the theory that the metrical frame only speciﬁes the number of syllables, but not the CV structure of each syllable. Subsequent experi- ments showed that shared number of syllables with no seg- mental overlap and shared stress pattern without segmental overlap give rise to no priming. Accordingly, it is the integra- tion of the word’s phonological segments with the metrical frame that underlies the priming effect. Finally, in a study by Meyer, Roelofs, and Schiller, de- scribed by Levelt et al. (1999), Meyer et al. examined words with the default stress pattern for Dutch. In this case, no met- rical frame should be retrieved and so none can be shared across response words. Meyer et al. found that for words that shared their initial CVs and that had the default stress pattern for Dutch, shared metrical structure did not increase priming. The next process in the theory is phonetic encoding in which talkers establish a gestural score (see section titled “Feature Systems”) for each phonological word. This phase of talking is not well worked out by Levelt et al. (1999), and it is the topic of the next major section (“Speech Produc- tion”). Accordingly, I will not consider it further here. Disagreements Between the Theories of Dell, 1986, and Levelt et al., 1999 Two salient differences between the theory of Dell (1986), developed largely from speech error data, and that of Levelt et al. (1999), developed largely from speeded naming data, concern feedback and syllabiﬁcation. Dell’s model includes feedback. The theory of Levelt et al. and Roelof and Meyer’s (1998) model WEAVER do not. In Dell’s model, phones are slotted into a syllable frame, whereas in the theory of Levelt et al., they are slotted into a metrical frame that speciﬁes the number of syllables, but not their internal structure. As for the disagreement about feedback, the crucial error data supporting feedback consist of such errors as saying winter for summer, in which the target and the error word share both form and meaning. In Dell’s (1986) model, form can affect activation of lexical items via feedback links in the network. Levelt et al. (1999) suggest that these errors are monitoring failures. Speakers monitor their speech, and they often correct their errors. Levelt et al. suggest that the more phonologically similar the target and error words are, the more likely the monitor is to fail to detect the error. The second disagreement is about when during planning phonological segments are syllabiﬁed. In Dell’s (1986) model, phones are identiﬁed with syllable positions in the lexicon, and they are slotted into abstract syllable frames in the course of planning for production. In the theory of Levelt et al. (1999), syllabiﬁcation is a late process, as it has to be to allow resyllabiﬁcation to occur. There is evidence favoring Speech Production 247 both sides. As described earlier, Roelofs and Meyer (1998) reported that implicit priming occurs across response words that share stress pattern, number of syllables, and phones at the beginning of the word, but shared syllable structure does not increase priming further. Sevald, Dell, and Cole (1995) report apparently discrepant ﬁndings. Their task was to have speakers produce a pair of nonwords repeatedly as quickly as possible in a 4-s interval. They measured mean syllable pro- duction time and found a 30-ms savings if the nonwords shared the initial syllable. For example, the mean syllable production time for KIL KIL.PER (where the “.” signals the syllable boundary) was shorter than for KILP KIL.PER or KIL KILP.NER. Remarkably, they also found shorter produc- tion times when only syllable structure was shared (e.g., KEM TIL.PER). These ﬁndings show that, at whatever stage of planning this effect occurs, syllable structure matters, and an abstract syllable frame is involved. This disagreement, like the ﬁrst, remains unresolved (see also Santiago & MacKay, 1999). SPEECH PRODUCTION Communication by language use requires that speakers act in ways that count as linguistic. What are the public events that count as linguistic? There are two general points of view. The more common one is that speakers control their actions, their movements, or their muscle activity. This viewpoint is in common with most accounts of control over voluntary activity (see chapter by Heuer in this volume). A less common view, however, is that speakers control the acoustic signals that they produce. A special characteristic of public linguistic events is that they are communicative. Speech activity causes an acoustic signal that listeners use to determine a talker’s message. As the next major section (“Speech Perception”) will re- veal, there are also two general views about immediate ob- jects of speech perception. Here the more common view is that they are acoustic. That is, after all, what stimulates the perceiver’s auditory perceptual system. A less common view, however, is that they are articulatory or gestural. An irony is that the most common type of theory of pro- duction and the most common type of theory of perception do not ﬁt together. They have the joint members of commu- nicative events producing actions, but perceiving acoustic structure. This is unlikely to be the case. Communication requires prototypical achievement of parity, and parity is more likely to be achieved if listeners perceive what talkers produce. In this section, I will present instances of both types of production theory, and in the next section, both types of perception theory. The reader should keep in mind that considerations of parity suggest that the theories should be linked. If talkers aim to produce particular acoustic pattern- ings, then acoustic patterns should be immediate perceptual objects. However, if talkers aim to produce particular ges- tures, then that is what listeners should perceive. How Acoustic Speech Signals Are Produced Figure 9.1 shows the vocal tract, the larynx, and the respira- tory system. Articulators of the vocal tract include the jaw, the tongue (with relatively independent control of the tip or blade and the tongue body), the lips, and the velum. Also in- volved in speech is the larynx, which houses the vocal folds, and the lungs. In prototypical production of speech, acoustic energy is generated at a source, in the larynx or oral cavity. In production of vowels and voiced consonants, the vocal folds are adducted. Air ﬂow from the lungs builds up pressure be- neath the folds, which are blown apart brieﬂy and then close again. This cycling occurs at a rapid rate during voiced speech. The pulses of air that escape whenever the folds are blown apart are ﬁltered by the oral cavity. Vowels are pro- duced by particular conﬁgurations of the oral cavity achieved by positioning the tongue body toward the front (e.g., for /i/) or back (e.g., for /a/) of the oral cavity, close to the palate (e.g., /i/, /u/) or farther away (e.g., /a/), with lips rounded (/u/) or not. In production of stop consonants, there is a complete Figure 9.1 The speech sound producing system (from Borden, Harris, & Raphael, 1994). Reprinted with permission. [ Image not available in this electronic edition.] 248 Speech Production and Perception stoppage of airﬂow through the oral cavity for some time due to a constriction that, in English, occurs at the lips (/b/, /p/, /m/), with the tongue tip against the alveolar ridge of the palate (/d/, /t/, /n/) or with the tongue body against the velum (/g/, /k/, / ŋ /). For the nasal consonants, /m/, /n/, and / ŋ /, the velum is lowered, allowing airﬂow through the nose. For fricatives, the constriction is not complete, so that airﬂow is not stopped, but the constriction is sufﬁciently narrow to cause turbulent, noisy airﬂow. This occurs in English, for ex- ample, in /s/, /f/, and / θ/ (the initial consonant of, e.g., theta ). Consonants of English can be voiced (vocal folds adducted) or unvoiced (vocal folds abducted). The acoustic patterning caused by speech production bears a complex relation to the movements that generate it. In many instances the relation is nonlinear, so that, for example, a small movement may generate a marked change in the sound pattern (as, for example, when the narrow constriction for /s/ becomes the complete constriction for /t/). In other instances, a fairly large change in vocal tract conﬁguration can change the acoustic signal rather little. Stevens (e.g., 1989) calls these “quantal regions,” and he points out that language communities exploit them, for example, to reduce the requirement for extreme articulatory precision. Some Properties of Speech That a Production Theory Needs to Explain Like all intentional biological actions, speaking is coordi- nated action. Absent coordination, as Weiss (1941) noted, ac- tivity would consist of “unorganized convulsions.” What is coordination? It is (cf. Turvey, 1990) a reduction in the de- grees of freedom of an organism with a consequent reduction in its dimensionality. This reduces the outputs the system can produce, restricting them to the subset of outcomes consistent with the organism’s intentions. Although it is not (wholly) biological, I like to illustrate this idea using the automobile. Cars have axles between the front wheels so that, when the driver turns the steering wheel, both front wheels are con- strained to turn together. The axle reduces the degrees of free- dom of movement of the car-human system, preventing movements in which the car’s front wheels move indepen- dently, and it lowers the dimensionality of the system by link- ing the wheels. However, the reduction in power is just what the driver wants; that is, the driver only wants movements in which the wheels turn cooperatively. The lowering of the dimensionality of the system creates macroscopic order consistent with an actor’s intentions; that is, it creates a special purpose device. In the domain of action, these special purpose devices are sometimes called “coordi- native structures” (Easton, 1972) or synergies. In the vocal tract, they are linkages among articulators that achieve coor- dinated action. An example is a transient linkage between the jaw and two lips that achieves lip closure for /b/, /p/, and /m/ in English. An important characteristic of synergies is that they give rise to motor equivalence: that is, the ability to achieve the same goal (e.g., lip closure in the example above), in a vari- ety of ways. Speakers with a bite block held between their teeth to immobilize the jaw (at a degree of opening too wide for normal production of /i/, for example, or too closed for normal production of /a/) produce vowels that are near nor- mal from the ﬁrst pitch pulse of the ﬁrst vowel they produce (e.g., Lindblom, Lubker, & Gay, 1979). An even more strik- ing ﬁnding is that speakers immediately compensate for on- line articulatory perturbations (e.g., Abbs & Gracco, 1984; Kelso, Tuller, Vatikiotis-Bateson, & Fowler, 1984; Shaiman, 1989). For example, in research by Kelso et al. (1984), on an unpredictable 20% of trials, a jaw puller pulled down the jaw of a speaker producing It’s a bab again as the speaker was closing his lips for the ﬁnal /b/ of bab. Within 20–30 ms of the perturbation, extra activity of an upper lip muscle (com- pared to its activity on unperturbed trials) occurred, and clo- sure for /b/ was achieved. When the utterance was It’s a baz again, jaw pulling caused extra activity in a muscle of the tongue, and the appropriate constriction was achieved. These responses to perturbation are fast and functional (cf. Löfquist, 1997). These immediate and effective compensations contrast with others. When Savariaux, Perrier, and Orliaguet (1995) had talkers produce /u/ with a lip tube that prevented round- ing, tongue backing could compensate for some acoustic consequences of the lip tube. Of 11 participants in the study, however, 4 showed no compensation at all (in about 20 attempts); 6 showed a little, but not enough to produce a nor- mal acoustic signal for /u/; just 1 achieved full compensation. Similarly, in research by Hamlet and Stone (e.g., 1978; Hamlet, 1988), after one week’s experience, speakers failed to compensate fully for an artiﬁcial palate that changed the morphology of their vocal tract. What is the difference be- tween the two sets of studies that explains the differential success of compensation? Fowler and Saltzman (1993) sug- gest that the bite block and on-line perturbation studies may use perturbations that approximately occur in nature, whereas the lip tube and the artiﬁcial palate do not. That is, competing demands may be placed on the jaw because ges- tures overlap in time. For example, the lip-closing gesture for /b/ may overlap with the gestures for an open vowel. The vowel may pull down the jaw so that it occupies a more open position for /b/ than it does when /b/ gestures overlap with those for the high vowel /i/. Responses to the bite block and Speech Production 249 Figure 9.2 Data from Fowler (1994). Plots for /b/, /d/ and /z/ of F2 at vowel midpoint by F2 at syllable onset. to on-line perturbations of the jaw may be immediate and ef- fective because talkers develop ﬂexible synergies for produc- ing vowels with a range of possible openings of the jaw and consonants with a range of jaw closings. However, nothing prevents lip protrusion in nature, and nothing changes the morphology of the vocal tract. Accordingly, synergies to compensate for those perturbations do not develop. Indeed, gestural overlap (that is, coarticulation) is a perva- sive characteristic of speech and therefore is a characteristic that speakers need to learn both to achieve and to compensate for. Coarticulation is a property of action that can only occur when discrete actions are sequenced. Coarticulation has been described in a variety of ways: as spreading of features from one segment to another (as when rounding of the lips from /u/ occurs from the beginning of a word such as strew )orasas- similation. However, most transparently, when articulatory activity is tracked, coarticulation is a temporal overlap of ar- ticulatory activity for neighboring consonants and vowels. Overlap occurs both in an anticipatory (right-to-left) and a car- ryover (perseveratory, left-to-right) direction. This characteri- zation in terms of gestural overlap is sometimes called coproduction . Its span can be segmentally extensive as when vowel-to-vowel coarticulation occurs over intervening con- sonants (e.g., Fowler & Brancazio, 2000; Öhman, 1966; Recasens, 1984). However, it is not temporally very extensive, spanning perhaps no more than about 250 ms (cf. Fowler & Saltzman, 1993). According to the frame theory of coarticula- tion (e.g., Bell-Berti & Harris, 1981), in anticipatory coarticu- lation of such gestures as lip rounding for a rounded vowel (e.g., Boyce, Krakow, Bell-Berti, & Gelfer, 1990) or nasaliza- tion for a nasalized consonant (e.g., Bell-Berti & Krakow, 1991; Boyce et al., 1990) the anticipating gesture is not linked to the gestures for other segments with which it overlaps in time; rather, it remains tied to other gestures for the segment, which it anticipates by an invariant interval. An interesting constraint on coarticulation is coarticulation resistance (Bladon & Al-Bamerni, 1976). This reﬂects the dif- ferential extent to which consonants or vowels resist coarticu- latory encroachment by other segments. Recasens’s research (e.g., 1984) suggests that resistance to vowels among conso- nants varies with the extent to which the consonants make use of the tongue body, also required for producing vowels. Accordingly, a consonant such as /b/ that is produced with the lips is less resistant than one such as /d/, which uses the tongue (cf. Fowler & Brancazio, 2000). An index of coarticulation re- sistance is the slope of the straight-line relation between F2 at vowel midpoint of a CV and F2 at syllable onset for CVs in which the vowel varies but the consonant is ﬁxed (see many papers by Sussman, e.g., Sussman, Fruchter, Hilbert, & Sorish, 1999a). Figure 9.2 shows data from Fowler (1994). The low resistant consonant /b/ has a high slope, indicating considerable coarticulatory effect of the vowel on /b/’s acoustic manifestations at release; the slope for /d/ is much shallower; that for /z/ is slightly shallower than that for /d/. Fowler (1999) argues that the straight-line relation occurs be- cause a given consonant resists coarticulation by different vowels to an approximately invariant extent; Sussman et al. (1999a; Sussman, Fruchter, Hilbert, & Sirosh, 1999b) argue that speakers produce the straight-line relation intentionally, because it fosters consonant identiﬁcation and perhaps learn- ing of consonantal place of articulation. A ﬁnal property of speech that will require an account by theories of speech production is the occurrence of phase tran- sitions as rate is increased. This was ﬁrst remarked on by Stetson (1951) and has been pursued by Tuller and Kelso (1990, 1991). If speakers begin producing /ip/, as rate in- creases, they shift to /pi/. Beginning with /pi/ does not lead to a shift to /ip/. Likewise, Gleason, Tuller, and Kelso (1996) found shifts from opt to top , but not vice versa, as rate in- creased. Phase transitions are seen in other action systems; for example, they underlie changes in gait from walk to trot to canter to gallop. They are considered hallmarks of nonlin- ear dynamical systems (e.g., Kelso, 1995). The asymmetry in direction of the transition suggests a difference in stability such that CVs are more stable than VCs (and CVCs than VCCs). Acoustic Targets of Speech Production I have described characteristics of speech production, but not its goals. Its goals are in contention. Theories that speakers control acoustic signals are less common than those that they control something motoric; however, there is a recent example in the work of Guenther and colleagues (Guenther, Hampson, & Johnson, 1998). Guenther et al. offer four reasons why 250 Speech Production and Perception targets are likely to be acoustic (in fact, are likely to be the acoustic signal as they are transduced by the auditory system). Opposing a theory that speakers control gestural constrictions (see section titled “Gestural Targets of Speech Production”) is that, in the authors’ view, there is not very good sensory infor- mation about many vocal tract constrictions (e.g., con- strictions for vowels where there is no tactile contact between the tongue and some surface). Moreover, although it is true that speakers achieve nearly invariant constrictions (e.g., they always close their lips to say /b/), this can be achieved by a model in which targets are auditory. Third, control over in- variant constriction targets would limit the system’s ability to compensate when perturbations require new targets. (This is quite right, but, in the literature, this is exactly where compen- sations to perturbation are not immediate or generally effec- tive. See the studies by Hamlet & Stone, 1978; Hamlet, 1988; Savariaux et al., 1995; Perkell, Matthies, Svirsky, & Jordan, 1993.) Finally, whereas many studies have shown directly (Delattre & Freeman, 1968) or by suggestive acoustic evi- dence (Hagiwara, 1995) that American English /r/ is produced differently by different speakers and even differently by the same speaker in different phonetic contexts, all of the gestural manifestations produce a similar acoustic product. In the DIVA model (Guenther et al., 1998), planning for production begins with choice of a phoneme string to pro- duce. The phonemes are mapped one by one onto target re- gions in auditory-perceptual (speech-sound) space. The maps are to regions rather than to points in order to reﬂect the fact that the articulatory movements and acoustic signals are dif- ferent for a given phoneme due to coarticulation and other perturbations. Information about the model’s current location in auditory-perceptual space in relation to the target region generates a planning vector, still in auditory-perceptual space. This is mapped to a corresponding articulatory vector, which is used to update articulatory positions achieved over time. The model uses mappings that are learned during a bab- bling phase. Infant humans babble on the way to learning to speak. That is, typically between the ages of 6 and 8 months, they produce meaningless sequences that sound as if they are composed of successive CVs. Guenther et al. propose that, during this phase of speech development, infants map in- formation about their articulations onto corresponding con- ﬁgurations in auditory-perceptual space. The articulatory information is from orosensory feedback from their articula- tory movements and from copies of the motor commands that the infant used to generate the movements. The auditory per- ceptual information is from hearing what they have pro- duced. This mapping is called a forward model; inverted, it generates movement from auditory-perceptual targets. To this end, the babbling model learns two additional mappings, from speech-sound space, in which (see above) auditory- perceptual target regions corresponding to phonemes are rep- resented as vectors through the space that will take the model from its current location to the target region, and from those trajectories to trajectories in articulatory space. An important idea in the model is that targets are regions rather than points in acoustic-auditory space. This allows the model to exhibit coarticulation and, with target regions of appropriate ranges of sizes, coarticulation resistance. The model also shows compensation for perturbations, because if one target location in auditory-perceptual space is blocked, the model can reach another location within the target region. Successful phoneme production does not require achievement of an invariant conﬁguration in either auditory-perceptual or articulatory space. This property of the model underlies its failure to distinguish responses to perturbation that are imme- diately effective from those that require some relearning. The model shows immediate compensations for both kinds of per- turbation. It is silent on phase transitions. Gestural Targets of Speech Production Theories in which speakers control articulation rather than acoustic targets can address all or most of the reasons that underlay Guenther et al.’s (1998) conclusion that speakers control perceived acoustic consequences of production. For example, Guenther et al. suggest that if talkers controlled constrictions, it would unduly limit their ability to compen- sate for perturbations where compensation requires changing a constriction location, rather than achieving the same con- striction in a different way. A response to this suggestion is that talkers do have more difﬁculty when they have to learn a new constriction. The response of gesture theorists to /r/ as a source of evidence that acoustics are controlled will be pro- vided after a theory has been described. Figure 9.3 depicts a model in which controlled primitives are the gestures of Browman and Goldstein’s

bundet

(e.g., 1986) ar- ticulatory phonology (see section titled “Feature Systems”). Figure 9.3 Haskins’ Computational Gestural Model. Speech Production 251 Figure 9.4 Tract Variables for gestures and the articulators comprising their coordinative structures. Gestures create and release constrictions in the vocal tract. Figure 9.4 displays the tract variables that are controlled when gestures are produced and the gestures’ associated articulators. In general, tract variables specify constriction locations (CLs) and constriction degrees (CD) in the vocal tract. For example, to produce a bilabial stop, the constriction location is a speciﬁed degree of lip protrusion and the con- striction degree is maximal; the lips are closed. The articula- tors that achieve these values of the tract variables are the lips and the jaw. The linguistic gestural model of Figure 9.3 generates ges- tural scores such as that in Figure 9.5. The scores specify the gestures that compose a word and their relative phasing. Ges- tural scores serve as input to the task dynamic model (e.g., Saltzman, 1991; but see Saltzman, 1995; Saltzman & Byrd, 1999). Gestures are implemented as two-tiered dynamical (mass-spring) systems. At an initial level the systems refer to tract variables, and the dynamics are of point attractors. These dynamics undergo a one-to-many transformation to articulator space. Because the transformation is one-many, tract variable values can be achieved ﬂexibly. Because the gestural scores specify overlap between gestures, the model coarticulates; moreover (e.g., Saltzman, 1991), it mimics some of the ﬁndings in the literature on coarticulation resis- tance. In particular, the high resistant consonant /d/ achieves its target constriction location regardless of the vowels with which it overlaps; the constriction location of the lower resis- tant /g/ moves with the location of the vowel gesture. The model also compensates for the kinds of perturbations to which human talkers compensate immediately (bite blocks and on-line jaw or lip perturbations in which invariant constrictions are achieved in novel ways). It does not show the kinds of compensations studied by Hamlet and Stone (1978), Savariaux et al. (1995), or Perkell et al. (1993), in which new constrictions are required. (The model, unlike that of Guenther et al., 1998, does not learn to speak; accordingly, it cannot show the learning that, for example, Hamlet and Stone ﬁnd in their human talkers.) The model also fails to exhibit phase transitions although it is in the class of models (nonlinear dynamical systems) that can. Evidence for Both Models: The Case of /r/ One of the strongest pieces of evidence convincing Guenther et al. (1998) that targets of production are acoustic is the highly variable way in which /r/ is produced. This is because of claims that acoustic variability in /r/ production is less than articulatory variability. Ironically, /r/ also ranks as strong evidence favoring gestural theory among gesture theorists. Indeed, in this domain, /r/ contributes to a rather beautiful recent set of investigations of composite phonetic segments. The phoneme /r/ is in the class of multigestural (or com- posite ) segments, a class that also includes /l/, /w/, and the nasal consonants. Krakow (1989, 1993, see also 1999) was the ﬁrst to report that two salient gestures of /m/ (velum low- ering and the oral constriction gesture) are phased differently in onset and coda positions in a syllable. In onset position, the velum reaches its maximal opening at about the same time as the oral constriction is achieved. In coda position, the velum reaches maximum opening as the oral articulators (the lips for /m/) begin their closing gesture. Similar ﬁndings have been reported for /l/. Browman and Goldstein (1995b), fol- lowing earlier observations by Sproat and Fujimura (1993; see also Gick, 1999), report that in onset position, the ter- minations of tongue tip and tongue dorsum raising were simultaneous, whereas the tongue dorsum gesture led in coda position. Gick (1999) found a similar relation between lip and tongue body gestures for /w/. As Browman and Goldstein (1997) remark, in multi- gestural consonants, in coda position, gestures with wider Figure 9.5 Gestural score for the word pan. 252 Speech Production and Perception constriction degrees (that is, more open gestures) are phased earlier with respect to gestures having more narrow con- striction degrees; in onset position, the gestures are more synchronous. Sproat and Fujimura (1993) suggest that the component gestures of composite segments can be identiﬁed, indeed, as vocalic (V; more open) or consonantal ©. This is interesting in light of another property of syllables. They tend, universally, to obey a sonority gradation such that more vowel-like (sonorous) consonants tend to be closer to the syl- lable nucleus than less sonorous consonants. For example, if /t/ and /r/ are going to occur before the vowel in a syllable of English, they are ordered /tr/. After the vowel, the order is /rt/. The more sonorous of /t/ and /r/ is /r/. Gestures with wider constriction degrees are more sonorous than those with narrow constriction degrees, and, in the coda position, they are phased so that they are closer to the vocalic gesture than are gestures with narrow constriction degrees. A reason for the sonority gradient has been suggested; it permits smooth opening and closing actions of the jaw in each syllable (Keating, 1983). Goldstein (personal communication, October 19, 2000) suggests that the tendency for /r/ to become something like / ɔi/ in some dialects of American English (Brooklyn; New Orleans), so that bird (whose /r/-colored vowel is / /) is pro- nounced something like boid, may also be due to the phasing characteristics of coda C gestures. The phoneme /r/ may be produced with three constrictions: a pharyngeal constriction made by the tongue body, a palatal constriction made by the tongue blade, and a constriction at the lips. If the gestures of the tongue body and lips (with the widest constriction de- grees) are phased earlier than the blade gesture in coda posi- tion, the tongue and lip gestures approximate those of / ɔ /, and the blade gesture against the palate is approximately that for /i/. But what of the evidence of individual differences in /r/ production that convinced Guenther et al. (1998) that speech production targets are auditory-perceptual? One answer is that the production differences can look smaller than they have been portrayed in the literature if the gestural focus on vocal tract conﬁgurations is adopted. The striking differences that researchers have reported are in tongue shape. However, Delattre and Freeman (1968), characteristically cited to underscore the production variability of /r/, make this re- mark: “Different as their tongue shapes are, the six types of American /r/’s have one feature in common—they have two constrictions, one at the palate, another at the pharynx” (p. 41). That is, in terms of constriction location, a gestural parameter of articulatory phonology, there is one type of American English /r/, not six. SPEECH PERCEPTION The chapter began with the language knower. Then it explored how such an individual might formulate a linguistic message at the phonological level of description and implement the message as vocal tract activity that causes an acoustic speech signal. For an act of communication to be completed, a per- ceiver (another language knower) must intercept the acoustic signal and use it to recover the speaker’s message. In this sec- tion, the focus is on how perception takes place. Phonetic Perception Preliminary Issues I have suggested that a constraint on development of theories of phonological competence, planning, production, and per- ception should be an understanding that languages are likely to be parity fostering. Two parity-fostering characteristics are phonological forms that can be made public, and preservation of those forms throughout a communicative exchange. If the- orists were to hew to expectations that languages have these properties, then we would expect to ﬁnd perception theories in which perceptual objects are planned and produced phono- logical forms. We do not quite ﬁnd that, because, as indicated in the introduction, research on perception, production, plan- ning, and phonological description all have progressed fairly independently. However, there is one respect in which perception theories intersect fairly neatly with production theories. They partition into two broad classes that divide according to the theorists’ claims about immediate objects of speech perception. The ma- jority view is that objects are acoustic. This is not an implausi- ble view, given that acoustic signals are stimuli for speech perception. The minority view is that objects are gestural. Considerations of parity suggest a pairing of acoustic theories of speech perception with production theories like that of Guenther et al. (1998) in which speakers aim to produce acoustic signals with required properties. Gestural theories of speech perception are consistent with production theories, such as that of Saltzman and colleagues, in which speakers aim to produce gestures with particular properties. Another issue that divides theorists is whether speech perception is special—that is, whether mental processes that underlie speech perception are unique to speech, perhaps tak- ing place in a specialization of the brain for speech (a phonetic module, as Liberman & Mattingly, 1985, propose). There is reason to propose that speech processing is special. In speaking, talkers produce discrete, but temporally overlapping, Speech Perception 253 gestures that correspond in some way to the phonological forms that listeners must recover. Coarticulation ensures that there is no temporally discrete, phone-sized segmental struc- ture in the acoustic signal corresponding to phonological forms and that the acoustic signal is everywhere context sensitive. If listeners do recover phonological forms when they listen, this poses a problem. Listeners have to use the continuous acoustic signal to recover the discrete context-invariant phonological forms of the talker’s message. Because, in general, acoustic signals are not caused by sequences of discrete, coarticulated mechanical events, speech does appear to pose a unique problem for listeners. However, there is also a point of view that the most conservative or parsimonious ﬁrst guess should be that pro- cessing is not special. Until the data demand postulating a specialization, we should attempt to explain speech percep- tion by invoking only processes that are required to explain other kinds of auditory perception. It happens that acoustic theorists generally take this latter view. Some gestural theo- rists take the former. Acoustic Theories of Speech Perception There are a great many different versions of acoustic theory (e.g., Diehl & Kluender, 1989; Kuhl, 1987; Massaro, 1987, 1998; Nearey, 1997; Stevens & Blumstein, 1981; Sussman et al., 1999a). Here, Diehl and Kluender’s auditory enhance- ment theory will illustrate the class. Acoustic theories are deﬁned by their commitment to im- mediate perceptual objects that are acoustic (or auditory—that is, perceived acoustic) in nature. One common idea is that auditory processing renders an acoustic object that is then classiﬁed as a token of a particular phonological category. Au- ditory enhancement theory makes some special claims in ad- dition (e.g., Diehl & Kluender, 1989; Kluender, 1994). One is that there is lots of covariation in production of speech and in the consequent acoustic signal. For example, as noted earlier, rounding in vowels tends to covary with tongue backness. The lips and the tongue are independent articulators; why do their gestures covary as they do? The answer from auditory en- hancement theory is that both the rounding and the tongue backing gestures lower a vowel’s second formant. Accord- ingly, having the gestures covary results in back vowels that are acoustically highly distinct from front (unrounded) vowels. In this and many other examples offered by Diehl and Kluender, pairs of gestures that, in principle, are independent conspire to make acoustic signals that maximally distinguish phonological form. This should beneﬁt the perceiver of speech. Another kind of covariation occurs as well. Characteris- tically, a given gesture has a constellation of distinct acoustic consequences. A well-known example is voicing in stop consonants. In intervocalic position (as in rapid vs. rabid ), voiced and voiceless consonants can differ acoustically in 16 different ways or more (Lisker, 1978). Diehl and Kluender (1989) suggest that some of those ways, in phonological seg- ments that are popular among languages of the world, are mu- tually enhancing. For example, voiced stops have shorter clo- sure intervals than do voiceless stops. In addition, they tend to have voicing in the closure, whereas voiceless stops do not. Parker, Diehl, and Kluender (1986) have shown that low- amplitude noise in an otherwise silent gap between two square waves makes the gap sound shorter than it sounds in the absence of the noise (as it indeed is). This implies that, in speech, voicing in the closure reinforces the perception of a shorter closure for voiced than voiceless consonants. This is an interesting case, because, in contrast to rounding and back- ing of vowels where two gestures reinforce a common acoustic property (a low F2), in this case, a single gesture— approximation of the vocal folds during the constriction ges- ture for the consonant—has two or more enhancing acoustic consequences. Diehl and Kluender (1989; see also Kluender, 1994) suggest that language communities “select” gestures that have multiple, enhancing acoustic consequences. A ﬁnal claim of the theory is that speech perception is not special and that one can see the signature of auditory pro- cessing in speech perception. A recent example of such a claim is provided by Lotto and Kluender (1998). In 1980, Mann had reported a ﬁnding of “compensation for coarticu- lation.” She synthesized an acoustic continuum of syllables that ranged from a clear /da/ to a clear /ga/ with many more ambiguous tokens in between. The syllables differed only in the direction of the third formant transition, which fell for /da/ and rose for /ga/. She asked listeners to identify members of the continuum when they were preceded by either of the two precursor syllables /al/ or /ar/. She predicted and found that listeners identiﬁed more ambiguous continuum members as /ga/ in the context of precursor /al/ than /ar/. The basis for Mann’s prediction was the likely effect of coarticulation by /l/ and /r/ on /d/ and /g/. The phoneme /l/ has a tongue tip con- striction that, coarticulated with /g/, a back consonant, is likely to pull /g/ forward; /r/ has a pharyngeal constriction that, coarticulated with /d/, is likely to pull /d/ back. When listeners reported more /g/s after /al/ and more /d/s after /ar/, they appeared to compensate for the fronting effects that /l/ should have on /g/ and the backing effects of /r/ on /d/. Lotto and Kluender (1998) offered a different account. They noticed that, in Mann’s stimulus set, /l/ had a very high 254 Speech Production and Perception ending frequency of F3, higher than the starting F3s of any members of the /da/-to-/ga/ continuum. The phoneme /r/ had a very low ending frequency of F3, lower than the starting frequency of any members of the continuum. They proposed that the ending F3 frequencies of /al/ and /ar/ were exerting a contrast effect on the starting F3s of the continuum members. Contrast effects are pervasive in perception research across the sensory modalities (e.g., Warren, 1985, who, however, does not refer to them as contrast effects). For example, when individuals judge the heaviness of weights (Guilford & Park, 1931), they judge an intermediate weight lighter if they have just hefted a heavier weight than if they have just hefted a lighter weight. Lotto and Kluender suggested that the very high ending F3 of /l/ made following F3 onsets of continuum members effectively lower (and so more /g/-like) than they were; the very low F3 of /r/ made onset F3s effectively higher and more /d/-like. They tested their hypothesis by substituting high and low sinewave tones for the precursor /al/ and /ar/ syllables of Mann (1980), and they found more /g/ judgments following the high than the low precursor tone. This cannot be compen- sation for coarticulation. It is, rather, according to Lotto and Kluender (1998), a signature of auditory processing showing up in speech perception judgments. Comparisons like this between perception of speech and of nonspeech analogues has provided one way of testing claims of auditory theories. Parker et al. (1986) tested whether two acoustic properties were mutually enhancing. The test by Lotto and Kluender tested for evidence of audi- tory processing in speech perception. Generally, investigators have used speech/nonspeech comparisons as a way to test whether speech processing is specialized and distinct from auditory processing. Many tests have found closely similar response patterns to speech and closely similar nonspeech signals (e.g., Sawusch & Gagnon, 1995). As we will see, however, not all have. Another test of auditory theories has been to compare re- sponses by humans and nonhumans to speech signals. Clearly, nonhumans do not have specializations for human speech perception. If they show some of the markers of human speech perception, then it is not necessary to suppose that a specialization is responsible for the markers in humans. There are some striking ﬁndings here. Kuhl and Miller (1978) trained chinchillas in a go–no go procedure to move to a different compartment of a cage when they heard one end- point of an acoustic voice onset time (VOT) continuum, but not when they heard a syllable at the other end. Following training, they were tested on all continuum members between the two endpoints as well as on the endpoints themselves. This allowed Kuhl and Miller to ﬁnd a boundary along the continuum at which the chinchillas’ behavior suggested that a voiced percept had replaced a voiceless one. Remarkably, the boundaries were close to those of humans, and there was an even more remarkable ﬁnding. In human speech, VOTs are longer for farther back places of articulation. That is, in English, /pa/ has a shorter VOT than /ta/, which has a shorter VOT than /ka/ (e.g., Zue, 1980). This may be because voic- ing cannot resume following a voiceless consonant until there is a sufﬁcient drop in pressure across the larynx. With back places of constriction, the cavity above the larynx is quite small and the pressure correspondingly higher than for front constrictions. English listeners place VOT boundaries at shorter values for /pa/ than for /ta/ and for /ta/ than for /ka/, as do chinchillas (Kuhl & Miller, 1978). It is not known what stimulus property or auditory system property might underlie this outcome. However, most investigators are conﬁdent that chinchillas are not sensitive to transglottal pressure differ- ences caused by back and front oral constrictions in human speech. Another striking ﬁnding, now with quail, is that of Lotto, Kluender, and Holt (1997) that quail show “compensation for coarticulation” given stimuli like those used by Mann (1980). Readers may be asking why anyone is a gesture theorist. However, gesture theories, like acoustic theories, derive from evidence and from theoretical considerations. Moreover, the- orists argue that many of the claims and ﬁndings of acoustic theories are equally compatible with gesture theories. For ex- ample, ﬁndings that language communities gravitate toward phones that have mutually distinctive acoustic signals is not evidence that perceptual objects are acoustic. In gesture the- ories, the acoustic signal is processed; it is used as informa- tion for gestures. If the acoustic signals for distinct gestures are distinct, that is good for the gesture perceiver. The most problematic ﬁndings for gesture theorists may be on the issue of whether speech perception is special. The neg- ative evidence is provided by some of the speech/ nonspeech and human/nonhuman comparisons. Here, there are two lines of attack that gesture theorists can mount. One is to point out that not all such comparisons have resulted in similar re- sponse patterns (for speech/nonspeech, see below; for human/ nonhuman, see, e.g., range effects in Waters & Wilson, 1976; see also Sinnott, 1974, cited in Waters & Wilson, 1976). If there are real differences, then the argument against a special- ization weakens. A second line of attack is to point out that the logic of the research in the two domains is weak. It is true that if humans and nonhumans apply similar processes to acoustic speech signals (and if experiments are designed appropri- ately), the two subject groups should show similar response patterns to the stimuli. However, the logic required by the research is the reverse of that. It maintains that if humans Speech Perception 255 and nonhumans show similar response patterns, then the processes applied to the stimuli are the same. This need not hold (cf. Trout, 2001). The same can be said of the logic of speech/nonspeech comparisons. Gesture Theories of Speech Perception There are two gesture theories in the class, both largely associated with theorists at Haskins Laboratories. Gesture theories are deﬁned by their commitment to the view that im- mediate objects of perception are gestural. One of these theo- ries, the motor theory (e.g., Liberman & Mattingly, 1985; Liberman & Whalen, 2000), also proposes that speech per- ception is special. The other, direct realist theory (Best, 1995; Fowler, 1986, 1996), is agnostic on that issue. The motor theory of speech perception was the ﬁrst ges- ture theory. It was developed by Liberman (1957, see also 1996) when he obtained experimental ﬁndings that, in his view, could not be accommodated by an acoustic theory. He and his colleagues were using two complementary pieces of technology, the sound spectrograph and the pattern playback, to identify the acoustic cues for perception. They used the spectrograph to make speech visible in the informative ways that it does, identiﬁed possible cues for a given consonant or vowel, and reproduced those cues by painting them on an ac- etate strip that, input to the pattern playback, was transformed to speech. If the acoustic structure preserved on acetate was indeed important for identifying the phone, it could be iden- tiﬁed as a cue. One very striking ﬁnding in that research was that, due to coarticulation, acoustic cues for consonants especially were highly context sensitive. Figure 9.6 provides a schematic spectrographic display of the syllables /di/ and /du/. Although natural speech provides a much richer signal than that in Figure 9.6, the depicted signals are sufﬁcient to be heard as /di/ and /du/. The striking ﬁnding was that the information critical to identiﬁcation of these synthetic syllables was the transition of the second formant. However, that transition is high in frequency and rising in /di/, but low and falling in /du/. In the context of the rest of each syllable, the consonants sound alike to listeners. Separated from context, they sound different, and they sound the way they look like they should sound: two “chirps,” one high in pitch and one lower. Liberman (e.g., 1957) recognized that, despite the context sensitivity of the acoustic signals for /di/ and /du/, naturally produced syllables do have one thing in common. They are produced in the same way. In both syllables, the tongue tip makes a constriction behind the teeth. Listeners’ percepts ap- peared to track the speaker’s articulations. A second striking ﬁnding was complementary. Stop con- sonants can be identiﬁed based on their formant transitions, as in the previous example, or based on a burst of energy that, in natural speech, precedes the transitions and occurs as the stop constriction is released. Liberman, Delattre, and Cooper (1952) found that a noise burst centered at 1440 Hz and placed in front of the vowels /i/ or /u/ was identiﬁed predom- inantly as /p/. However in front of /a/, it was identiﬁed as /k/. In this case, an invariant bit of acoustic structure led to dif- ferent percepts. To produce that bit of acoustic structure be- fore /i/ or /u/, a speaker has to make the constriction at the lips; to produce it before /a/, he or she has to make the con- striction at the soft palate. These ﬁndings led Liberman to ask: “when articulation and the sound wave go their separate ways, which way does the perception go?” (Liberman, 1957, p. 121). His answer was: “The answer so far is clear. The per- ception always goes with articulation.” Although the motor theory was developed to explain unexpected research ﬁndings, Liberman and colleagues pro- posed a rationale for listeners’ perception of gestures. Speak- ers have to coarticulate. Liberman and colleagues (e.g., Liberman, Cooper, Shankweiler, & Studdert-Kennedy, 1967) suggested that coarticulation is necessary to evade the limits of the temporal resolving power of the listener’s ear. These limits were proposed to underlie the failure of Haskins re- searchers more than 50 years ago to train people to use an acoustic alphabet intended for use in a reading machine for the blind (see Liberman, 1996). Listeners could not perceive sequences of discrete sounds at anything close to the rates at which they perceive speech. Coarticulation provides a con- tinuous signal evading the temporal resolving power limits of the ear, but it creates a new problem. The relation between phonological forms and acoustic speech structure is opaque. Liberman et al. (e.g., 1967) suggested that coarticulation re- quired a specialization of the brain to achieve it. What system would be better suited to deal with the acoustic complexities Figure 9.6 Schematic depiction of the synthetic syllables, /di/ and /du/. 256 Speech Production and Perception Figure 9.7 Schematic depiction of categorical identiﬁcation and dis- crimination. Identification of ba Identification of da Discrimination 100 80 60 40 20 0 23456da ba Continuum Percent identification or discrimination to which coarticulation gives rise than the system responsible for generating coarticulated speech? In later versions of the motor theory, this hypothesized specialization was identiﬁed as a phonetic module (cf. Fodor, 1983). There is an independent route to a conclusion that speech perception yields gestures. Fowler’s (e.g., 1986, 1996; see also Best, 1995; Rosenblum, 1987) direct realist theory de- rived that claim by developing a theory of speech perception in the context of a universal theory of perceptual function. That theory, developed by James Gibson (e.g., 1966, 1979), notes that perceptual systems constitute the only means that animals have to know their world. By hypothesis, they serve that function in just one general way. Stimulus structure at the sense organs is not perceived itself. Rather, it serves as infor- mation for its causal source in the environment, and the en- vironment is thereby perceived. In vision, for example, light that reﬂects from objects in the environment is structured by the properties of the objects and takes on structure that is distinctive to those properties. Because the structure is dis- tinctive to the properties, it can serve as information for them. Environmental events and objects, not the reﬂected light, are perceived. Fowler (1996) argued that, if even speech per- ception were wholly unspecial, listeners would perceive ges- tures, because gestures cause the structure in stimulation to the ear. And the auditory system (or the phonetic module), no less than the visual system, uses information in stimulation at the sense organ to reveal the world of objects and events to perceivers. What does the experimental evidence show? An early ﬁnding that Liberman (1957) took to be compatible with his ﬁndings on /di/-/du/ and /pi/-/ka/-/pu/ was categorical per- ception. This was a pair of ﬁndings obtained when listeners made identiﬁcation and discrimination judgments of stimuli along an acoustic continuum. Figure 9.7 displays schematic ﬁndings for a /ba/-to-/da/ continuum. Although the stimuli form a smooth continuum (in which the second formant tran- sition is gradually shifted from a trajectory for /ba/ to one for /da/), the identiﬁcation function is very sharp. Most stimuli along the continuum are heard either as a clear /ba/ or as a clear /da/. Only one or two syllables in the middle of the con- tinuum are ambiguous. The second critical outcome was obtained when listeners were asked to discriminate pairs of syllables along the continuum. The ﬁnding was that discrim- ination was near chance among pairs of syllables both mem- bers of which listeners identiﬁed as /ba/ or both /da/, but it was good between pair members that were equally acousti- cally similar as the /ba/ pairs and the /da/ pairs, but in which listeners heard one as /ba/ and the other as /da/. In contrast, say, to colors, where perceivers can easily discriminate colors that they uniformly label as blue, to a ﬁrst approximation, listeners could only discriminate what they labeled distinc- tively. The early interpretation of this ﬁnding was that it revealed perception of gestures, because the place of articula- tion difference between /ba/ and /da/, unlike the acoustic dif- ference, is categorical. This interpretation was challenged, for example, by Pisoni (e.g., Pisoni & Tash, 1974). In their study, Pisoni and Tash showed that same responses to pairs of syllables that were la- beled the same but that differed acoustically were slower than to identical pairs of syllables. Accordingly, listeners have at least ﬂeeting access to within-category differences. Despite this and other ﬁndings, the name categorical perception has endured, but now it is typically used only to refer to the data pattern of Figure 9.7, not to its original interpretation. A set of ﬁndings that has a natural interpretation in gesture theories is the McGurk effect (named for one of its discover- ers; McGurk and MacDonald, 1976). This effect is obtained when a videotape of a speaker mouthing a word or syllable (say, /da/) is dubbed with a different, appropriately selected, syllable (say, /ma/). With eyes open, listeners hear a syllable that integrates information from the two modalities. (In the example, they hear /na/, which takes its place of articulation from /da/ and its manner and voicing from /ma/.) The integra- tion is expected in a theory in which gestures are perceived, because both modalities provide information about gestures. There is, of course, an alternative interpretation from acoustic theories. The effect may occur because of our vast experience both seeing and hearing speakers talk. This experience may be encoded as memories in which compatible sights and sounds are associated (but see Fowler and Dekle, 1991). There are other ﬁndings that gesture theorists have taken to support their theory. For example, researchers have shown Speech Perception 257 that performance discriminating pairs of syllables can be better for stimuli that differ in one acoustic cue for a gesture than for stimuli that differ in that cue and one other for the same gesture (e.g., Fitch, Halwes, Erickson, & Liberman, 1980). This is unexpected on acoustic grounds. It occurs just when the two cues are selected so that the stimuli of a pair are identiﬁed as the same gesturally, whereas the pair differing in one cue are not always. Another ﬁnding is that people are re- markably rapid shadowers of speech under some conditions (e.g., Porter & Castellanos, 1980; Porter & Lubker, 1980). This has been interpreted as evidence that perceiving speech is perceiving gestures that constitute the instructions for the shadowing response. A third kind of ﬁnding has been research designed to show that listeners parse acoustic speech signals along gestural lines (e.g., Fowler & Smith, 1986; Pardo & Fowler, 1997). For example, when two ges- tures, say devoicing a preceding stop consonant and produc- tion of intonational accent, have convergent effects on the fundamental frequency (F0) pattern on a vowel, listeners do not hear the combined effects as the vowel’s intonation or pitch. They hear the contribution to F0 made by the devoic- ing gesture as information for devoicing (Pardo & Fowler, 1997). Finally, Fowler, Brown, and Mann (2000) have re- cently disconﬁrmed the contrast account of compensation for coarticulation offered by Lotto and Kluender (1998). They used the McGurk effect to show that, when the only infor- mation distinguishing /al/ from /ar/ was optical, and the only information distinguishing /da/ from /ga/ was acoustic, par- ticipants provided more /ga/ responses in the context of pre- cursor /al/ than /ar/. This cannot be a contrast effect. Fowler et al. concluded that the effect is literally compensation for coarticulation. Motor theorists have also attempted to test their idea that speech perception is achieved by a phonetic module. Like acoustic theorists, they have compared listeners’ responses to speech and to similar nonspeech signals, now with the expec- tation of ﬁnding differences. One of the most elegant demon- strations was provided by Mann and Liberman (1983). They took advantage of duplex perception, in which, in their ver- sion, components of a syllable were presented dichotically. The base, presented to one ear, included steady-state for- mants for /a/ preceded by F1 and F2 transitions consistent with either /d/ or /g/. An F3 transition, presented to the other ear, distinguished /da/ from /ga/. Perception is called duplex because the transitions are heard in two different ways at the same time. At the ear receiving the base, listeners hear a clear /da/ or a clear /ga/ depending on which transition was pre- sented to the other ear. At the ear receiving the transition, lis- teners hear a nonspeech chirp. On the one hand, this can be interpreted as evidence for a speech module, because how else, except with a separate perceptual system, can the same acoustic fragment be heard in two different ways at once? (However, see Fowler & Rosenblum, 1990, for a possible an- swer to the question.) On the other hand, it can provide the means of an elegant speech/nonspeech comparison, because listeners can be asked to attend to the syllable and make pho- netic judgments that will vary as the critical formant transi- tion varies, and they can be asked to attend to the chirps and make analogous judgments about them. Presented with a continuum of F3 transitions to one ear and the base to the other, and under instructions to discriminate syllable pairs or chirp pairs, listeners responded quite differently depending on the judgment, even though both judgments were based on the same acoustic pattern. Figure 9.8 shows that their speech discrimination judgments showed a sharply peaked pattern similar to that in Figure 9.7. Their chirp judgments showed a nearly monotonically decreasing pattern. This study, among others, shows that not all comparisons of speech and non- speech perception have uncovered similarities. Learning and Speech Perception So far, it may appear as if speech perception is unaffected by a language user’s experience talking and listening. It is af- fected, however. Experience with the language affects how listeners categorize consonants and vowels, and it affects the internal structure of native language phonological categories. It also provides language users with knowledge of the relative frequencies with which consonants and vowels follow one another in speech (e.g., Pitt & McQueen, 1998; Vitevitch & Luce, 1999) and with knowledge of the words of the Figure 9.8 Results of speech and nonspeech discriminations of syllables and chirps (Mann & Liberman, 1983). Speech Nonspeech 100 90 80 70 60 50 2-5 3-6 4-7 5-8 6-91-4 Stimulus Pair Percent discrimination 258 Speech Production and Perception language. It is currently debated (e.g., Norris, McQueen, & Cutler, 1999; Samuel, 2000) whether particularly lexical knowledge, in fact, affects speech perception, but it is clear that it affects how listeners ultimately identify consonants and vowels. Knowledge of Categories The concept of category (see chapter by Goldstone & Kersten in this volume) remains rather fuzzy, although it is clear that it is required for understanding speech perception. Language users treat sets of physically distinct tokens of consonants and vowels as functionally equivalent. For example, English listeners treat tokens of /t/ as members of the same category when /t/s differ in aspiration due to variation in position in a syllable or stress, and when they differ due to speaking rate, coarticulatory context, dialect, foreign accent, and idio- syncratic speaker characteristics. (They treat them as func- tionally equivalent, for example, when they count physically distinct /t/s before /ap/ all as consonants of the word top.) The concept of category is meant to capture this behavior. Func- tional equivalence of physically distinct tokens may or may not imply that listeners represent consonants and vowels as abstract types. The section titled “Another Abstractness Issue: Exemplar Theories of the Lexicon” described exem- plar theories of linguistic knowledge in which clusters of rel- evantly similar tokens underlie behaviors suggestive of type memories. Accordingly, the reader should interpret the fol- lowing discussion of categories as neutral between the pos- sibilities that abstract types are or are not components of linguistic competence. From the earliest ages at which they are tested, infants show evidence of categorization. On the one hand, they exhibit something like categorical perception. Eimas, Siqueland, Jusczyk, and Vigorito (1971) pioneered the use of a high- amplitude sucking technique to test infants as young as one month of age. Infants sucked on a nonnutritive nipple. If they sucked with sufﬁcient vigor they heard a speech syllable, for example, /ba/. Over time, infants increased their sucking rate under those conditions, but eventually they showed habitua- tion: Their sucking rate declined. Following that, Eimas et al. presented different syllables to all infants except those in the control group. They presented a syllable that adult listeners heard as /pa/ or one that was acoustically as distant from the original /ba/ as the /pa/ syllable but that adults identiﬁed as /ba/. Infants dishabituated to the ﬁrst syllable, showing that they heard the difference, but they did not dishabituate to the second. Kuhl and colleagues (e.g., Kuhl & Miller, 1982) have shown that infants classify by phonetic type syllables that they readily discriminate. Kuhl and Miller trained 6-month- old infants to turn their head when they heard a phonetic change in a repeating background vowel (from /a/ to /i/). Then they increased the difﬁculty of the task by presenting as background /a/ vowels spoken by different speakers or with different pitch contours. These vowels are readily discrimi- nated by infants, but adults would identify all of them as /a/. When a change occurred, it was to /i/ vowels spoken by the different speakers or produced with the different pitch con- tours. Infants’ head turn responses demonstrated that they detected the phonetic identity of the variety of /a/ vowels and the phonetic difference between them and the /i/ vowels. We know, then, that infants detect phonetic invariance over irrelevant variation. However, with additional experience with their native language, they begin to show differences in what they count as members of the same and different cate- gories. For example, Werker and Tees (1984) showed that English-learning infants at 6–8 months of age distinguished Hindi dental and retroﬂex voiceless stops. However, at 10–12 months they did not. English- (non-Hindi-) speaking adults also had difﬁculty making the discrimination, whereas Hindi adults and three Hindi 10–12 month olds who were tested made the discrimination readily. One way to understand the English-learning infants’ loss in sensitivity to the phonetic distinction is to observe that, in English, the distinction is not contrastive. English alveolar stops are most similar to the Hindi dental and retroﬂex stops. If an English speaker (per- haps due to coarticulation) were to produce a dental stop in place of an alveolar one, it would not change the word being produced from one word into another. With learning, cate- gories change their structure to reﬂect the patterning of more and less important phonetic distinctions of the language to which the learner is exposed. In recent years, investigators have found that categories have an internal structure. Whereas early ﬁndings from cate- gorical perception implied that all category members, being indiscriminable, must be equally acceptable members of the category, that is not the case, as research by Kuhl (e.g., 1991) and by Miller has shown. Kuhl (e.g., 1987) has suggested that categories are orga- nized around best instances or prototypes. When Grieser and Kuhl (1989) created a grid of vowels, all identiﬁed as /i/ by listeners (ostensibly; but see Lively & Pisoni, 1995) but dif- fering in their F1s and F2s, listeners gave higher goodness ratings to some tokens than to others. Kuhl (1991) showed, in addition, that listeners (adults and infants aged 6–7 months, but not monkeys) showed poorer discrimination of /i/ vowels close to the prototype (that is, the vowel given the highest goodness rating) than of vowels from a nonprototype (a vowel given a low goodness rating), an outcome she called Speech Perception 259 the “magnet effect.” Kuhl, Williams, Lacerda, Stevens, and Lindblom (1992) showed that English and Swedish infants show magnet effects around different vowels, reﬂecting the different vowel systems of their languages. We should not think of phonological categories as having an invariant prototype organization, however. Listeners iden- tify different category members as best exemplars in different contexts. This has been shown most clearly in the work of Joanne Miller and colleagues. Miller and colleagues (e.g., Miller & Volaitis, 1989) have generated acoustic continua ranging, for example, from /bi/ to /pi/ and beyond to a very long VOT /p/ designated /p/. Listeners make goodness judg- ments to the stimuli (in the example, they rate the goodness of the consonants as /p/s), and Miller and colleagues get data like those in Figure 9.9. (The VOT continuum is truncated at the long end in the ﬁgure.) The functions have a peak and graded sides. Miller and collaborators have shown that the location of the best rated consonant along an acoustic contin- uum can vary markedly with rate of production, syllable structure, and other variables. An effect of rate is shown in Figure 9.9 (where the legend’s designations “125 ms” and “325 ms” are syllable durations for fast and slow produc- tions, respectively). Faber and Brown (1998) showed a change in the prototype with coarticulatory context. These ﬁndings suggest that the categories revealed by these studies have a dynamical character (cf. Tuller, Case, & Kelso, 1994). How should the ﬁndings of Kuhl and colleagues and of Miller and colleagues be integrated? It is not yet clear. Kuhl (e.g., Kuhl & Iverson, 1995) acknowledges that her ﬁndings are as consistent with a theory in which there are actual pro- totypes in memory as with one in which prototypicality is an emergent property of an exemplar memory. It may be easier in an exemplar theory to understand how categories can change their structure dynamically. Possibly, Kuhl’s magnet effect can also be understood from the framework of Miller’s (e.g., Miller & Volaitis, 1989) ﬁndings if both sets of ﬁndings are related to Catherine Best’s perceptual assimilation model (PAM; e.g., Best, 1994). PAM is a model that captures consequences of perceptual speech learning. In the model, experience with the language eventu- ates in the formation of language-speciﬁc categories. When listeners are given two nonnative consonants or two vowels to discriminate, and they fall into the same native category, discrimination is very poor if the phones are equally good ex- emplars of the category. Discrimination is better if one is judged a good and one a poor exemplar. This can be under- stood by looking at Figure 9.9. Tokens that fall near the peak of the goodness function sound very similar to listeners, and they sound like good members of the category. However, one token at the peak and one over to the left or right side of the function sound different in goodness and therefore presum- ably in phonetic quality. Functions with ﬂat peaks and accel- erating slopes to the sides of the function would give rise to a magnet effect. That is, tokens surrounding the peak would be difﬁcult to discriminate, but equally acoustically similar to- kens at the sides of the function (so a nonprototype and a token near to it) would differ considerably in goodness and be easily discriminable. Lexical and Phonotactic Knowledge Word knowledge can affect how phones are identiﬁed, as can knowledge of the frequencies with which phones follow one another in speech. Ganong (1980) showed that lexical knowl- edge can affect how a phone is identiﬁed. He created pairs of continua in which the phone sequence at one end was a word but the sequence at the other end was a nonword. For exam- ple, in one pair of continua, VOT was varied to produce a gift- to- kift continuum and a giss- to- kiss continuum. Ganong found that listeners provided more g responses in the gift-kift continuum than in the giss - kiss continuum. That is, they tended to give responses suggesting that they identiﬁed real words preferentially. This result has recently been repli- cated with audiovisual speech. Brancazio (submitted) has shown that participants exhibit more McGurk integrations if they turn acoustically speciﬁed nonwords into words (e.g., acoustic besk dubbed onto video desk, with the integrated McGurk response being desk ) than if they turn acoustically speciﬁed words into nonwords (e.g., acoustic bench dubbed on to video dench ). Ganong’s (1980) result has at least two interpretations. One is that lexical information feeds down and affects per- ceptual processing of consonants and vowels. An alternative is that perceptual processing of consonants and vowels is Figure 9.9 Goodness ratings along a /bi/-/pi/-/pi/ continuum. Data simi- lar to those of Miller and Volaitis (1989). 125 ms 325 ms Goodness rating 2 0 120 VOT (ms) 4 6 8 10 ….. 260 Speech Production and Perception encapsulated from such feedback; however, when the proces- sor yields an ambiguous output, lexical knowledge is brought to bear to resolve the ambiguity. In the ﬁrst account, the effect of the lexicon is on perceptual processing; in the second it is on processing that follows perception of phones. The Ganong paradigm has been used many times in creative attempts to distinguish these interpretations (e.g., Fox, 1984; Miller & Dexter, 1988; Newman, Sawusch, & Luce, 1997). However, it remains unresolved. A second ﬁnding of lexical effects is phonemic restoration (e.g., Samuel, 1981, 1996; Warren, 1970). When the acoustic consequences of a phoneme are excised from a word (in Warren’s classic example, the /s/ noise of legislature ) and are replaced with noise that would mask the acoustic signal if it were present, listeners report hearing the missing phoneme and mislocate the noise. Samuel (1981) showed that when two versions of these words are created, one in which the acoustic consequences are present in the noise and one in which they are absent, listeners asked to make a judgment whether the phone is present or absent in the noise show lower perceptual sensitivity to phones in words than in non- words. That the effect occurs on the measure of perceptual sensitivity (d  ) suggests that, here, lexical knowledge is ex- erting its effect on phoneme perception itself. (However, that d  can be so interpreted in word recognition experiments has been challenged; see Norris, 1995.) A ﬁnal lexical effect occurs in experiments on compensa- tion for coarticulation. Mann and Repp (1981) found compen- sation for /s/ and / ʃ / on members of a /ta/-to-/ka/ continuum such that the more front /s/ fostered /ka/ responses, and the more back / ʃ / fostered /ta/ responses. Elman and McClelland (1988) used compensation for coarticulation in a study that seemingly demonstrated lexical feedback on perceptual pro- cessing of consonants. They generated continua ranging from /d/ to /g/ (e.g., dates to gates ) and from /t/ to /k/ (e.g., tapes to capes ). Continuum members followed words such as Christ- mas and Spanish in which the ﬁnal fricatives of each word (or, in another experiment, the entire ﬁnal syllables) were re- placed with the same ambiguous sound. Accordingly, the only thing that made the ﬁnal fricative of Christmas an /s/ was the listeners’knowledge that Christmas is a word and Christmash is not. Lexical knowledge, too, was all that made the ﬁnal fricative of Spanish an / ʃ /. Listeners showed compensation for coarticulation appropriate for the lexically speciﬁed frica- tives of the precursor words. This result is ascribed to feedback effects on perception, because compensation for coarticulation is quite evidently an effect that occurs during perceptual processing of phones. However, Pitt and McQueen (1998) challenged the feedback interpretation with ﬁndings appearing to show that the effect is not really lexical. It is an effect of listeners’ knowledge of the relative frequencies of phone sequences in the language, an effect that they identify as prelexical and at the same level of processing as that on which phonemes are perceived. Pitt and McQueen note that in English, /s/ is more likely to fol- low the ﬁnal vowel of Christmas than is / ʃ /, and / ʃ / is more common than /s/ following the ﬁnal vowel of Spanish. (If readers ﬁnd these vowels—ostensibly / ə / and /I/ according to Pitt and McQueen—rather subtly distinct, they are quite right.) These investigators directly pitted lexical identity against phone sequence frequency and found compensation for coarticulation fostered only by the transition probability variable. Lately, however, Samuel (2000) reports ﬁnding a true lexical effect on phoneme perception. The clear result is that lexical knowledge affects how we identify consonants and vowels. It is less clear where in processing the lexical effect comes in. Pitt and McQueen’s study introduces another knowledge variable that can affect phone identiﬁcation: knowledge of the relative transition frequencies between phones. Although this logically could be another manifestation of our lexical knowledge, Pitt and McQueen’s ﬁndings suggest that it is not, because lexical and transition-probability variables dis- sociate in their effects on compensation for coarticulation. A conclusion that transition probability effects arise prelexi- cally is reinforced by recent ﬁndings of Vitevitch and Luce (1998, 1999). There are many models of spoken-word recognition. They include the pioneering TRACE (McClelland & Elman, 1986), Marslen-Wilson’s (e.g., 1987) cohort model, the neighborhood activation model (NAM; Luce, 1986; Luce & Pisoni, 1998), the fuzzy logical model of perception (FLMP; e.g., Massaro, 1987, 1998), and shortlist (e.g., Norris, 1994). (A more recent model of Norris et al., 1999, Merge, is currently a model of phoneme identiﬁcation; it is not a full- ﬂedged model of word recognition.) I will describe just two models, TRACE and a nameless recurrent network model described by Norris (1993); these models represent extremes along the dimension of interactive versus feedforward only (autonomous) models. In TRACE, acoustic signals are mapped onto phonetic fea- tures, features map to phonemes, and phonemes to words. Features activated by acoustic information feed activation forward to the phonemes to which they are linked. Phonemes activate words that include them. Activation also feeds back from the word level to the phoneme level and from the phoneme level to the feature level. It is this feedback that identiﬁes TRACE as an interactive model. In the model, there is also lateral inhibition; forms at a given level inhibit forms at the same level with which they are incompatible. Lexical References 261 effects on phoneme identiﬁcation (e.g., the Ganong effect and phonemic restoration) arise from lexical feedback. Given an ambiguous member of a gift- to- kift continuum, the word gift will be activated at the lexical level and will feed activation back to its component phonemes, including /g/, thereby fos- tering identiﬁcation of the ambiguous initial consonant as /g/. Lexical feedback also restores missing phonemes in the phonemic restoration effect. In TRACE, knowledge of transition probabilities is the same as knowledge of words. That is, words and nonwords with high transition probabilities include phoneme sequences that occur frequently in words of the lexicon. TRACE cannot generate the dissociations between effects of lexical knowl- edge and transition probabilities that both Pitt and McQueen (1998) and Vitevitch and Luce (1998) report. A second short- coming of TRACE is its way of dealing with the temporally extended character of speech. To permit TRACE to take in utterances over time, McClelland and Elman (1986) used the brute force method of replicating the entire network of feature, phone, and word nodes at many different points in modeled time. Norris’s (1993) recurrent network can handle temporally extended input without massive replication of nodes and links. The network has input nodes that receive as input sets of features for phonemes. The feature sets for successive phonemes are input over time. Input units link to hidden units, which link to output units. There is one set of output units for words and one for phonemes. The hidden units also link to one another over delay lines. It is this aspect of the net- work that allows it to learn the temporally extended phoneme sequences that constitute words. The network is trained to ac- tivate the appropriate output unit for a word when its compo- nent phonemes’ feature sets are presented over time to the input units and to identify phonemes based on featural input. The network has the notable property that it is feedforward only; that is, in contrast to TRACE, there is no top-down feedback from a lexical to a prelexical level. Recurrent net- works are good at learning sequences, and the learning re- sides in the hidden units. Accordingly, the hidden units have probabilistic phonotactic knowledge. Norris has shown that this model can exhibit the Ganong effect and compensation for coarticulation; before its time, it demonstrated ﬁndings like those of Pitt and McQueen (1998) in which apparently top-down lexical effects on compensation for coarticulation in fact arise prelexically and depend on knowledge of transi- tion probabilities. This type of model (see also Norris et al., 1999) is remarkably successful in simulating ﬁndings that had previously been ascribed to top-down feedback. How- ever, the debate about feedback is ongoing (e.g., Samuel, 2000). SUMMARY Intensive research on language forms within experimen- tal psychology has only a 50-year history, beginning with the work by Liberman and colleagues at Haskins Laborato- ries. However, this chapter shows that much has been learned in that short time. Moreover, the scope of the research has broadened considerably, from an initial focus on speech perception only to current research spanning the domains of competence, planning production, and perception. Addi- tionally, in each domain, the experimental methodolgies developed by investigators have expanded and include some remarkably useful ways of probing the psychology of phonology. Theoretical developments have been considerable, too. Within each domain, competing theoretical views have grown that foster efforts to sharpen the theories and to distin- guish them experimentally. Moreover, we now have theories in domains, such as planning, where earlier there were none. The scope and depth of our understanding of language forms and their role in language use has grown impressively. A rel- atively new development that is proving very useful is the use of models that implement theories. The models of Dell (1986) and Levelt et al. (1999) of phonological planning, of Guenther et al. (1998) and Saltzman (1991) on speech pro- duction, and of McClelland and Elman (1986) and Norris (1994), among others, of speech perception all help to make theoretical differences explicit and theoretical claims testable. We have much more to learn, of course. My own view, made clear in this chapter, is that enduring advances depend on more cross-talk across the domains of competence, plan- ning, production, and perception. REFERENCES Abbs, J., & Gracco, V. (1984). Control of complex gestures: Orofa- cial muscle responses to load perturbations of the lip during speech. Journal of Neurophysiology, 51, 705–723. Abler, W. (1989). On the particulate principle of self-diversifying systems. Journal of Social and Biological Structures, 12, 1–13. Baars, B., Motley, M., & MacKay, D. G. (1975). Output editing for lexical status from artiﬁcially elicited slips of the tongue. Journal of Verbal Learning and Verbal Behavior, 14, 382–391. Bell-Berti, F., & Harris, K. (1981). A temporal model of speech production. Phonetica, 38, 9–20. Bell-Berti, F., & Krakow, R. (1991). Anticipatory velar lowering: A coproduction account. Journal of the Acoustical Society of America, 90, 112–123. 262 Speech Production and Perception Best, C. T. (1994). The emergence of native-language phonological inﬂuences in infants: A perceptual assimilation model. In J. Goodman & H. Nusbaum (Eds.), The development of speech perception: The transition from speech sounds to spoken words (pp. 167–224). Cambridge, MA: MIT Press. Best, C. T. (1995). A direct realist perspective on cross-language speech perception. In W. Strange & J. J. Jenkins (Eds.), Cross- language speech perception (pp. 171–204). Timonium, MD: York Press. Bladon, A., & Al-Bamerni, A. (1976). Coarticulation resistance in English /l/. Journal of Phonetics, 4, 137–150. Boyce, S., Krakow, R., Bell-Berti, F., & Gelfer, C. (1990). Converg- ing sources of evidence for dissecting articulatory movements into gestures. Journal of Phonetics, 18, 173–188. Brancazio, L. (2002). Lexical inﬂuences in audiovisual speech per- ception . Manuscript submitted for publication. Browman, C., & Goldstein, L. (1986). Towards an articulatory phonology. Phonology Yearbook, 3, 219–252. Browman, C., & Goldstein, L. (1990). Tiers in articulatory phonol- ogy, with some implications for casual speech. In J. Kingston & M. Beckman (Eds.), Papers in laboratory phonology: Vol. 1. Between the grammar and the physics of speech (pp. 341–376). Cambridge, England: Cambridge University Press. Browman, C., & Goldstein, L. (1995a). Dynamics and articulatory phonology. In R. Port & T. van Gelder (Eds.), Mind as motion: Explorations in the dynamics of cognition (pp. 175–193). Cambridge, MA: MIT Press. Browman, C., & Goldstein, L. (1995b). Gestural syllable position effects in American English. In F. Bell-Berti & L. Raphael (Eds.), Producing speech: Contemporary issues. (pp. 19–33). New York: American Institute of Physics. Browman, C., & Goldstein, L. (1997). The gestural phonology mode. In W. Hulstijn, H. F. M. Peters, & P. H. H. Van Lieshout (Eds.), Speech production: Motor control, brain research and ﬂuency disorders (International Congress Series, No. 1146). Amsterdam: Elsevier. Chomsky, N., & Halle, M. (1968). The sound pattern of English. New York: Harper and Row. Clark, H. (1996). Using language . Cambridge, England: Cambridge University Press. Delattre, P., & Freeman, D. (1968). A dialect study of American r’s by x-ray motion picture. Linguistics, 44, 29–68. Dell, G. S. (1986). A spreading-activation theory of retrieval in speech production. Psychological Review, 93, 283–321. Dell, G. S., Burger, L., & Svec, W. (1997). Language production and serial order: A functional analysis and a model. Psychological Review, 104, 127–147. Dell, G. S., & Juliano, C. (1996). Computational models of phono- logical encoding. In T. Dijkstra & K. DeSmedt (Eds.), Com- putational psycholinguistics: AI and connectionist models of language processing (pp. 328–359). Philadelphia: Taylor & Francis. Dell, G. S., Juliano, C., & Govindjee,

bundet

A. (1993). Structure and content in language production: A theory of frame constraints. Cognitive Science, 17, 149–195. Diehl, R., & Kluender, K. (1989). On the objects of speech percep- tion. Ecological Psychology, 1, 121–144. Easton, T. (1972). On the normal use of reﬂexes. American Scientist, 60, 591–599. Eimas, P., Siqueland, E., Jusczyk, P., & Vigorito, J. (1971). Speech perception in infants. Science, 171, 303–306. Elman, J. & McClelland, J. (1988). Cognitive penetration of the mechanisms of perception: Compensation for coarticulation of lexically restored phonemes. Journal of Memory and Language, 27 , 143–165. Faber, A., & Brown, J. (1998). The effect of consonant context on vowel goodness rating [Abstract]. Journal of the Acoustical Society of America, 104, 1759 (abstract). Fitch, H., Halwes, T., Erickson, D., & Liberman, A. (1980). Percep- tual equivalence of two acoustic cues for stop-consonant man- ner. Perception and Psychophysics, 27, 343–350. Fodor, J. A. (1983). The modularity of mind: An essay on faculty psychology . Cambridge, MA: MIT Press. Fowler, C. A. (1986). An event approach to the study of speech per- ception from a direct-realist perspective. Journal of Phonetics, 14, 3–28. Fowler, C. A. (1994). Invariants, speciﬁers cues: An investigation of locus equations as information for place of articulation. Percep- tion & Psychophysics, 55, 597–610. Fowler, C. A. (1996). Listeners do hear sounds, not tongues. Journal of the Acoustical Society of America, 99, 1730–1741. Fowler, C. A. (1999). The orderly output constraint is not wearing any clothes. Behavioral and Brain Science, 21, 265–266. Fowler, C. A., & Brancazio, L. (2000). Coarticulation resistance of American English consonants and its effects on transconsonantal vowel-to-vowel coarticulation. Language and Speech, 43, 1–41. Fowler, C. A., Brown, J., & Mann, V. (2000). Contrast effects do not underlie effects of preceding liquid consonants on stop identiﬁ- cation in humans. Journal of Experimental Psychology: Human Perception and Performance, 26, 877–888. Fowler, C. A., & Dekle, D. J. (1991). Listening with eye and hand: Crossmodal contributions to speech perception. Journal of Experimental Psychology: Human Perception and Performance, 17, 816–828. Fowler, C. A., & Levy, E. (1995). Talker-listener attunements to speech events. Journal of Contemporary Legal Studies, 6, 305–328. Fowler, C. A., & Rosenblum, L. D. (1990). Duplex perception: A comparison of monosyllables and slamming doors. Journal of Experimental Psychology: Human Perception and Performance, 16, 742–754. Fowler, C. A., Rubin, P., Remez, R., & Turvey, M. (1980). Implica- tions for speech production of a general theory of action. In References 263 B. Butterworth (Ed.), Language production: Vol. 1. Speech and talk (Vol. 1, pp. 373–420). London: Academic Press. Fowler, C. A., & Saltzman, E. (1993). Coordination and coarticula- tion in speech production. Language and Speech, 36, 171–195. Fowler, C. A., & Smith, M. (1986). Speech perception as “vector analysis”: An approach to the problems of segmentation and invariance. In J. Perkell & D. Klatt (Eds.), Invariance and variability of speech processes (pp. 123–136). Hillsdale, NJ: Lawrence Erlbaum. Fox, R. (1984). Effect of lexical status on phonetic categorization. Journal of Experimental Psychology: Human Perception and Performance, 10, 526–540. Fromkin, V. (1973). Speech errors as linguistic evidence . The Hague, the Netherlands: Mouton. Ganong, W. F. (1980). Phonetic categorization in auditory word perception. Journal of Experimental Psychology: Human Per- ception and Performance, 6, 110–125. Garrett, M. (1980). Levels of processing in speech production. In B. Butterworth (Ed.), Language production: Vol. 1. Speech and talk (pp. 177–220). London: Academic Press. Gibson, J. J. (1966). The senses considered as perceptual systems . Boston: Houghton Mifﬂin. Gibson, J. J. (1979). The ecological approach to visual perception . Boston: Houghton Mifﬂin. Gick, B. (1999). The articulatory basis of syllable structure: A study of English glides and liquids. Unpublished doctoral dissertation. Yale University, New Haven, CT. Gleason, P., Tuller, B., & Kelso, J.A. S. (1996). Syllable afﬁliation of ﬁnal consonant clusters undergoes a phase transition over speak- ing rates. Proceedings of the International Conference on Speech and Language Processing (pp. 276–278). Philadelphia, PA. Goldinger, S. D. (1996). Words and voices: Episodic traces in spo- ken word identiﬁcation and recognition memory. Journal of Experimental Psychology: Learning, Memory and Cognition, 22, 1166–1183. Goldinger, S. D. (1998). Echoes of echoes? An episodic theory of lexical access. Psychological Review, 105, 251–279. Grieser, D., & Kuhl, P. (1989). Categorization of speech by infants: Support for speech-sound prototypes. Developmental Psychol- ogy, 25, 577–588. Guenther, F., Hampson, M., & Johnson, D. (1998). A theoretical investigation of reference frames for the planning of speech. Psychological Review, 105, 611–633. Guilford, J., & Park, D. (1931). The effect of interpolated weights upon comparative judgments. American Journal of Psychology, 43, 589–599. Hagiwara, R. (1995). Acoustic realizations of American /r/ as produced by women and men. UCLA Working Papers in Pho- netics, 90 . Hamlet, S. (1988). Speech compensation for prosthodontially created palatal asymmetries. Journal of Speech and Hearing Research, 31, 48–53. Hamlet, S., & Stone, M. (1978). Compensatory alveolar consonant production induced by wearing a dental prosthesis. Journal of Phonetics, 6 , 227–248. Hintzman, D. (1986). “Schema abstraction” in a multiple trace memory model. Psychological Review, 93 , 411–428. Hockett, C. (1955). A manual of phonetics . Bloominton: Indiana University Press. Humbolt, W. Von. (1972). Linguistic variability and intellectual development. (G. C. Buck & F. A. Raven, Trans.) Philadelphia: University of Pennsylvania Press. (Original work published 1836) Jakobson, R., Fant, G., & Halle, M. (1962). Preliminaries to speech analysis . Cambridge, MA: MIT Press. Keating, P. (1983). Comments on the jaw and syllable structure. Journal of Phonetics, 11, 401–406. Kelso, J. A. S. (1995). Dynamic patterns: The self-organization of brain and behavior. Cambridge, MA: MIT Press. Kelso, J. A. S., Tuller, B., Vatikiotis-Bateson, E., & Fowler, C. A. (1984). Functionally-speciﬁc articulatory cooperation following jaw perturbation during speech: Evidence for coordinative struc- tures . Journal of Experimental Psychology: Human Perception and Performance, 10, 812–832. Kent, R., & Tjaden, K. (1997). Brain functions underlying speech. In W. Hardcastle & J. Laver (Eds.), The handbook of phonetic sciences (pp. 221–255). Oxford, England: Blackwell Publishers. Kluender, K. (1994). Speech perception as a tractable problem in cognitive science. In M. A. Gernsbacher (Ed.), Handbook of Psy- cholinguistics (pp. 173–217). San Diego, CA: Academic Press. Krakow, R. (1989). The articulatory organization of syllables: A kinematic analysis of labial and velar gestures. Unpublished doctoral dissertation. Yale University, New Haven, CT. Krakow, R. (1993). Nonsegmental inﬂuences on velum movement patterns: Syllables, segments, stress and speaking rate. In M. Huffman & R. Krakow (Eds.), Phonetics and phonology: Vol. 5. Nasals, nasalization and the velum (pp. 87–116). New York: Academic Press. Krakow, R. (1999). Physiological organization of syllables: A review. Journal of Phonetics, 27, 23–54. Kuhl, P. (1987). The special-mechanisms debate in speech research: Categorization tests on animals and infants. In S. Harnad (Ed.), Categorical perception: The groundwork of cognition (pp. 355– 386). Cambridge, England: Cambridge University Press. Kuhl, P. (1991). Human adults and human infants show a “percep- tual magnet effect” for the prototypes of speech categories, mon- keys do not. Perception & Psychophysics, 50, 93–107. Kuhl, P., & Iverson, P. (1995). Linguistic experience and the per- ceptual magnet effect. In W. Strange (Ed.), Speech perception and linguistic experience: Issues in cross-language research (pp. 121–154). Baltimore: York Press. Kuhl, P., & Miller, J. D. (1978). Speech perception by the chinchilla: Identiﬁcation functions for synthetic VOT stimuli. Journal of the Acoustical Society of America, 63, 905–917. 264 Speech Production and Perception Kuhl, P., & Miller, J. D. (1982). Discrimination of auditory target dimensions in the presence or absence of variation in a second di- mension by infants. Perception & Psychophysics, 31, 279–292. Kuhl, P., Williams, K., Lacerda, F., Stevens, K., & Lindblom, B. (1992). Linguistic experience alters phonetic perception in infants by six months of age. Science, 255, 606–608. Levelt, W. J. M., Roelofs, A., & Meyer, A. (1999). A theory of lexi- cal access in speech production. Behavioral and Brain Sciences, 22, 1–38. Liberman, A. M. (1957). Some results of research on speech per- ception. Journal of the Acoustical Society of America, 29, 117–123. Liberman, A. M. (1996). Speech: A special code. Cambridge, MA: Bradford Books. Liberman, A. M., Cooper, F., Shankweiler, D., & Studdert-Kennedy, M. (1967). Perception of the speech code. Psychological Review, 74, 431–461. Liberman, A. M., Delattre, P., & Cooper, F. (1952). The role of se- lected stimulus variables in the perception of the unvoiced-stop consonants. American Journal of Psychology, 65, 497–516. Liberman, A. M., & Mattingly, I. (1985). The motor theory revised. Cognition, 21, 1–36. Liberman, A. M., & Whalen, D. H. (2000). On the relation of speech to language. Trends in Cognitive Sciences, 4, 187–196. Lieberman, P. (1991). Uniquely human: Speech, thought and selﬂess behavior . Cambridge, MA: Harvard University Press. Lindblom, B. (1986). Phonetic universals in vowel systems. In J. Ohala & J. Jaeger (Eds.), Experimental phonology (pp. 13–44). Orlando, FL: Academic Press. Lindblom, B. (1990). On the notion of “possible speech sound.” Journal of Phonetics, 18, 135–142. Lindblom, B., Lubker, J., & Gay, T. (1979). Formant frequencies of some ﬁxed mandible vowels and a model of speech motor pro- gramming by predictive simulation. Journal of Phonetics, 7, 147–161. Lindblom, B., & Maddieson, I. (1988). Phonetic universals in con- sonant systems. In L. Hyman & C. N. Li (Eds.), Language, speech and mind (pp. 62–78). London: Routledge. Lisker, L. (1978). Rapid vs rabid: A catalogue of acoustic features that may cue the distinction. Haskins Laboratories Status Report on Speech Research, 54, 127–132. Lively, S. & Pisoni, D. (1995). On prototypes and phonetic cate- gories: A critical assessment of the perceptual magnet effect in speech perception. Journal of Experimental Psychology: Human Perception and Performance, 23, 1665–1679. Löfquist, A. (1997). Theories and models of speech production. In W. Hardcastle & J. Laver (Eds.), The handbook of phonetic sciences (pp. 405–426). Oxford, England: Blackwell. Lotto, A., & Kluender, K. (1998). General contrast effects in speech perception: Effect of preceding liquid on stop consonant identiﬁ- cation. Perception & Psychophysics, 60, 602–619. Lotto, A., Kluender, K., & Holt, L. (1997). Perceptual compen- sation for coarticulation by Japanese quail ( coturnix coturnix japonica ). Journal of the Acoustical Society of America, 102, 1134–1140. Luce, P. (1986). Neighborhoods of words in the mental lexicon. Unpublished doctoral dissertation. Indiana University. Bloomington, Indiana. Luce, P., & Pisoni, D. (1998). Recognizing spoken words: The neighborhood activation model. Ear and Hearing, 19, 1–36. Maddieson, I. (1984). Patterns of sounds . Cambridge, England: Cambridge University Press. Mann, V. (1980). Inﬂuence of preceding liquid on stop-consonant perception. Perception & Psychophysics, 28, 407–412. Mann, V., & Liberman, A. (1983). Some differences between pho- netic and auditory modes of perception. Cognition, 14, 211–235. Mann, V., & Repp, B. (1981). Perceptual assessment of fricative- stop coarticulation. Journal of the Acoustical Society of America, 69, 1153–1169. Marslen-Wilson, W. (1987). Functional parallelism in spoken word recognition. Cognition, 25, 71–102. Marslen-Wilson, W., & Welsh, A. (1978). Processing interactions and lexical access during word recognition in continuous speech. Cognitive Psychology, 10, 29–63. Massaro, D. (1987). Speech perception by ear and eye: A paradigm for psychological inquiry . Hillsdale, NJ: Lawrence Erlbaum. Massaro, D. (1998). Perceiving talking faces . Cambridge, MA: MIT Press. McClelland, J., & Elman, J. (1986). The TRACE model of speech perception. Cognitive Psychology, 18, 1–86. McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746–748. Meyer, A. (1990). The time course of phonological encoding in lan- guage production: The encoding of successive syllables of a word. Journal of Memory and Language, 29, 524–545. Meyer, A. (1991). The time course of phonological encoding in lan- guage production: Phonological encoding inside a syllable. Journal of Memory and Language, 30, 69–89. Meyer, A., & Schrieffers, H. (1991). Phonological facilitation in picture-word interference experiments: Effects of stimulus onset asynchrony and types of interfering stimuli. Journal of Exper- imental Psychology: Learning, Memory and Cognition, 17, 1146–1160. Miller, J. L., & Dexter, E. R. (1988). Effects of speaking rate and lexical status on phonetic perception. Journal of Experi- mental Psychology: Human Perception and Performance, 14, 369–378. Miller, J. L., & Volaitis, L. (1989). Effect of speaking rate on the perceptual structure of a phonetic category. Perception & Psychophysics, 46, 505–512. Mish, Frederick C. (1990). Webster’s Ninth New Collegiate Dic- tionary. Springﬁeld, MA: Merriam-Webster, Inc. References 265 Motley, M. (1980). Veriﬁcation of “Freudian slips” and semantically prearticulatory editing via laboratory-induced spoonerisms. In V. Fromkin (Ed.), Errors in linguistic performance: Slips of the tongue, ear, pen, and hand (pp. 133–147). New York: Academic Press. Mowrey, R., & MacKay, I. (1990). Phonological primitives: Elec- tromyographic speech error evidence. Journal of the Acoustical Society of America, 88, 1299–1312. Nearey, T. (1997). Speech perception as pattern recognition. Journal of the Acoustical Society of America, 101, 3241–3254. Newman, R., Sawusch, J., & Luce, P. (1997). Lexical neighbor- hood effects in phonemic processing. Journal of Exper- imental Psychology: Human Perception and Performance, 23, 873–889. Norris, D. (1993). Bottom up connectionist models of “interaction.” In G. Altmann & R. Shillcock (Eds.), Cognitive models of speech processing: The second Sperlonga meeting (pp. 211–234). Hillsdale, NJ: Lawrence Erlbaum. Norris, D. (1994). Shortlist: A connectionist model of continuous word recognition. Cognition, 52, 189–234. Norris, D. (1995). Signal detection theory and modularity: On being sensitive to the power of bias models of semantic priming. Journal of Experimental Psychology: Human Perception and Performance, 21, 935–939. Norris, D., McQueen, J., & Cutler, A. (1999). Merging information in speech recognition: Feedback is never necessary. Behavioral and Brain Sciences, 23, 299–370. Ohala, J. (1996). Listeners hear sounds not tongues. Journal of the Acoustical Society of America, 99, 1718–1728. Ohman, S. (1966). Coarticulation in VCV utterances: Spectro- graphic measurements. JASA, 39, 151–168. Orden, G. van, Pennington, B., & Stone, G. (1990). Word identiﬁca- tion in reading and the promise of subsymbolic psycholinguis- tics. Psychological Review, 97, 488–522. Pardo, J., & Fowler, C. A. (1997). Perceiving the causes of coartic- ulatory acoustic variation: Consonant voicing and vowel pitch. Perception & Psychophysics , 59, 1141–1152. Parker, E. M., Diehl, R. L., & Kluender, K. R. (1986). Trading rela- tions in speech and nonspeech. Perception & Psychophysics, 39, 129–142. Perkell, J., Matthies, M., Svirsky, M., & Jordan, M. (1993). Trading relations between tongue-body raising and lip rounding in pro- duction of the vowel /u/: A pilot “motor equivalence” study. Journal of the Acoustical Society of America, 93, 2948–2961. Pinker, S. (1994). The language instinct. NewYork:William Morrow. Pisoni, D. B., & Tash, J. (1974). Reaction times to comparisons within and across phonetic boundaries. Perception & Psy- chophysics, 15, 285–290. Pitt, M., & McQueen, J. (1998). Is compensation for coarticulation mediated by the lexicon? Journal of Memory and Language, 39, 347–370. Porter, R., & Castellanos, F. X. (1980). Speech production measures of speech perception: Rapid shadowing of VCV syllables. Journal of the Acoustical Society of America, 67, 1349–1356. Porter, R., & Lubker, J. (1980). Rapid reproduction of vowel-vowel sequences: Evidence for a fast and direct acoustic-motoric link- age. Journal of Speech and Hearing Research, 23, 593–602. Prince, A., & Smolensky, P. (1993). Optimality theory: Constraint interaction and satisfaction. Unpublished manuscript, Rutgers University, New Brunswick, NJ, University of Colorado, Boulder. Pulleyblank, D. (1997). Optimality theory and features. In D. Archangeli & D. T. Langendoen (Eds.), Optimality theory: An overview (pp. 59–101). Malden, MA: Blackwell. Recasens, D. (1984). V-to-C coarticulation in Catalan VCV sequences: An articulatory and acoustical study. Journal of Phonetics, 12, 61–73. Roelofs, A., & Meyer, A. (1998). Metrical structure in planning the production of spoken words. Journal of Experimental Psychol- ogy: Learning, Memory and Cognition, 24, 922–939. Rosenblum, L. D. (1987). Towards an ecological alternative to the motor theory. Perceiving-acting workshop, 2, 25–28. Ryle, G. (1949). The concept of mind . New York: Barnes and Noble. Saltzman, E. (1991). The task dynamic model in speech production. In H. F. M. Peters, W. Hulstijn, & C. W. Starkweather (Eds.), Speech motor control and stuttering (pp. 37–52). Amsterdam: Elsevier Science. Saltzman, E. (1995). Intergestural timing in speech production: Data and modeling. Proceedings of the XIIIth International Congress of Phonetic Sciences, Stockholm, 2, 84–88. Saltzman, E., & Byrd, D. (1999). Dynamic simulations of a phase window model of relative timing. Proceedings of the XIVth International Congress of Phonetic Sciences, San Francisco, 3, 2275–2278. Samuel, A. (1981). Phonemic restoration: Insights for a new methodology. Journal of Experimental Psychology: General, 110, 474–494. Samuel, A. (1996). Does lexical information inﬂuence the percep- tual restoration of phonemes. Journal of Experimental Psychol- ogy: General, 125, 28–51. Samuel, A. (2000). Some empirical tests of Merge’s architecture. In A. Cutler, J. McQueen, & R. Zondervan (Eds.), Proceedings of the workshop on spoken word access processes (pp. 51–54). Nijmegen: Max-Planck-Institute for Psycholinguistics. Santiago, J., & MacKay, D. G. (1999). Constraining production the- ories: Principled motivation, consistency, homunculi, under- speciﬁcation, failed predictions and contrary data. Behavioral and Brain Sciences, 22, 55–56. Savariaux, C., Perrier, P., & Orliaguet, J. P. (1995). Compensation strategies for the perturbation of the rounded vowel using a lip tube: A study of the control space in speech production. Journal of the Acoustical Society of America, 98, 2428–2442. 266 Speech Production and Perception Sawusch, J., & Gagnon, D. (1995). Auditory coding, cues and coherence in phonetic perception. Journal of Experimental Psychology: Human Perception and Performance, 21, 635–652. Sevald, C. A., Dell, G. S., & Cole, J. S. (1995). Syllable structure in speech production: Are syllables chunks or schemas. Journal of Memory and Language, 34, 807–820. Shaiman, S. (1989). Kinematic and electromyographic responses to perturbation of the jaw. Journal of the Acoustical Society of America, 86, 78–88. Shattuck-Hufnagel, S. (1979). Speech errors as evidence for a serial- ordering mechanism in sentence production. In W. Cooper & E. Walker (Eds.), Sentence processing: Psycholinguistic studies presented to Merrill Garrett (pp. 295–342). Hillsdale, NJ: Lawrence Erlbaum. Shattuck-Hufnagel, S. (1983). Sublexical units and suprasegmental structure in speech production planning. In P. MacNeilage (Ed.), The production of speech (pp. 109–136). New York: Springer- Verlag. Sinnott, J. M. (1974). A comparison of speech sound discrimination in humans and monkeys . Unpublished doctoral dissertation, University of Michigan. Sproat, R., & Fujimura, O. (1993). Allophonic variation in English /l/ and its implications for phonetic implementation. Journal of Pho- netics, 21 , 291–311. Stetson, R. H. (1951). Motor phonetics. Amsterdam: North Holland. Stevens, K. (1989). On the quantal nature of speech. Journal of Phonetics, 17, 3–45. Stevens, K., & Blumstein, S. (1981). The search for invariant corre- lates of phonetic features. In P. Eimas & J. Miller (Eds.), Perspectives on the study of speech (pp. 1–38). Hillsdale, NJ: Lawrence Erlbaum. Stone, G., Vanhoy, M., & Van Orden, G. (1997). Perception is a two- way street: Feedforward and feedback in visual word recogni- tion. Journal of Memory and Language, 36, 337–359. Studdert-Kennedy, M. (1998). The particulate origins of language generativity: From syllable to gesture. In J. Hurford, M. Studdert- Kennedy, & C. Knight (Eds.), Approaches to the evolution of language (pp. 202–221). Cambridge, England: Cambridge University Press. Sussman, H., Fruchter, D., Hilbert, J., & Sirosh, J. (1999a). Linear correlates in the speech signal: The orderly output constraint. Behavioral and Brain Sciences, 21, 287–299. Sussman, H., Fruchter, D., Hilbert, J., & Sirosh, J. (1999b). Human speech: A tinkerer’s delight. Behavioral and Brain Sciences, 21, 287–299. Trout, J. D. (2001). The biological basis of speech: What to infer from talking to the animals. Psychological Review, 108, 523–549. Tuller, B., Case, P., & Kelso, J. A. S. (1994). The nonlinear dynam- ics of speech categorization. Journal of Experimental Psychol- ogy: Human Perception and Performance, 20, 3–16. Tuller, B., & Kelso, J. A. S. (1990). Phase transitions in speech production and their perceptual consequences. In M. Jeannerod (Ed.), Attention and performance: Vol. 13. Motor represen- tation and control (pp. 429–452). Hillsdale, NJ: Lawrence Erlbaum. Tuller, B., & Kelso, J. A. S. (1991). The production and perception of syllable structure. Journal of Speech and Hearing Research, 34, 501–508. Turvey, M. T. (1990). Coordination. American Psychologist, 45, 938–953. Vitevitch, M., & Luce, P. (1998). When words compete: Levels of processing in perception of spoken words. Psychological Science, 9, 325–329. Vitevitch, M., & Luce, P. (1999). Probabilistic phonotactics and neighborhood activation in spoken word recognition. Journal of Memory and Language, 40, 374–408. Warren, R. M. (1970). Perceptual restoration of missing speech sounds. Science, 167, 392–393. Warren, R. M. (1985). Criterion shift rule and perceptual homeosta- sis. Psychological Review, 92, 574–584. Waters, R. S., & Wilson, W. A., (1976). Speech perception by rhe- sus monkeys: The voicing distinction in synthesized labial and velar stop consonants. Perception & Psychophysics, 19, 285–289. Weiss, P. (1941). Self differentiation of the basic pattern of coordi- nation. Comparative Psychology Monograph, 17, 21–96. Werker, J., & Tees, R. (1984). Cross-language speech perception: Evidence for perceptual reorganization during the ﬁrst year of life. Infant Behavior and Development, 7, 49–63. Zue, V. (1980). Acoustic characteristics of stop consonants: A controlled study . Bloomington: Indiana University Linguistics Club. PART FOUR HUMAN PERFORMANCE 269 EFFICIENCY OF SELECTION 270 Failures of Selectivity 270 Selection by Location and Other Features 276 PREATTENTIVE AND ATTENTIVE PROCESSING 280 Distributed Attention Paradigms 280 Inattention Paradigms: Dual-Task Experiments 282 Further Explorations of Preattentive Processing 282 Attention, Types and Tokens 285 CLOSING COMMENTS 287 REFERENCES 288 We live in a sea of information. The amount of information available to our senses vastly exceeds the information- processing capacity of our brains. How we deal with this overload is the topic of this chapter— attention . Consider your experience as you read this page. You are focused on just a word or two at a time. The rest of the page is available but is not being actively processed at this time. Indeed, quite apart from the other words on the page, there are many stim- uli impinging on you that you are probably not aware of, such as the pressure of your chair against your back. Of course, as soon as that pressure is mentioned you probably shifted your attention to that source of stimulation, at which point you most likely stopped reading brieﬂy. Some external stimuli do not need to be pointed out to you in order for you to become aware of them—for example, a mosquito buzzing around your face or the backﬁre of a car outside your window. These simple observations point to the selectivity of attention, its ability to shift quickly from one stimulus or train of thought to another, the difﬁculty we have in attending to more than one thing at a time, and the ability of some stimuli to capture attention. These are all important aspects of the topic of at- tention that will be explored in this chapter. Perhaps the most fundamental point about attention is its selectivity. Attention permits us to play an active role in our interaction with the world; we are not simply passive recipi- ents of stimuli. A great deal of the theoretical focus of re- search on attention has been concerned with how we come to select some information while ignoring the rest. Work in the years immediately following World War II led to the devel- opment of a theory that holds that information is ﬁltered at an early stage in perceptual processing (Broadbent, 1958). Ac- cording to this approach, there is a bottleneck in the sequence of processing stages involved in perception. Whereas physi- cal properties such as color or spatial position can be ex- tracted in parallel with no capacity limitations, further per- ceptual analysis (e.g., identiﬁcation) can be performed only on selected information. Thus, unattended stimuli, which are ﬁltered out as a result of attentional selection, are not fully perceived. Subsequent research was soon to call ﬁlter theory into question. One striking result comes from Moray’s (1959) studies using the dichotic listening paradigm, in which head- phones are used to present separate messages to the two ears. The subject is instructed to shadow one message (i.e., repeat it back as it is spoken); the other message is unattended. Ordinarily, there is little awareness of the contents of the unattended message (e.g., Cherry, 1953). However, Moray found that when a message in the unattended ear is preceded by the subject’s own name, the likelihood of report- ing the unattended message is increased. This suggests that CHAPTER 10 Attention HOWARD EGETH AND DOMINIQUE LAMY The preparation of this chapter was supported in part by grants to Howard Egeth from NIMH (R01MH57388) and the FAA (2001-G- 020). The authors would like to thank Alice Healy, Andy Leber, Melanie Palomares, Robert Proctor, Irving Weiner, and Steve Yantis for helpful comments, Robert Rauschenberger for preparation of the ﬁgures, and Terri Dannettel for technical help in the preparation of the manuscript. 270 Attention the unattended message had not been entirely excluded from further analysis. Treisman (1960) proposed a modiﬁcation of the ﬁlter model that was designed to handle this problem. She assumed the existence of a ﬁlter that attenuated the informa- tional content of an unattended input without eliminating it entirely. This model was capable of predicting the occasional intrusion of meaningful material from an unattended mes- sage. However, the discrepant data that led Treisman to pro- pose an attenuator instead of an all-or-none ﬁlter led other theorists in quite another direction. Thus, according to the late-selection view (e.g., Deutsch & Deutsch, 1963), percep- tual processing operates in parallel and selection occurs after perceptual processing is complete (e.g., after identiﬁcation), with capacity limitations arising only from later, response- related processes. After nearly three decades of intensive research and de- bate, recent reviews have suggested that the apparent contro- versy between the two views may stem from the fact that the empirical data in support of each of them has typically been drawn from different paradigms. For instance, Yantis and Johnston (1990) noted that evi- dence favoring the existence of late selection was typically obtained with divided-attention paradigms (e.g., Duncan, 1980; Miller, 1982). These ﬁndings showed only that there can be selection after identiﬁcation, rather than entailing that selection must occur after identiﬁcation. Yantis and Johnston (1990) set out to determine whether early selection is at all possible. By creating optimal conditions for the focusing of attention, they showed that subjects were able to ignore irrel- evant distractors, thus demonstrating the perfect selectivity that is diagnostic of early selection. Yantis and Johnston proposed a hybrid model with a ﬂexible locus for visual selection—namely, an early locus when the task involves ﬁl- tering out irrelevant objects, and a late locus, after identiﬁca- tion, when the task requires processing multiple objects. Kahneman and Treisman (1984) noted that whereas the early-selection approach initially gained the lion’s share of empirical support (e.g., von Wright, 1968), later studies pre- sented mounting evidence in favor of the late-selection view (e.g., Duncan, 1980). They attributed this dichotomy to a change in paradigm that took place in the ﬁeld of attention beginning in the late 1970s. Speciﬁcally, early studies used the ﬁltering paradigm, in which subjects are typically over- loaded with relevant and irrelevant stimuli and required to perform a complex task. Later studies used the selective-set paradigm, in which subjects are typically presented with few stimuli and required to perform a simple task. Thus, based on the observation that the conditions prevailing in the two types of study are very different, Kahneman and Treisman cau- tioned against any generalization across these paradigms. Lavie and Tsal (1994) elaborated on this idea by proposing that perceptual load may determine the locus of selection. They showed that early selection is possible only under con- ditions of high perceptual load (viz., when the task at hand is demanding or when the number of different objects in the display is large), whereas results typical of late selection are obtained under conditions of low perceptual load. In other words, when the task is not demanding, the spare capacity that is unused by that task is automatically diverted to the processing of irrelevant stimuli. The idea of a ﬁxed locus of selection (whether early or late) implies a distinction between a preattentive stage, in which all information receives a preliminary but superﬁcial analysis, followed by an attentive stage, in which only se- lected parts of the information receive further processing (Neisser, 1967). The preattentive stage has been further char- acterized as being automatic (i.e., triggered by external stim- ulation), spatially parallel, and unlimited in capacity, whereas the attentive stage is controlled (i.e., guided by the observer’s goals and intentions), spatially restricted to a limited region, and limited in capacity. Within this framework, an important question becomes, To what extent are stimuli processed dur- ing the preattentive stage? One implication of the proposed resolutions of the early- versus-late debate (Kahneman & Treisman, 1984; Lavie & Tsal, 1994; Yantis & Johnston, 1990) is that one cannot draw inferences from ﬁndings concerning the locus of selection to the question of how extensive preattentive processing is. That is, how efﬁcient selection can be and what is accomplished during the preattentive stage are separate issues. For instance, the idea of a ﬂexible locus of selection advanced by Yantis and Johnston (1990) implies that the level at which selection can be accomplished does not reveal intrinsic capacity limi- tations but depends only on task demands, and thus does not tell anything about preattentive processing. Similarly, the ﬁnding that perceptual load is a major determinant of selection efﬁciency (Lavie & Tsal, 1994) makes a useful methodological contribution, because it shows that a failure of selectivity does not reveal how extensively unattended ob- jects are processed, but may instead reﬂect the mandatory al- location of unused attentional resources to irrelevant objects. EFFICIENCY OF SELECTION Failures of Selectivity Various factors affect the efﬁciency of attentional selection. As was mentioned earlier, Lavie and Tsal (1994) proposed that low perceptual load may impair selectivity because spare Efﬁciency of Selection 271 attentional resources are automatically allocated to irrele- vant distractors. Similarly, grouping between target and dis- tractors may impair attentional selectivity. Another case of selectivity failure is evident in the ability of certain known- to-be-irrelevant stimuli to capture attention automatically. Effects of Grouping on Selection The principles of perceptual organization articulated by the Gestalt psychologists at the beginning of the last century (e.g., proximity, similarity, good continuation) correlate cer- tain stimulus characteristics with the tendency to perceive certain parts of the visual ﬁeld as belonging together—that is, as forming the same perceptual object. (For a fuller discus- sion of the Gestalt principles see the chapter by Palmer in this volume.) Kahneman and Henik (1981) considered the possi- bility that such grouping principles may impose strong con- straints on visual selection, with attention selecting whole objects rather than unparsed regions of space. Beginning in the early 1980s, this object-based view of selection has gained increased empirical support from a variety of experi- mental paradigms. Rock and Gutman (1981) showed object-speciﬁc atten- tional beneﬁts in an early study. Subjects were presented with a sequence of 10 stimuli, each of which consisted of two overlapping outline drawings of novel shapes, one drawn in red and one in green. Thus, in each of the overlapping pairs, the two shapes occupied essentially the same overall location in space. Subjects were required to make aesthetic judgments concerning only those stimuli in one speciﬁc color (e.g., the red stimuli). At the end of the sequence, they were given a surprise recognition test. Subjects were much more likely to report attended items (those about which they had rendered aesthetic judgments) as old than to report unattended items as old. In fact, unattended items were as likely to be recognized as were new items. This ﬁnding shows that attention can be directed to one of two spatially overlapping items. Note, however, that object- based selection was required by the task, which leaves open the possibility that object-based selection may not be manda- tory. Moreover, the fact that the unattended stimulus was not recognized does not necessarily entail that it was not perceived; in particular, it may have been forgotten during the interval between presentation and the recognition test. In a later article, Duncan (1984) explicitly laid out the dis- tinction between space-based and object-based views of at- tention and tested them with a perceptual version of the Rock and Gutman (1981) memory task. In Duncan’s study, object- based selection was no more task relevant than space-based selection. Subjects were presented with displays containing two objects: an outline box and a line that was struck through the box (see Figure 10.1). The box was either short or tall, and had a gap on either its left or right side. The line was dashed or dotted and was slanted either to the right or to the left. Subjects were found to judge two properties of the same object as readily as one property. However, there was a decre- ment in performance when they had to judge two properties belonging to two different objects. These results showed a difﬁculty in dividing attention between objects that could not be accounted for by spatial factors, because the objects were superimposed in the same spatial region. This very inﬂuential study has generated a whole body of research concerned with the issue of object-based selection, al- though it has been criticized by several authors (e.g., Baylis & Driver, 1993). Later studies where the problems associated with Duncan’s study were usually overcome also demon- strated a cost in dividing attention between two objects (e.g., Baylis & Driver, 1993; but see Davis, Driver, Pavani, & Shepherd, 2000, for a spatial interpretation of object-based effects obtained using divided attention tasks). Recently, Watson and Kramer (1999) added an important contribution to this line of research by attempting to specify a priori the stimulus characteristics that deﬁne the objects upon which selection takes place. They proposed a framework that allows one to predict whether object-based effects will be found, depending on stimulus characteristics. Borrowing from Palmer and Rock’s (1994) theory of perceptual organization, they distinguished among three hierarchically organized lev- els of representation: (a) single, uniformly connected (UC) regions, deﬁned as connected regions with uniform visual properties such as color or texture; (b) grouped-UC regions, which are larger representations made up of multiple single- UC regions grouped on the basis of Gestalt principles; and © parsed-UC regions, which are smaller representations seg- regated by parsing single-UC regions at points of concavity Figure 10.1 Two sample stimuli used in the study of object-based atten- tion. Each stimulus consisted of two objects (a box and a line passing through the box). See text for further details. Source: Reprinted from Duncan (1984), with permission from the American Psychological Association. 272 Attention (e.g., the pinched middle of an hourglass is such a region of concavity; it permits parsing the hourglass into its two main parts, the upper and lower chambers). They used complex familiar objects (pairs of wrenches) and had subjects identify whether one or two predeﬁned target properties were present in these objects (see Fig- ure 10.2). They examined under which conditions object- based effects (i.e., a performance cost for trials in which two targets belong to different wrenches rather than to the same wrench) could be obtained for each of the three representa- tional levels. They found that (a) object-based effects are ob- tained when the to-be-judged object parts belong to the same single-UC region, but not when they are separate single-UC regions, and concluded that the default level at which selec- tion occurs is the single-UC level; and (b) selection may occur at the grouped-UC level when it is beneﬁcial to per- forming the task or when this level has been primed. The ﬁnding that it is easier to divide attention between two properties when these belong to the same object suggests that perceptual organization affects the distribution of attention. Another empirical strategy used to reveal these effects is to show that subjects are unable to ignore distractors when these are grouped with the to-be-attended target (e.g., Banks & Prinzmetal, 1976). Other studies following this line of rea- soning used the Eriksen response competition paradigm (Eriksen & Hoffman, 1973), where the presence of distractors ﬂanking the target and associated with the wrong response is shown to slow choice reaction to the target (see the chapter by Proctor and Vu in this volume). They demonstrated that dis- tractors grouped with the target (e.g., by common color or contour) slow response more than do distractors that are not grouped with it, even when target-distractor distance is the same in the two conditions (Kramer & Jacobson, 1991). Perhaps the strongest support for the idea that attention se- lects perceptual groups rather than unparsed locations was provided by Egly, Driver, and Rafal’s (1994) spatial cueing study. Subjects had to detect a luminance change at one of the four ends of two outline rectangles (see Figure 10.3). One end was precued. On valid-cue trials, the target appeared at the cued end of the cued rectangle, whereas on invalid-cue trials, it appeared either at the uncued end of the cued rectan- gle, or in the uncued rectangle. The distance between the cued location and the location where the target appeared was identical in both invalid-cue conditions. On invalid-cue trials, targets were detected faster when they belonged to the same object as the cue, rather than to the other object. Several replications were reported, with detection (e.g., Lamy & Tsal, 2000; Vecera, 1994) as well as identiﬁcation tasks (e.g., Lamy & Egeth, 2002; Moore, Yantis, & Vaughan, 1998). Although some individual studies have been criticized or proved difﬁcult to replicate and limiting conditions for object-based selection have been identiﬁed (Lamy & Egeth, in press; Watson & Kramer, 1999), the overall picture that emerges from this selective review is that the segmentation of Figure 10.2 Sample stimuli and results from Experiment 1 of Watson and Kramer (1999). Each wrench in the two upper panels is homogeneously colored, and thus, according to Palmer and Rock (1994), may be character- ized as a single uniformly connected (UC) region. The wrenches in the two lower panels, having stippled handles between solid black ends, each consist of multiple (i.e., three) UC regions. Subjects searched the display for the presence of two targets: an open end (shown as the upper right end in each panel), and a bent end (shown on the upper left end on the different-wrench examples, and the lower right end of the same-wrench examples). Mean reaction-time differences are shown on the right of the ﬁgure. They show a same-object effect for the single-UC wrenches, but not for the wrenches composed of multiple UC regions. Source: Reprinted from Watson and Kramer (1999), with permission of the Psychonomic Society. Stimulus Examples Same Object Benefit Different - Same Wrench Reaction Time (RT) Different Wrench Same Wrench 66 msec. RT (msec.)  4 msec.  5 15355575 UC Regions Single Multiple Efﬁciency of Selection 273 the visual ﬁeld into perceptual groups imposes constraints on attentional selection. It is important to note, however, that this conclusion does not necessarily imply that grouping processes are preattentive. Indeed, in all the studies surveyed above, at least one part of the relevant object (i.e., of the per- ceptual group for which object-based effects were measured) was attended. As a result, one may conceive of the possibility that attending to an object part causes other parts of this ob- ject to be attended. For this reason, a safer avenue to investigate whether grouping requires attention may be to measure grouping ef- fects when the relevant perceptual group lies entirely outside the focus of attention. The studies pertaining to this issue will be discussed in the section on “Preattentive and Attentive Processing.” Capture of Attention by Irrelevant Stimuli Goal-directed or top-down control of attention refers to the ability of the observer’s goals or intentions to determine which regions, attributes, or objects will be selected for further visual processing. Most current models of attention assume that top-down selectivity is modulated by stimulus- driven (or bottom-up ) factors, and that certain stimulus prop- erties are able to attract attention in spite of the observer’s effort to ignore them. Several models, such as the guided search model of Cave and Wolfe (1990), posit that an item’s overall level of attentional priority is the sum of its bottom-up activation level and its top-down activation level. Bottom-up activation is a measure of how different an item is from its neighbors. Top-down activation (Cave & Wolfe, 1990) or inhibition (Treisman & Sato, 1990) depends on the degree of match between an item and the set of target properties speci- ﬁed by task demands. However, the relative weight allocated to each factor and the mechanisms responsible for this allo- cation are left largely unspeciﬁed. Curiously enough, no par- ticular effort has been made to isolate the effects on visual search of bottom-up and top-down factors, which were typi- cally confounded in the experiments held to support these theories (see Lamy & Tsal, 1999, for a detailed discussion). For instance, the fact that search for feature singletons is efﬁ- cient has been demonstrated repeatedly (e.g., Egeth, Jonides, & Wall, 1972; Treisman & Gelade, 1980) and has been termed pop-out search (or parallel feature search ). It is often assumed that this phenomenon reﬂects automatic capture of attention by the feature singleton. However, in typical pop- out search experiments, the singleton target is both task rele- vant and unique. Thus, it is not possible to determine in these studies whether efﬁcient search stems from top-down factors, bottom-up factors, or both (see Yantis & Egeth, 1999). Recently, new paradigms have been designed that allow one to disentangle bottom-up and top-down effects more rig- orously. The general approach has been to determine the ex- tent to which top-down factors may modulate the ability of an irrelevant salient item to capture attention. Discontinuities, such as uniqueness on some dimension (e.g., color, shape, orientation) or abrupt changes in luminance, are typically used as the operational deﬁnition of bottom-up factors or stimulus salience . Based on the evidence that has accumu- lated in the last decade or so, two opposed theoretical pro- posals have emerged. Some authors have suggested that preattentive processing is driven exclusively by bottom-up factors such as salience, with a role for top-down factors only later in processing (e.g., M. S. Kim & Cave, 1999; Theeuwes, Atchley, & Kramer, 2000). Others have proposed that atten- tional allocation is always ultimately contingent on top- down attentional settings (e.g., Bacon & Egeth, 1994; Folk, Remington, & Johnston, 1992). A somewhat intermediate viewpoint is that pure, stimulus-driven capture of attention is produced only by the abrupt onset of new objects, whereas other salient stimulus properties do not summon attention when they are known to be irrelevant (e.g., Jonides & Yantis, 1988). Several sets of ﬁndings have shaped the current state of the literature on how bottom-up and top-down factors af- fect attentional priority. Beginning in the early 1990s, Theeuwes (e.g., 1991, 1992; Theeuwes et al., 2000) carried out several experiments sug- gesting that attention is captured by the element with the highest bottom-up salience in the display, regardless of whether this element’s salient property is task relevant. Cap- ture was measured as slower performance in parallel search Figure 10.3 Examples of typical sequences of events in Experiments 1 and 2 of the study by Egly, Driver, and Rafal (1994). The white lines in the cue display represent the cue. The ﬁlled end of a bar represents the target. The target for the valid trial is in the same spatial location (upper right) as the cue. There are two types of invalid trials. In one, the target is the on the same bar as the cue, but at the opposite end, and thus requires a within-object shift of attention from the preceding cue. In the other, the target is on the uncued bar; this target requires a between-objects switch of attention from the cue. Note that the distance between the target and the cue is equal in the two types of invalid trial. different object same object or FIXATION CUE ISI TARGET (invalid) TARGET (valid)     274 Attention even when the target’s unique feature value was known (see Pashler, 1988a, for an earlier report of this effect). Theeuwes concluded that when subjects are engaged in a parallel search, perfect top-down selectivity based on stimulus fea- tures (e.g., red or green) or stimulus dimensions (e.g., shape or color) is not possible. Bacon and Egeth (1994) questioned this conclusion. Using a distinction initially suggested by Pashler (1988a), they proposed that in Theeuwes’s (1992) experiment, two search strategies were available: (a) singleton detection mode, in which attention is directed to the location with the largest local feature contrast, and (b) feature search mode, which en- tails directing attention to items possessing the target visual feature. Indeed, the target was deﬁned as being a singleton and as possessing the target attribute. If subjects used single- ton detection mode, both relevant and irrelevant singletons could capture attention, depending on which exhibited the greatest local feature contrast. To test this hypothesis, Bacon and Egeth (1994) designed conditions in which singleton de- tection mode was inappropriate for performing the task. As a result, the disruption caused by the unique distractor dis- appeared. They concluded that irrelevant singletons may or may not cause distraction during parallel search for a known target, depending on the search strategy employed. Another set of experiments revealed that abrupt onsets do produce involuntary attentional capture (Hillstrom & Yantis, 1994; Jonides & Yantis, 1988), whereas feature singletons on dimensions such as color and motion do not (e.g., Jonides & Yantis, 1988). These authors concluded that (a) abrupt onsets are unique in their ability to summon attention to their loca- tion automatically, and (b) feature singletons do not capture attention when they are task irrelevant. The idea that the ability of a salient stimulus to cap- ture attention depends on top-down settings—speciﬁcally, on whether subjects use singleton detection mode or feature search mode—is consistent with the contingent attentional cap- ture hypothesis (e.g., Folk et al., 1992).According to this theory, attentional capture is ultimately contingent on whether a salient stimulus property is consistent with top-down attentional control settings. The settings are assumed to reﬂect current be- havioral goals determined by the task to be performed. Once the attentional system has been conﬁgured with appropriate control settings, a stimulus property that matches the settings will produce “on-line” involuntary capture to its location. Stimuli that do not match the top-down attention settings will not cap- ture attention. Folk et al. (1992) provided support for this claim using a novel spatial cuing paradigm. In Experiment 3, for instance, subjects saw a cue display followed by a target display (see Figure 10.5). They were required to decide whether the target Figure 10.4 Sample stimuli from the studies of Theeuwes (1991, 1992). The subject always searched for a green circle among green diamonds (two left panels; form condition), or among red circles (two right panels; color condition), either without a distractor (top panels), or with a distractor (bot- tom panels). The line segment within the target element was horizontal or vertical (subjects had to indicate which); the line segments in the other forms were tilted 22.5 deg from horizontal or vertical. Source: Reprinted from Theeuwes (1992), with permission of the Psychonomic Society. form color green red when an irrelevant salient object was present. For instance, Theeuwes (1991, 1992) presented subjects with displays con- sisting of varying numbers of colored circles and diamonds arranged on the circumference of an imaginary circle (see Figure 10.4). A line segment varying in orientation appeared inside each item, and subjects were required to determine the orientation of the line segment within a target item. In one condition, the target item was deﬁned by its unique form (e.g., it was the single green diamond among green circles). In another condition, it was deﬁned as the color singleton (e.g., it was the single red square among green squares). On half of the trials, an irrelevant distractor unique on an irrele- vant dimension might be present. For instance, when the target item was a green diamond among green circles, a red circle was present. Theeuwes (1991) found that the presence of the irrelevant singleton slowed reaction times (RTs) signif- icantly. However, this effect occurred only when the irrele- vant singleton was more salient than the singleton target, suggesting that items are selected by order of salience. In a later study, Theeuwes (1992) reported distraction effects Efﬁciency of Selection 275 Figure 10.5 Sample cue displays and target displays used to investigate contingent attentional capture. In these examples, the cues appear in the left- hand location and the targets in the right-hand location (thus any trials com- posed from these particular components would be considered invalid trials). See text for further details. Source: Reprinted from Folk, Remington, and Johnston (1992), with permission of the American Psychological Association. Color cue Onset cue Color target Onset target was an xor an “ = ” sign. The ta r get was deﬁned either as a color singleton target (e.g., the single red item among white items) or as an onset target (i.e., a unique abruptly onset item in the display). Two types of distractors were used. A color distractor consisted of four colored dots surrounding a poten- tial target location, and arrays of white dots surrounded the remaining three locations. An onset distractor consisted of a unique array of four white dots surrounding one of the poten- tial target locations. The two distractor types were factorially combined with the two target types, with each combination presented in a separate block. The locations of the distractor and target were uncorrelated. The authors reasoned that if a distractor were to capture attention, a target sharing its loca- tion would be identiﬁed more rapidly than a target appearing at a different location. Thus, they measured capture as the difference in performance between conditions in which dis- tractors appeared at the target location versus nontarget loca- tions. The question was whether capture would depend on the match between the salient property of the distractor and the property deﬁning the target. The results showed that it did: Whereas capture was found when the distractor and target shared the same property, virtually no capture was observed when they were deﬁned by different properties. The foregoing discussion of attentional capture suggests that the conditions under which involuntary capture occurs remain controversial. Studies that reached incompatible con- clusions usually presented numerous procedural differences. For instance, Folk (e.g., Folk et al., 1992) and Yantis (e.g., Yantis, 1993) disagree on what status should be assigned to new (or abruptly onset) objects. Yantis claims that abrupt on- sets capture attention irrespective of the observer’s inten- tions, whereas Folk argues that involuntary capture by abrupt onsets happens only when subjects are set to look for onset targets. Note, however, that Yantis’s experiments typically in- volved a difﬁcult search, for instance, one in which the target was a speciﬁc letter among distracting letters (e.g., Yantis & Jonides, 1990) or a line differing only slightly in orientation from surrounding distractors (e.g., Yantis & Egeth, 1999). In contrast, Folk’s subjects typically searched for, say, a red tar- get among white distractors—that is, for a target that sharply differed from the distractors on a simple dimension (e.g., Folk et al., 1992). Thus, the two groups of studies differed as to how much top-down guidance was available to ﬁnd the tar- get. This factor may possibly account for the better selectiv- ity obtained in Folk’s studies. Further research is needed to settle this issue. The main point of agreement seems to be that an irrelevant feature singleton will not capture attentionautomatically when the task does not involve searching for a singleton target. This ﬁnding has been obtained using three different paradigms, under which attentional capture was gauged using different measures: a difference between distractor-present versus dis- tractor-absent trials (Bacon & Egeth, 1994); a difference be- tween trials in which the target and cue occupy the same versus different locations in spatial cueing tasks (e.g., Folk et al., 1992); and the difference between trials in which the target and salient item do versus do not coincide (e.g., Yantis & Egeth, 1999). Although most of the evidence provided by Theeuwes (e.g., 1992) for automatic capture was drawn from studies in which the target was a singleton, his position on whether cap- ture occurs when the target is not a singleton is not entirely clear (see, e.g., Theeuwes & Burger, 1998). Note, however, that in the current state of the literature, the implied distinction between singleton detection mode, in which any salient distractor will capture attention, and feature search mode, in which only singletons sharing a task-relevant feature will capture attention, suffers from two problems. First, it is based on the yet-untested assumption that the singleton detection mode of processing is faster or less cogni- tively demanding than is the feature search mode. Indeed, one observes that subjects will use the feature search mode only if the singleton detection mode is not an option. For instance, when the strategy of searching for the odd one out is not available (e.g., Bacon & Egeth, 1994, Experiments 2 & 3), 276 Attention an irrelevant singleton does not capture attention. However, the same irrelevant singleton does capture attention when subjects search for a singleton target with a known feature (e.g., Bacon & Egeth, 1994, Experiment 1). Capture by the ir- relevant singleton occurs despite the fact that using the sin- gleton detection mode will tend to guide attention ﬁrst toward a salient nontarget on 50% of the trials (or even on 100% of the trials; see M. S. Kim & Cave, 1999), whereas using the feature search mode will tend to guide attention directly to the target on 100% of the trials. The intuitive explanation for the fact that subjects use a strategy that is nominally less efﬁ- cient is that the singleton-detection processing mode itself must be structurally more efﬁcient. Yet, no study to date has put this assumption to test. Second, in studies in which subjects must look for a unique target with a known feature, there is often an element of circularity in inferring from the data which processing mode subjects use. Indeed, if an irrelevant singleton captures attention, then the conclusion is that subjects used the single- ton detection mode. If, in contrast, no capture is observed, the conclusion is that they used the feature search mode. However, the factors that induce subjects to use one mode rather than the other when both modes are available remain unspeciﬁed. Selection by Location and Other Features The foregoing section was concerned with factors that limit selectivity. Next, we turn to a description of the mechanisms underlying the different ways by which attention can be di- rected toward to-be-selected or relevant areas or objects. Selection by Location “Attention is quite independent of the position and accom- modation of the eyes, and of any known alteration in these organs; and free to direct itself by a conscious and voluntary effort upon any selected portion of a dark and undifferenced ﬁeld of view” (von Helmholtz, 1871, p. 741, quoted by James, 1890/1950, p. 438). Since this initial observation was made, a large body of research has investigated people’s abil- ity to shift the locus of their attention to extra-foveal loci without moving their eyes (e.g., Posner, Snyder, & Davidson, 1980), a process called covert visual orienting (Posner, 1980). Covert visual orienting may be controlled in one of two ways, one involving peripheral (or exogenous) cues, and the other, central (or endogenous) cues. Peripheral cues tradi- tionally involve abrupt changes in luminance—usually, abrupt object onsets, which on a certain proportion of the trials appear at or near the location of the to-be-judged target. With central cues, knowledge of the target’s location is pro- vided symbolically, typically in the center of the display (e.g., an arrow pointing to the target location). Numerous experi- ments have shown that detection and discrimination of a tar- get displayed shortly after the cue is improved more on valid trials—that is, when this target appears at the

bundet

same location as the cue (peripheral cues) or at the location speciﬁed by the cue (central cues)—than on invalid trials, in which the target appears at a different location. Some studies also include neu- tral trials or no-cue trials, in which none of the potential tar- get locations is primed (but see Jonides & Mack, 1984, for problems associated with the choice of neutral cues). Neutral trials typically yield intermediate levels of performance. Pe- ripheral and central cues have been compared along two main avenues. Some studies have focused on differences in the way at- tention is oriented by each type of cue. The results from this line of research have suggested that peripheral cues capture attention automatically (but see the earlier section, “Capture of Attention by Irrelevant Stimuli,” for a discussion of this issue), whereas attentional orienting following a central cue is voluntary (e.g., Müller & Rabbitt, 1989; Nakayama & Mackeben, 1989). Moreover, attentional orienting to the cued location was found to be faster with peripheral cues than with central cues. For instance, in Muller and Rabbitt’s (1989) stud y , subjects had to ﬁnd a ta r get ( T ) among distractors (+ ) in one of four boxes located around ﬁxation. The central cue was an arrow at ﬁxation, pointing to one of the four boxes. The peripheral cue was a brief increase in the bright- ness of one of the boxes. With peripheral cues, costs and beneﬁts grew rapidly and reached their peak magnitudes at cue-to-target onset asynchronies (SOAs) in the range of 100– 150 ms. With central cues, maximum costs and beneﬁts were obtained for SOAs of 200–400 ms. Other studies have focused on differences in information processing that occur as a consequence of the allocation of at- tention by peripheral versus central cues. Two broad classes of mechanisms have been proposed to describe the effects of spatial cues. According to the signal enhancement hypothesis (e.g., Henderson, 1996), attention strengthens the stimulus representation by allocating the limited capacity available for perceptual processing. In other words, attention facilitates perceptual processing at the cued location. According to the uncertainty or noise reduction hypothesis (e.g., Palmer, Ames, & Lindsay, 1993) spatial cues allow one to exclude distractors from processing by monitoring only the relevant location rather than all possible ones. Thus, cueing attention to a speciﬁc location reduces statistical uncertainty or noise effects, which stem from information loss and decision Efﬁciency of Selection 277 limits, not from changes in perceptual sensitivity or limits of information-processing capacity. In order to test the two hypotheses against each other, sev- eral investigators have sought to determine whether spatial cueing effects would be observed when the target appears in an otherwise empty ﬁeld. The signal enhancement hypothesis predicts such effects, as the allocation of attentional resources at the cued location should facilitate perceptual processing at that location, even in the absence of noise. In contrast, the noise reduction hypothesis predicts no cueing effects with sin- gle-element displays, because no spatial uncertainty or noise reduction should be required in the absence of distractors. This line of research has generated conﬂicting ﬁndings, with reports of small effects (Posner, 1980), signiﬁcant effects (e.g., Henderson, 1991) or no effect (e.g., Shiu & Pashler, 1994). Relatively subtle methodological differences have turned out to play a crucial role. For instance, Shiu and Pashler (1994) criticized earlier single-target studies (Henderson, 1991) on the grounds that the masks presented at each potential location after the target display may have been confusable with the target, thus making the precue useful in reducing the noise associated with the masks. They compared a condition in which masks were presented at all potential lo- cations vs. a condition with a single mask at the target loca- tion. Precue effects were found only in the former condition, supporting the idea that these reﬂect noise reduction rather than perceptual enhancement. However, recent evidence showed that spatial cueing effects can be found with a single target and mask, and are larger with additional distractors or masks. These ﬁndings suggest that attentional allocation by spatial precues leads both to signal enhancement at the cued location and noise reduction (e.g., Cheal & Gregory, 1997; Henderson, 1996). Most of the reviewed studies employed informative pe- ripheral cues, which precludes the possibility of determining whether the observed effects of attentional facilitation should be attributed to the exogenous or to the endogenous component of attentional allocation, or to both. Studies that employed non-informative peripheral cues (Henderson, 1996; Luck & Thomas, 1999) showed that these lead to both perceptual enhancement and noise reduction. Recently, Lu and Dosher (2000) directly compared the effects of periph- eral and central cues and reported results suggesting a noise reduction mechanism of central precueing and a combination of noise reduction and signal enhancement for peripheral cueing. To conclude, the current literature points to notable differ- ences in the way attention is oriented by peripheral vs. central cues, as well as differences in information processing when attention is directed by one type of spatial cue vs. the other. Is Location Special? The idea that location may deserve a special status in the study of attention has generated a considerable amount of re- search, and the origins of this debate can be traced back to the notion that attention operates as a spotlight (e.g., Broadbent, 1982; Eriksen & Hoffman, 1973; Posner et al., 1980), which has had a major inﬂuence on attention research. According to this model, attention can be directed only to a small contigu- ous region of the visual ﬁeld. Stimuli that fall within that region are extensively processed, whereas stimuli located outside that region are ignored. Thus, the spotlight model— as well as models based on similar metaphors, such as zoom lenses (e.g., Eriksen & Yeh, 1985) and gradients (e.g., Downing & Pinker, 1985; LaBerge & Brown, 1989)— endows location (or space) with a central role in the selection process. Later theories making assumptions that markedly depart from spotlight theories also assume an important role for location in visual attention (see Schneider, 1993 for a review). These include for instance Feature Integration Theory (Treisman & Gelade, 1980), the Guided Search model (Cave & Wolfe, 1990; Wolfe, 1994), van der Heidjen’s model (1992, 1993), and the FeatureGate model (Cave, 1999). A comprehensive survey of the debate on whether or not location is special is beyond the scope of the present en- deavor (see for instance, Cave & Bichot, 1999; Lamy & Tsal, 2001, for reviews of this issue). Here, two aspects of this debate will be touched on, which pertain to the efﬁciency of selection. First, we shall brieﬂy review the studies in which selectivity using spatial vs. non-spatial cues is compared. Then, the idea that selection is always ultimately mediated by space, which entails that selection by location is intrinsically more direct, will be contrasted with the notion that attention selects space-invariant object-based representations. Selection by Features Other Than Location. Numer- ous studies have shown that advance knowledge about a non-spatial property of an upcoming target can improve per- formance (e.g., Carter, 1982). Results arguing against the idea that attention can be guided by properties other than lo- cation are typically open to alternative explanations (see Lamy & Tsal, 2001, for a review). For instance, Theeuwes (1989) presented subjects with two shapes that appeared si- multaneously on each side of ﬁxation. The target was deﬁned as the shape containing a line segment, whereas the distractor was the empty shape. Subjects responded to the line’s orien- tation. The target was cued by the form of the shape within which it appeared, or by its location. Validity effects were obtained with the location cue but not with the form cue. The author concluded that advance knowledge of form