Introduction

Most of the scientific research on the perception of material qualities of objects has focused on the perception of texture. By texture, we mean the microstructure of surfaces as opposed to the large-scale macrostructure of objects (e.g., form, shape). The surface texture is independent of the macrostructure on which it is superimposed. The material from which objects are constructed also possesses other prominent properties, such as softness or hardness, slipperiness and friction, and thermal qualities such as thermal flow and thermal conductivity. The general issues we raise in this chapter with respect to texture perception pertain as well to these other properties of object materials. However, because there has been relatively little scientific research on these topics to date, we will consider them at the end of the chapter.

The perception of surface texture is a multidimensional task; however, the most salient dimensions remain somewhat uncertain (see Hollins, Faldowski, Rao, & Young, 1993; Hollins & Risner, 2000). When we describe the texture of a surface, we may focus on dimensions such as roughness/smoothness, bumpiness, or jaggedness, or perhaps on the degree of element cluster that produces the surface microstructure. The perception of surface texture is also multisensory. That is, regardless of the properties to which we attend, we may use any or all of the following modalities during perception: haptics, vision, and audition. (The haptic system uses sensory inputs from mechanoreceptors in the skin, muscles, tendons, and joints, and typically involves voluntary manual exploration of surfaces, objects, and their spatial layout.)

The availability of more than one sensory source may serve a number of different functions. First, when the information obtained via different sensory modalities about a given property is exactly the same, the modalities will provide redundant cues about the targeted property. Second, even if the sensory sources provide concordant information, one modality may provide more accurate or more precise information than another. Alternately or in addition, it may obtain that information faster than another modality. Third, the modalities may actually provide discrepant information about a targeted property. Fourth, the senses may simultaneously provide qualitatively different yet complementary information about a texture. For example, vision provides information about surface properties such as color, lightness, and reflectance, whereas touch provides information about roughness, hardness, slipperiness, and thermal properties.

The following discussion considers relevant empirical findings, theoretical perspectives, and the implications of such fundamental scientific work for the design of multisensory interfaces for teleoperation and virtual environments.