|
|
With the emergence of relatively cheap multimedia delivery systems incorporating bitmapped graphics and high fidelity continuous audio and video, there is an increasing need for the user interface designer to be informed in their choice of output media for a particular task. Our research is investigating the affect of different media on the formulation of mental models in the solution of complex tasks. The guiding principle is to understand why one medium has an advantage over another in a given task context. To do this one must examine how these media are utilised by the user. An experiment is outlined which will investigate this.
Multimedia, media selection, mental models, expressiveness, tractability, complex tasks
It is a truism that any interaction with a computer relies on the mediation of the machine's internal state to the user in a way which suggests its use. This begs the question, 'how do we represent this state to the user?' Multimedia technologies such as continuous or discrete digital audio and video, bitmapped graphics and high speed animation have greatly increased the display options available. However, in doing so the interface designer's chances of producing an inadequate interface are greatly increased. To attempt to remedy this situation research in a variety of fields from social science to computer science has identified a number of factors which inform this media choice.
The solution being investigated by this research stems from the phenomenology of mental representations users form of a task domain by virtue of the conveying media.
Work by Stenning and Oberlander [5] investigated the strengths of different representation systems, or media, for conveying the same problem. In particular, the empirically proven advantages of graphical over linguistic representations to aid teaching of syllogistic reasoning problems.
From this discussion the notion of expressiveness is defined for a representation system. This describes the extent to which a single instance of a medium (a sentence of the representation system) can present more than one model of the domain. The more abstraction provided by the medium, the more expressive it is. This quality is then used to explain how tasks become more solvable if the number of domain states that must be considered is reduced by using an expressive medium. The medium effectively collapses the domain state space allowing a manageable state set to be open to inspection. Stenning and Oberlander postulate the ideal representation system will present the task domain at a level congruent with the difficulty of the task, viz. expressive enough for the task.
Stenning and Oberlander also offer a subsumption of expressiveness into constraints imposed on a representation system by itUs syntax and/or semantics. These constraints limit the way in which instances of a medium can by interpreted as views of the domain. For example, if only those representations whose syntax/semantics dictate no representation could be interpreted in a way which would violate the domain, then the system is self-consistent. In other words, there is only one way an instance can be interpreted and consequently the system is isomorphic, one representaion instance portrays one domain instance. This is the lowest level of expressiveness a system can have. Thus, the fewer constraints which are imposed the more expressive the medium.
However, it is not enough to consider the fine grain detail of a medium to find its expressiveness, one must also examine its representation at a global level. It is here that we see emergent properties of the medium which are not obvious from a piecemeal discussion of syntactic or semantic constraints. An example is a scatterplot. According to StenningUs self-consistency constraint, a scatter plot is fully isomorphic. But when one considers a collection of points the inexperienced eye may see trends in the data. The raw data has been aggregated or abstracted making the problem space more tractable because the scatterplot shows a number of domain states in a single representating instance.
The key point is that all these qualities are inherent in the makeup of the medium, it is simply a case of identifying which task will best utilise these constraints and become more tractable.
In another illustration of how expressive media make problems tractable, one can consider the differences between novice and expert behaviour in a complex control domain. Moray [3] suggests the main difference between experienced and inexperienced operators will be the exhibition of open-loop behaviour which is based on in-depth knowledge of the systemUs characteristics i.e. a rich mental model. The behaviour verges on the pathological since there is no need to explore all the solution state transitions thanks to heuristic knowledge. The naive operator, on the other hand, will exhibit closed-loop behaviour attempting to blindly visit all of the necessary system states. i.e. using a simplistic mental model
However, if a medium were provided which abstracted across the disperate states of the complex task, naive subjects would be able to approach the skill of experts. This phenomenon was shown by Grossen and Carnine [2] who raised the expertise of learning impaired students to those of gifted students using an expressive graphical teaching aid. Conversely, simple tasks do not require any collapsing of the solution state space so naive operators can perform adequately.
This results suggests what intuition would corroborate, that the mental representations formed of a problem depend on how the problem is conveyed. However, it is the concept of congruence between medium expressiveness and task complexity which we think is the reason for this and which is under investigation.
Given the connection between the complexity of a task domain, its representation and the internalisation of this representation, we are now in the process of designing an experiment to investigate this interaction. The experiment will concentrate on:
1) Attempting to identify expressiveness as a constant quality of any given medium (or family of media).
2) Investigating the effects of a mismatch between medium expressiveness and task complexity. This is alluded to by Bransford et al [1] who noted subjects struggled with simple problems (which would require less expressive representations) being presented by very expressive media.
A complex domain has been chosen to provide the necessarily rich and controllable complexity of task. It is based on the Crossman Waterbath [4] a real-time problem which exhibits multi-dimensional relationships. Instead of controlling water flow by valves, subjects must control vehicular traffic flow by adjusting the green and red time of traffic lights.
To represent this domain a number of auditory and visual media are provided .e.g. dynamic graph, table, bar chart, schematic animation, still video, continuous tone, real-sound . When solving tasks a keystroke log , recording of subject verbalisations, record of correct answers and the number of simulation steps to completion is taken. These wide range of sources are required to provide cross-referencing of data.
For each of three complexity levels subjects will answer a range of problems which will move from easy to difficult with a variety of media. Media will be initially grouped into expressiveness categories allowing the investigation of interaction between complexity and expressiveness.
To examine the subjects mental models the method exemplified by Verhage [6] will be adopted. This begins with an a priori description of the domain in terms of relationships between inputs, state variables and outputs. A similar model is constructed from the latter results of each subject and compared with the optimum. It will be evident where subtle system characteristics are identified or where a simplistic view of the system is used.
The experiment will give an indication of task/media interactions in terms of representational congruence. It will also give designers the kernal of a fresh, 'mental representation based' description of media, irrespective of the task. Further work will investigate expressiveness of media when used in other types of task.
1. Bransford, J. D., J. R. Barklay and J. J. Franks Sentence Memory: A constructive versus interpretive approach, Cognitive Psychology, Vol. 3, pp 193-209, 1973
2. Grossen , G. and D. Carnine Diagramming a Logic Strategy: Effects on difficult problem types and transfer, Learning Disibility Quarterly, Vol. 3, pp 168-182, 1990
3. Moray, N. Aquisition of Process Control Skills, IEEE Trans. on Man, Mach. & Cyber, Vol. 16(4), pp 497-504, 1986
4. Sanderson, P. M and A. G. Verhage. State Space and Verbal Protocols for Studying the Human Operator in Process Control, Ergonomics, Vol. 32 (11), pp 1343-1372, 1989
5. Stenning, K., J. Oberlander A Cognitive Theory of Graphical and Linguistic Reasoning: Logic and Implementation, Cognitive Science, Vol. 19(1), pp 97-140, 1995
6. Verhage, A. G. Designing Visual Displays to Enhance Operator Knowledge in a Process Control Simulation, Ph.D Thesis, EPRL, University of Illinois, 1991