User centred design
User Models & Mapping
Aiding the User
Usability Assessment & Method of Evaluation
Shneiderman  suggests that
the concept of a "participatory design strategy" where
the user of a system is involved, iteratively, in its design and
evaluation is a controversial one. It seems obvious that involving
the user of a potential system in its development can only bring
advantages. As noted by Baroudi cited in Shneiderman  "user
involvement brings more information about tasks, an opportunity
to argue over design decisions... and the potential for increased
user acceptance of the final system". Indeed Keen cited in
Shneiderman  recognised the problem of "counterimplementation"
(basically a reluctance to welcome and adopt a system) and suggests
that early participation of users will enable their concerns to
be heard and resolved.
Basically the user should know what
is wanted from the system so why should the designer try to second
guess what he wants if he can just ask him in the first place?
Norman  considers user centred
design to be "a philosophy based on the needs and interests
of the user, with an emphasis on making products usable and understandable".
In this chapter I will detail the theory and practical implementation
of this user centred design philosophy, giving examples where
necessary to illustrate and aid my arguments.
Interaction between the user and
a system can be deemed to be the communication required to complete
tasks. Grandjean  suggests that these "points of interchange
from man to machine and from machine to man -interfaces-
are of paramount importance". I will, in this section, describe
why the interface has such an important role to play in interaction
and will also discuss various models of interaction that endeavour
to aid the designer of interactive systems.
K in which Grandjean  shows a simple "Man-machine
system". He explains that the cycle involves the user reacting
to the display control by operating the control of the machine
to effect the required change. The control instrument informs
the user of the result of his action "e.g. how much water
has been mixed in with reagents" and the display instrument
relays whether this action has been effective. In turn this information
will aid the user in deciding whether further action is needed.
Grandjean  proposes that in
the system "the man holds the key position because the decisions
rest with him". It can therefore be demonstrated that the
decisions he makes rests on the accuracy of the feedback he receives,
and his "perception [and] interpretation" of it, from
the interface (in this case the control and display instrument).
Therefore display instruments are
not just the end result of an action but the impetus for a new
one to be made. The accuracy of the perceived and interpreted
information is paramount and there has been much research into
the effectiveness of display instrumentation to achieve this.
Depending on the context of its role, a display instrument may
prove suitable for one task but not for another.
Grandjean  considers this
in terms of two types of displays; firstly a digital counter and
secondly a moving pointer against a fixed scale. The counter display
is "very good" in terms of its "ease of reading"
but "poor" in the "detection of change". Alternatively
the pointer display is "acceptable" in terms of its
"ease of reading" but very good in the "detection
Thus there is need to consider appropriate
display instrumentation when designing systems. The designer will
need to take into account which is the most significant requirement
of a system's design before choosing which type of display to
of Execution & Evaluation
When a user comes up against an
interface, he may know what he ultimately wants from the system,
such as moving a car, but he may "not know which physical
variables to adjust, or in what way to adjust them" [Norman
cited in Booth 1992], i.e. he may not know how to drive. When
I started to learn to drive I just assumed actually moving a car
was just an extension of natural movement (which it is once you
can drive!), every other driver seemed to be able to move the
car automatically without thinking.
Hutchins et al cited in Preece
et al  developed a framework which describes a "distance
between the user's goals and the means of achieving them through
the system". This gap is termed the "gulf of execution"
[Norman 1988 and Hutchins et al cited in Preece et al
1994]. Dix et al  recommend that "the interface
should therefore aim to reduce this gulf".
Preece et al  suggest
that this should be done by "designing the input characteristics
to match the users' psychological capabilities", effectively
mapped in other words (mapping is considered in User Models & Mapping) improving guessability.
Similarly the user needs to be able
to "interpret the physical state of the system and to determine
how well [their] expectations and intentions have been met"
[Norman 1988]. The amount of effort required to do this reflects
the "gulf of evaluation" according to Norman .
Preece et al  suggest
that by "changing the output characteristics of the system"
this gulf can be reduced. This change may take the form of more
effective or appropriate displays (or perhaps the introduction
of them) or quicker feedback generally. The windows hour glass
timer, for instance, sometimes indicates that the system is currently
processing when it may have actually crashed.
The greater the size of the gulfs
the harder it will be for the user to carry out the task. The
onus should be on the designer, rather than the user to "bridge"
both gulfs to achieve the smallest possible distances between
the physical system and the goal.
General Framework for Interaction
Dix et al  suggest
that Norman's  model "concentrates wholly on the user's
view of the system ... and ... does not attempt to deal with the
systems communication through the interface". As a result
Abowd and Beale  expand on this model in order to provide
a "general interaction framework which will allow analysis".
I only cite their paper in order to reinforce my views on the
way in which users see the interface and to highlight the similarity
between both models.
Abowd and Beale's  unifying
framework for interaction is shown at figure
L where the oval in the centre of the diagram represents the
interface, S represents the system and U the user. The "systems
language" is referred to as "core" whilst the "user's
language" is referred to as the "task". Despite
the model having 4 "translation steps", to explicitly
involve the system's own communication, it can be seen that progress
from U to I to S still reflects the gulf of execution and completion
of the interaction cycle from S to O to U can be considered to
represent the gulf of evaluation.
Between each node on the model,
the "translation steps" are "qualitative assessments"
on the quality of translation between the languages. Abowd and
Beale  refer to them as distances. "The most important
measure" of distance in the framework is the difference between
the "semantics of intention (goal formulation)" and
the "semantics of evaluation (goal assessment)" and
this is referred to as the semantic distance. The distance is
an assessment as to how effective the user can establish if his
actions (the undertaking of the task) has achieved the desired
result (the goal).
It can therefore be seen that effective
interface design will demonstrate the reduction of this semantic
distance so that it is "as small as possible" [Abowd
and Beale 1991]. This is in direct agreement with the notion that
"bridging the two gulfs [of execution and evaluation]",
as recommended by Preece et al , is paramount to
overcome the "mismatch between the user's way of thinking
about their tasks and the system's representation [of them]".
In this section I have detailed
what interaction is and how interaction models have given an understanding
to the cognitive processes that take place at the interface. The
models have stressed the importance of perception (User Perception) and feedback (Feedback) and how the quality of these elements can ultimately affect
the overall usability of an interactive system. This discussion
has identified the notion of mapping and it is this that I now
turn my attention to.
Models & Mapping
In this section I will initially
discus an area of user centred design that will focus on the user
himself in the form of a user model (not to be confused with the
cognitive model that a user has of a particular system). I will
analyse why the understanding of the user group is a vital ingredient
in the usability recipe. Secondly I will examine the theory of
mapping in which many designs exploit generic human methods of
movement and cultural understanding that can make a system or
device more natural to learn and to use.
Nobody would consider designing
a car that needed three arms to operate it because it would be
obviously absurd, after all we are all human and only have two
arms (baring accident of course). Modelling or profiling may at
first glance seem irrelevant for most systems but consider the
following example of a simple task.
Whilst on holiday in Spain recently
I attempted to make a telephone call back to the UK from a public
phone. The instructions displayed in the phone stand were multi-lingual
so I assumed there would be no problem; firstly because it was
an international call, I was instructed to dial 07 and wait for
the higher tone; then I had to dial the country code which I could
select from a list of about 30 countries. It was at this stage
that I became stuck since the countries were listed in Spanish
(which I don't understand). I couldn't even make a guess since
the country I needed was Reino Unido (United Kingdom), code -
Despite the best intentions of the
display design to take into account the possibility that not all
phone users will understand Spanish, it had not gone far enough
to make the system robust for ALL potential users (probably because
the system was not tested using single language users).
Some users may have very specific
profiles that require specialist designs. For instance blind people
are a specific user group whose needs must be considered at the
design stages to ensure that a system or product will be suitable.
A main stream Connect 4 board game using red and yellow counters
will be totally useless for blind players because they will not
be able to identify the colour of the counters. The design of
the game counters for this particular user group must therefore
be changed to take into account the peculiar disability. This
has been done by drilling a hole in one set of counters enabling
them to be distinguishable from the other [RNIB 1996].
Therefore it can be seen that there
is an element of 'know your audience' when designing systems to
ensure that they are suitable for the whole user group.
After living in the same house since
I was born, I reckoned I knew which light switch was which:
My house is two storey and I have
the ability to switch the upstairs landing light on or off from
the downstairs hallway or from the landing itself . The switch
on the landing is a single switch but the switch in the hallway
also encompasses the switch for the hallway too. The double switch
panel was made of Bakelite and the switch for upstairs was situated,
on the panel, above the one for downstairs. The house was re-wired
five years ago and this old Bakelite double switch was replaced
with a more modern one. The switches for both the upstairs and
downstairs lights are located horizontally aside each other. It
looks nicer than the old panel but, despite the time span, I still
haven't a clue which one switches which light.
The moral here is to use "natural
mapping", as discussed by Norman , to fit the real
world task onto the control for that task. In this way there is
no need to make a conscious decision, which may indeed by the
wrong one. The top switch for the top light seems to be an obvious
solution rather than the right (I think it's right) switch being
for the top light.
Putting this concept into an IT
context Cuomo & Bowen  suggest that "effective
USI design is to minimise human information processing or cognitive
demands on the computer system user". If there is a possibility
to map tasks onto controls using "physical analogies and
cultural standards" [Norman 1988] I suggest they should be
used because they reduce the need for the user to make a conscious
choice. I will now discuss the former of these mapping techniques
in considering another situation where the real world doesn't
seem to fit or naturally map its controls. The latter is discussed
in The Sign and the Use of Colour
Consider the situation in which
a person takes a bath. Abowd and Beale  consider this scenario
in terms of their "Unifying Framework for Interaction",
which I have discussed in General Framework for Interaction .
What seems to be a natural way of performing the task, i.e. filling
the bath with water from two taps, is quite poorly mapped because
the semantic distance (in which the bather has to determine whether
the intended goal has been accomplished) is quite large. Let me
The task concerned, involves two
goals. Firstly the bather needs to fill the bath with water. Secondly
he needs to obtain the correct temperature for the water. If one
of the taps controlled the amount of water and the other set the
temperature, all the bather would need to do is initially set
how much water he needed and what temperature he wanted. The output
would therefore be clear cut and the semantic distance shortened
as a result.
However in common reality both taps
contribute to, both, the water flow and the setting of the temperature.
The semantic distance increases because the bather has to constantly
check the water and temperature levels (the output) and make iterative
adjustments to the input (i.e. both taps) to accomplish this,
quite complex, task. If the taps did control the temperature and
water flow independently then this "complexity is squarely
placed on the system side" [Abowd and Beale 1991] as opposed
to the bather as it is in common reality.
Abowd and Beale  consider
that a large semantic distance in the example of filling a bath
is "precisely what should be avoided in a good interactive
system". Ironically however this design is still used despite
the fact that it is quite poor from the view of natural mapping.
I would consider the reason for this is two fold. Firstly to produce
a system of independent temperature and water flow is complex
(since water is usually delivered by two pipes, one hot and one
cold [Abowd and Beale 1991]) and/or relatively costly (a domestic
shower unit is an example of such a system). Secondly, since a
significant amount of the world's population have taken a bath
each week since they were born using hot and cold taps, the extra
effort they are putting in to undertake the task is a seemingly
natural process in its own right anyway and thus distracts them
from realising their extra effort.
The previous two examples have demonstrated
systems with poor mapping causing one to be inconvenient and the
other, due to the reasons given, to be unaffected. Not all interactive
systems however have this tolerance against inappropriate or unmatched
mapping. In safety critical systems, where lives depend on effective
interaction at the interface, it is even more important that "controls
and displays exploit natural mappings" [Norman 1988]. This
view is endorsed by Grandjean  who proposes that "any
controls that might be mistaken for each other should be so designed
that they can be identified without difficulty".
Consider McFarland cited in Grandjean
 who reports that "the American airforce in World War
II suffered 400 crashes in 22 months because the pilots mistook
some other lever for that controlling the under carriage".
In situations where split second decisions are needed effective
interface design cannot be undervalued. Bates  proposes
that the "safe and successful implementation of new systems
will depend on their design and operation". I have uncovered,
that after a disaster it had usually been the poor user who gets
the blame rather than the design (surely all 400 pilots can't
There are many instances of where
user 'error' has been cited as the initial cause but further investigation
has resulted in a more critical look at the systems themselves.
Johnson  refers to the "Kegworth and the Three Mile
Island disasters" in which "substitution errors"
were a "contributory" factor. In the case of Kegworth
the investigation concluded that "the on-board systems failed
to prevent pilots from shutting down a healthy engine" [Air
Accidents Investigation Branch cited in Johnson 1994]. It seems
that as research continues into analysing accident investigation
methods, the initial presumption to blame the user is now tempered
with a more objective review over the whole interactive system.
Returning to Grandjean's 
suggestion of controls consider figure M
by Joseph L. Seminara cited in Norman . These knobs can
be found in a nuclear plant and they have both been customised
to enhance identification. Ultimately this helps to avoid the
possibility of the operator pulling the wrong switch in two ways.
Visibly the look different but they also feel different too. They
may not look good but both controls have become more difficult
to substitute, in error, for each other.
In this section I have considered
the reasons for undertaking a profiling of the target user group
and explained why good design benefits from it. User modelling
is a large area of research in its own right and I discuss it
further in Customer Requirements . I have also described the advantages of mapping, which
has the ability to help reduce the interaction distances (or gulfs)
and have illustrated my arguments by discussing examples of good
and bad designs.
Interface design requires a knowledge
of how users will perceive the information given to them at the
interface whether it is a VDU screen, cash point or switch panel
etc. I will demonstrate, in this section, that people perceive
or 'see' their own version of the world and why recognition of
this must be appreciated in order to effect usable designs.
It has been argued that there is
two main categories of theoretical approach to human perception.
Constructivist theorists such as Gregory cited in Preece et
al  believe that "the process of seeing is an active
one in which our view of the world is constructed both from information
in the environment and from previously stored knowledge".
Alternatively the ecological approach as outlined by Gibson cited
in Preece et al  interprets perception to be "the
process of picking up information from the environment
and does not require any processes of construction or elaboration".
There is much evidence to support
the view of constructivist theory. Consider the pattern of dots
in figure N.
Upon viewing the photograph taken
by RC James, (in [Preece et al 1994]), some viewers may
not be able to perceive a recognisable pattern in the dots. However,
even if prompting is required to establish the scene of a sniffing
Dalmatian, a constructivist theorist would argue that "without
the prior knowledge... of what a dalmatian looks like... we would
not be able to make sense of the picture" [Preece et al
1994]. In effect the viewer is constructing a model of what is
expected and from the clues given.
Consider the following list:
1 2 3 4 S 6 7 8 9 10 l1 12
A viewer of this list may see an
S instead of a 5 but would he notice the l1 (the letter l and
the number one) instead of the expected 11? Again, in this example,
expectation may overrule what is actually seen by the eye.
In the next example shown at figure
O there is no doubt that the image shows a drawing of a woman.
However how old is the woman? It depends on the viewer; some may
perceive an old woman whilst others will see a young woman. If
this drawing has been witnessed before the viewer will see both
again highlighting that experience is drawn on. Also some viewers
may have subconsciously chosen to remember only one, until prompted.
Unfortunately such an unstable picture
can give rise to the viewer being confronted with an oscillating
set of images with which he is unable to concentrate on only one
of them. Pratchett  in his fantasy Discworld novel describes
a magic carpet which "had a complex pattern of golden dragons
on a blue background" which after lengthy staring seemed
to become "blue dragons on gold background" and "that
if you kept on trying to see both types of dragon at once your
brains would trickle out of your ears". I'm not suggesting
such an image would result in the same effects in the real world
but the potential for similar stress and inaccurate display can
Preece et al  further
contribute to the constructivist theory by suggesting that we
build on what we see by using our "prior knowledge and expectations".
Otherwise, as Gregory cited in Preece et al  points
out, considering that since "we are [just] given tiny distorted
up-side-down images in the eyes ...[with which we have to model
the world.]... this ...[would be]... nothing short of a miracle".
It is not surprising that each of
us will have our own limited view of the world.
Alternatively ecologists believe
that "perception is a direct process in which information
is simply detected" [Gibson cited in Preece et al
Returning to the example of the
door knob/push plate in the introduction to Guessability it was argued that the guessability
of the system was based on previous knowledge, similar in theory
to the constructivist approach. Conversely Gaver cited in Preece
et al , however, sites a similar example of a door
opening system to justify ecological theory, the argument being
based on the notion of affordances. Preece et al 
expounds this in terms of Gaver's example: A "thin vertical
door handle affords grasping , which in turn affords pulling...
[where as a] ..flat horizontal plate affords a pushing action
rather than a grasping action" implying that both views carry
some weight of evidence.
Perception therefore is an important
element in the role of understanding how the user meets the interface.
It has been established, as understood by constructivists, that
different viewers will perceive the same data in differing ways
based on their own "mental models" of things. As Norman
cited in Booth  confirms "these internal models...
by which people can... predict the world around them... tend to
be incomplete, unstable, do not have firm boundaries, are unscientific
Therefore any scene is not only
perceived on what can be seen but also on what is predicted based
on an individual's own mental picture which can thus be open to
a wide range of interpretation.
Preece et al  however
also summarise that design, considered in terms of ecologist theory,
is also susceptible to a range of interpretation based on the
notion of whether or not the affordances in question are "perceptually
obvious... [or]... ambiguous" resulting in errors at an interface.
I would not argue with this view which seems to be a good foundation
from which to design.
Consequently, I would suggest that
both theories subscribe to the belief that for a usable design
for an interactive device to be achieved, a reduction in the range
of possible errors in perception must be reached, whether the
system is predicted based on the, possibly, erroneous preconceived
ideas of the user or on the ambiguous affordances of the system
"Designing well is not easy"
as Norman  points out. I would suggest that the main reason
for this is because users happens to be human. However this may
be turned to advantage when designing interactive systems and
there are ways in which design can be improved without the need
to undertake great research or effort.
In contrast to affordances (as outlined
in Ecological Theory ) there are constraining methods by which
a designer is able to limit the user in operating an interactive
system . Norman  aptly summarises this by stating "that
affordances suggest the range of possibilities [whilst] constraints
limit the number of alternatives". According to Norman 
there are four types of constraints.
Firstly there are physical design
constraints in which a user will be constrained from undertaking
an action by the actual physical form of the device. One example
cited by Norman  is the design of the ubiquitous 3½"
floppy disk; despite the eight conceivable ways of putting the
disk into the drive bay only one is possible due to the physical
shape of the disk (try it!).
Norman  also considers "forcing
functions" which he deems to be "strong constraints".
One example he cites to demonstrate these is the NES whose design
doesn't constrain the user as it should. The instruction manual
for the game console cautions the user, in large upper case lettering,
to switch off the game before removing the game pack. The game
program may be corrupted if removed before the power is switched
from it. Norman  considers that this function (of switching
off power before game removal) should be forced and not just warned
about. It is interesting to note that its rival, the Sega Megadrive
also had the same design flaw and that Nintendo had this function
forced on its upgrade console, the SNES.
Secondly there are semantic constraints
which "rely upon the meaning of the situation to control
the set of possible actions" [Norman 1988]. For instance
an anti-glare screen should be placed at the front of a screen
even though it may well fit neatly on the top of a PC.
Thirdly cultural constraints may
be used to increase natural usability. I have already discussed
the use of colour to satisfy expectation making choice easier
(The Sign and the Use of Colour ) but
there are other ways in which cultural norms can reinforce and
constrain users decisions. Consider the following scenario which
gives a good example of where cultural constraints have not had
the desired effect.
The effect of driving on the right
hand side of the road on the continent seems to have bred a culture
in which pedestrians also pass on the right. People in the UK
tend to pass on the left. On a visit to a shopping complex in
Tenerife I came across a double escalator that appeared not to
be working so, being British, I walked down the left hand escalator.
When I stepped out of the escalator I glanced around, for no particular
reason, and to my surprise the stairs had started moving. I thought
no more about it until the next time I used the escalator and
noticed that it had an arrow and a no entry sign marked on the
floor in front of each stairway.
Each escalator would start to move
when a person passed into the stairway because they tripped a
light beam switch across their path. Because I had walked down
the 'up' escalator I had actually tripped the switch when I had
passed out of the stairway.
Even taking into account poor observation
the fact is that the constraint of a simple sign on the floor
denoting the 'up' and 'down' direction for each escalator was
not effective enough, some thing more physical, such as a one
way barrier, could have been employed.
Finally designers are able to use
logical constraints to improve the natural mapping of controls
against their function in the eyes of the user . The example of
the light switches I
cited gives a good explanation of logical constraint. Norman 
gives another illustration of logical constraint in the building
of a simple Lego model without the aid of a guide. Some builders
were left with one piece with only one place for it to go. Completion
of the model by the builder was easy because "logic dictates
that all pieces should be used with no gaps in the final product".
Therefore the last piece was logically constrained.
According to Lazonder & Van
Der Meir  "in learning to use software, people spend
at least 30% of their time dealing with errors". As a result
they suggest that the consideration and acknowledgement of errors,
during interface design, should be explored rather than avoided.
In terms of improving interfaces, the suggestion is an interesting
one since it ultimately implies that if a design is developed
with the goal being a total elimination of errors, then the interface,
by default, should be highly effective and would prove usable.
Mayhew  agrees with this sentiment up to a point and suggests
that "one goal for a software user interface is to minimise
user's errors... because it will most likely be impossible
to eliminate all errors".
Norman  also points out that
"designers make the mistake of not taking error into account".
Accepting that users will make slips from time to time should
be considered to be a core feature of the design process. This
will have a two fold effect; firstly the designer, being aware
of possible error, can aim to reduce them and secondly it will
encourage suitable error recovery to be incorporated into the
overall design for when the user does make mistakes.
the Burden of Task onto the System
One way in which a system can be
designed to aid the user is by enforcing the system to take the
burden whether this is thinking, analysing, responsibility, or
other processing that could be transferred to it. Let me explain
using an actual example of how the burden of task, in the form
of responsibility, can be transferred.
My local railway station, at Kirkby,
is at the end of the electrified Merseyrail network and at the
start of the diesel line that connects with Wigan and beyond.
The station has two platforms, end to end, which are separated
by a road bridge. This is not the only way they are separate.
The Merseyrail line is controlled by an up to date, computer controlled,
signalling system whilst the diesel line is controlled by a system
that has not changed since the start of the line in the middle
of the last century [Griffiths 1995]. It could be guessed which
one is the more fool-proof.
The diesel line, from Kirkby is
single-tracked until it reaches Rainford whereupon it splits into
two. Consequently if a train passes Rainford Junction heading
towards Kirkby, it must have sole use of this one track until
it is returns to Rainford, allowing another train access to the
Kirkby line. On initial consideration of the system, it appears
that the task of remembering whether a train has passed the junction,
and the subsequent signalling, was the responsibility of the signal
Unfortunately people are not very
good at remembering things, even a trainload of passengers can
be forgotten during a moment's lack of attention. Even in the
1850's it was realised that to entrust this task solely to one
man's memory would have been quite catastrophic; consider the
carnage if a simple signalling error is made by the signal operator
at this junction. The solution was a very simple one: a peculiar
ring wrapped in leather, about 800 mm in diameter, is requested
by and given to the driver of the passing train by the signal
operator as the train passes alongside the signal box. Any subsequent
train cannot proceed onto the used line, since the signal operator
will not be able to supply the ring.
It can therefore be demonstrated
that the task, of remembering if the line is busy or not, has
been placed onto a system rather than onto a solitary individual.
The system, in this case, refers to the sharing of the responsibility
(between both the driver and signal operator) which drastically
reduced the chance of an error.
In this section I have detailed
how usability can be increased by simply taking into account the
fact that the user is a human being, and as such, has certain
universal traits and habits that can be relied on; as Norman 
suggests we should indeed "design for error".
Assessment & Method of Evaluation
According to Johnson 
"the aim of human factor
evaluations is to identify inadequacies in design and to provide
the design team with a sufficient understanding of how the design
is inadequate, so that it can be redesigned without the same
inadequacies being present."
I wish to highlight three issues
from Johnson's statement. Firstly there is a need for a method
with which the design inadequacies need to be uncovered. Secondly
there is a communication method required to evaluate the information
gleaned from the first. I suggest that involving a user in both
methods will enable a design team to address the purpose of the
third issue; that of effecting a usable design.
However what is a usable design?
Holcomb & Tharp  remark that "only if the ultimate
users of a product are pleased is a product likely to succeed".
How better to do this than to assess and evaluate the system involving
the users of it. Therefore the testing, evaluation and reassessment
of subsequent versions of design as described by Johnson implies
an iterative methodology that I suggest would benefit from involving
Indeed Holcomb & Tharp 
suggests that users should be "brought into the development
cycle" because this "provides opportunity for feedback",
which I have already demonstrated to be a principal element to
affect overall usability.
In recent years usability assessment
and evaluation has risen in stature and is now seen as a useful
and important tool to aid design rather than being dismissed,
as it previously had, as insignificant criticism. In this section
I will detail methods used in usability testing and describe the
evaluation techniques used to discover design problems.
For a quick assessment on a proposed
design one cheap and yet highly effective method that can be employed
is a pen and paper exercise. This method involves presenting an
outline of a system's design on cards, story boards or simply
on paper to the user for consideration. Booth  describes
this kind of walk through as a "concept test" which
has the advantage of quickly identifying "concepts that the
user finds acceptable and those that are likely to cause confusion".
Users have the ability, even at this stage, to offer feedback
on the proposals by manually adding to or amending the designs.
I suggest that the role of this test is comparative to that of
a context diagram in systems analysis, giving both the designer
and user an overview of the envisaged system ensuring they both
start off with a common and relevant baseline.
Lewis cited in Cuomo & Bowen
 considers an evaluation method by which "a list of
theoretically derived questions about the User-System Interface
(USI)" is put to users about how they undertook selected
tasks. The interviewer will also ask the users what tasks in particular
they found difficult to do. The purpose of the test is to ensure
that there is an "Action-Goal" match within the system
Dix et al  suggest
that this method does not involve the user and that the "basic
intent" behind such a test is to discover design features
that "violate known cognitive principles". They note
that the test is undertaken by the "designer or an expert
in cognitive psychology" who works through each task in the
design noting how the interface will affect the user and whether
the required task can be completed effectively. Dix et al
 compares this to the way in which the software engineer
will go through the design code line by line which was where the
original idea came from [Yourdon cited in Springett & Grant
Originally cognitive walkthrough
tests were termed "walk-up-and-use interfaces" because
they had been developed for evaluating Automatic Teller Machines
[Cuomo & Bowen 1994].
Dutt et al  considered
this method to "be an effective method as it identifies task
related problems rather than problems of 'taste'" which seems
to suggest that users shouldn't be involved with this method confirming
Dix's description of the evaluation.
Springett & Grant  also
point out another further advantage in that the output from a
system can be "carefully examined" after each step in
the walkthrough has taken place.
Hostile and Simulated Users
As Booth  reports there is
more than one type of user. Friendly users are users who have
some knowledge about the system who are able to make constructive
comments that will be able to enhance designs that naive users
will not have the foresight or experience to suggest. Although
as Hewitt cited in Booth  points out friendly users may
"miss aspects of the system that often cause difficulties
for naive users" anyhow.
Hostile users as noted by Booth
 may have the advantage over naive and friendly users since
they have "no investment in the system" and will have
no fear trying to crash a system. They will be able to apply criticism
to systems exposing "inconsistencies and flaws" that
may have not been uncovered otherwise.
Hewitt cited in Booth  also
defines the possibility of simulating users in which the "progress
of several naive users is charted" which can be subsequently
retraced by the designers. The advantage here is that the designers
can follow a path through the design that they had not "previously
envisaged" [Hewitt cited in Booth 1992] enabling them to
redesign for error handling etc. that would not have been dealt
This method involves users speaking
their thoughts regarding the system to the designer (or a tester)
as they use it, in an informal atmosphere. The tester is on hand
to prompt the user only, without hindering or giving instruction,
and to "listen for clues as to how the user is dealing with
the system" [Lewis cited in Shneiderman 1992]. The advantage
is that the user explores the system in a work-like pattern rather
than following a particular route though the interface which may
not be representative of a real situation.
Users' views of a new system can
be canvassed by interview or by questionnaire. When interviewing
prospective users "the level of questioning can be varied
to suit the context" and also the line of questioning can
"probe the user more deeply on interesting issues as they
arise" [Dix et al 1993]. Dix et al  point
out that interviewing will give the user a chance to mention problems
that may "not have been anticipated by the designer"
of a system and is particularly useful "in eliciting information
about user preferences, impressions and attitudes".
Despite the usefulness of the interviewing
method which will give a general indication of "whether [or
not] a system is likely to be used and appreciated in the work
environment" it may be "open to bias" [Booth 1992].
This bias may be in the form of un-helpfulness on behalf of the
interviewees fearing the "risk that new technology... [which]
will create highly repetitive tasks which will require little
skill..." [Johansson cited in Grandjean 1988] (Keen cited
in Shneiderman  terms this as "counterimplementation").
Alternatively the interviewee may, fearing for his job security,
not wish to appear hostile and so not flag up any negative attributes
about the proposed system which again results in unconstructive
Alternatively questionnaires are
usually anonymous and thus have the advantage of being able to
extract a more honest and open response. Unfortunately due to
anonymity the evaluation is only one way and, unless the questions
are open, may only assess pre-defined areas that may not suffer
from usability problems anyway. Questionnaires may use a scalar
(e.g. the Likert or semantic differential scales as noted by Preece
et al ), multiple choice or ranked method of answering.
In the following section I will discuss the attributes of a sample
questionnaire (the SUMI) which uses a Likert scalar method.
The SUMI questionnaire (shown at
Appendix 1) was developed by the Human
Factors Research Group, University College, Cork, Ireland as part
of the European MUSiC project [NPL 1996] to measure, subjectively,
the level of satisfaction a user has with software releases. It
has "been developed, validated and standardised across Europe"
and is available in many languages [NPL 1995a] and is often used
as the definitive industry standard (Reuters, for example, use
this questionnaire extensively during usability testing. See Chapter
7). The questionnaire is often used as a baseline by which
subsequent product versions can be measured [NPL 1995a]
The SUMI contains a list of fifty
questions [NPL 1996] that the user can either agree or disagree
with or note an undecided result. It has the ability to give information
in terms of software "efficiency, affect (or likeability),
helpfulness, control and learnability, plus a global measure of
usability" [NPL 1996]. In turn, this allows for the discovery
of "overall strengths and weaknesses" [NPL 1996] of
the software allowing the designers to confirm good design practices
and concentrate on problem areas respectively.
Perhaps one reason why this questionnaire
has been so successful in the field is that it is only part of
the overall family of support for measuring usability assessment
available through NPL. For instance there is software support
for the SUMI, 'sumisco', allowing "computerised administration",
"scoring... [allowing] analysis using a database of standardised
samples" and report generation [NPL 1996] .
The SUMI may be suitable for empirical
analysis but its scalar answering restricts possible responses
that could be obtained if open questions were employed. However
as noted by Dix et al  "probing" questioning
will take more time and resources and, as with most usability
issues, a trade off has to be made.
I feel that the effectiveness of
SUMI, in terms of its own usability, is high because it meets
only the role set for it and does this well. The main advantage
in using this testing method is that it can quickly cover a wide
range of users and that the form itself is easily completed, relatively
cheap and effective as a collector of measurable data. As such
I suggest the SUMI to be an ideal and inexpensive tool that can
be used as an initial 'gut reaction' survey of users when testing
Booth  considers the requirement
for an independent "designer" or a "human factors
expert" in the review of systems. He suggests that the advantage
for such review is that "comments and criticisms are made
from a position of knowledge". However as IT systems continue
their onslaught into all areas of business, manufacturing, leisure
etc., so there is also a need for a greater understanding of the
context and language of these areas. Shneiderman  also recognised
this need and envisaged a professional growth in domain expertise
incorporating areas such as "geographic information, medical
laboratory instruments, or legal systems". Experts are therefore
not just experts in interface design but come from a wide range
Bastien & Scapin  however
note that "experts rely on their experience in order to make
a judgement on the ergonomics of a system". Their findings
indicate there is a plethora of inharmonious usability standards
due to this wide range of independent expertise. As a result
they suggest that: "the evaluation of user interfaces is
difficult and that much work is needed if dimensions are to serve
as a basis for an evaluation method". By dimension they mean
real, definable and specific areas or fields of measurement.
They propose, in their paper, that
the dimensions, as well as being defined "explicitly, unambiguously
and consistently" should be able to demonstrate their "utility
and usability". Therefore they propose that any dimensions
of ergonomic criteria should be
· "Valid" restricting
the reviewer to evaluate only those elements that were intended
to be evaluated, but also conversely
· "Thorough" enabling
the widest scope to be achieved in evaluating the particular
· "Reliable" producing
the same results under the same conditions.
Molich & Newbon cited in Bastien
& Scapin  suggest that if these conditions were not
met for ergonomic criteria "variability" would result
and "tests would become reliant on expertise only" which
would not provide comparability of result. In addition to the
variability of results, the expert review may not actually detect
"difficulties that hinder a naive user" [Booth 1992]
which is the initial purpose of the review.
In this section I have reported
the various methods used in usability testing and evaluation discussing
their advantages and disadvantages. There are other forms of assessment
and evaluation including task audits, field trials, follow-up
studies and field studies [Booth 1992] as well as using video
and audio taping of users working at interfaces in laboratory
situations (this is discussed further in Usability Testing).
Because not one method is perfect
for each situation, in practise, a combination of methods are
used to overcome each other's disadvantages. Indeed to assess
and evaluate a system effectively the correct methods needs to
be chosen and this is a task in itself (there has been much research
in this area in its own right).
Whatever methods of usability assessment
and evaluation techniques used I suggest that the user is the
one who should have the final usability vote because he is the
one left to use the system when the designers, programmers and
other experts are long gone.
Traditionally, software engineering
followed a waterfall model of design in which the design process
falls through to the next stage or activity upon completion of
that activity. These activities generally group into areas such
as "requirements analysis and definition, system and software
design, implementation and unit testing and interrogation and
system testing" [Preece et al 1994]. Each activity
is quite isolated and self contained. In this way the most "appropriate
techniques" [Dix et al 1993] can be applied to each
stage resulting in the best output for that particular activity.
However Dix et al  notes that "the analogy of
the waterfall is not completely faithful" since "in
practice the stages overlap .... The software process is not a
simple linear model", answers Sommerville cited in Preece
et al , "but involves a sequence of interactions
of the development activities".
With the growing awareness of usability
issues it can be seen that this model will not readily accommodate
user input effectively into the design process as a separate activity
because usability consideration needs "techniques which span
the entire [design] life cycle" [Dix et al 1993].
As a result Hix & Hartson  suggested a star model of
development with usability and evaluation being at the centre
of the development cycle.
Star Life Cycle
Hix & Hartson's  model
of development, shown at figure P, is called the star life cycle because of its shape.
In practice, any activity in the development cycle can be carried
out first and development of a system is not restricted to a rigorous
sequential process. The star method is thus "supportive of
both top-down and bottom-up development" and it can be noted
that it is "evaluation-centred" [Hix & Hartson 1993].
This allows iterative design to be accommodated since the designer
can work with any activity in the knowledge that the design process
will always pass through the centre point (usability & evaluation)
before he moves onto another one. Knowledge gleaned from evaluating
one aspect of the proposed system can thus be fed into the other
One area of significant and obvious
difficulty in designing interactive systems is that "the
customer and the user may not have a clear idea of what the system
will look like when it is done". Shneiderman  recognised
this problem in one of his "three pillars of design".
One way around the problem is to prototype the interface with
the user and revise designs with the feedback. Otherwise, as Shneiderman
 points out, "it is difficult, costly and time consuming
to make major changes to systems once they have been implemented".
The great advantage of prototyping,
of course, is the ability to involve users in the decision making
process (they are experts in their tasks after all) at an early
stage. There are various levels of prototyping. Firstly there
is the throw-away, or rapid, prototype in which only the knowledge
gained from the testing of it, with the user, is kept. Concept
tests, as in The Interface could be
considered to be in this category. The tools for these, often,
"low fidelity" [Preece et al 1994] prototypes
tend to be relatively cheap and are used to produce proposed designs
quickly. High fidelity prototypes, on the other hand, can be used
to mimic the envisaged system complete with its functionality
and interface. These can be costly but will give the user a considerable
preview of what the final system will be like.
However extensive prototyping is
not free from disadvantages. As modelled by Boehm cited in Preece
et al , an ever increasing spiral of prototypes may
create more problems (such as a lack of management control) and
thus be accompanied by spiralling costs. To avoid such costs Harrison
cited in Preece et al  considers the introduction
of his "W" model of prototyping. In this model, the
system is prototyped only once and only on a small scale so that
"the system requirements are fixed and a traditional approach
to development is undertaken." Preece et al .
As a consequence this "evolutionary" spiral of multiple
prototypes is regulated.
I introduced this chapter by taking
up the argument that user involvement during the development of
interactive systems would prove both effective and worthwhile.
However I have discovered that the usability approach to interface
design is not the sole cure-all to expedient design and
I have attempted to illustrate this during my discussion.
Indeed, Ives and Olson cited in
Shneiderman  point out there are disadvantages of a participatory
methodology. They put forward that user involvement could be "costly
and lengthen the implementation period, build antagonism with
those not involved or whose suggestions have been rejected".
They continue by suggesting that it may "force designers
to compromise their design to satisfy incompetent participants
[or] simply build opposition to implementation" generally.
It seems that the initial idea of involving users may introduce
as many problems as it solves.
However there has been enough interest,
in the issue, for the ISO to produce a draft specific standard
(ISO 13407) for user centred design [NPL 1995b] reinforcing both
the credibility of, and the need from industry for a standard
guideline on, participatory design. I would therefore suggest
that the involvement of users to aid in the design of interactive
systems is here to stay.