|
|
|
Consultant Usability Engineer Digital Equipment Corporation 550 King Street Littleton MA 01460-1289 USA +1 508 486 6275 comstock@delni.enet.dec.com |
PATHWORKS Technical Director Digital Equipment Corporation 550 King Street Littleton MA 01460-1289 USA +1 508 486 7581 duane@ranger.enet.dec.com |
The underlying architecture of complex software products profoundly influences their direction and usability. This paper shares an effort to embed usability within the architecture of complex network products. We began by attempting to build a conceptual model, but we ended by representing customers' and users' values in a Declaration of System Usability to guide product direction and system architecture decisions.
System usability, complexity, system architecture, software architecture, design techniques, networks
In this paper, we share how a group of usability engineers and technical directors at Digital Equipment Corporation embed usability within software system architecture.
We begin with a description of the context for this work - a large, complex family of networking products. We describe traditional approaches to achieving cross-product usability and relate usability to system architecture. We then describe our search for usability design principles based on users' experiences with networked computing. We initially thought a conceptual model would encapsulate the design principles in a way that would allow system architects to embed usability across the product family. Instead, we found a different device, a Declaration of System Usability. The Declaration does not replace other usability approaches; it is a companion or precursor to other approaches. It establishes a usability policy that guides high-level product design decisions, tuned to a specific set of customer uses. The paper closes with a description of how developing the Declaration changed the culture and products developed in our organization.
We hope our experience will motivate others to examine the relationship between usability and system architecture.
The Network Integration Software (NIS) group within Digital is the setting for the events related in this paper. NIS products like PATHWORKS aim to make it easier for people to productively use and manage their personal computers in a network environment.
Producing software for the PC LAN integration market is a major undertaking because computer networks are inherently very complex. They are a mixture of devices developed by many different vendors, different networking protocols and standards, different operating systems, and different computer types. They serve applications ranging from simple mail exchange to complex synchronization of distributed directories. In NIS, over 200 developers design and build thousands of software modules that control everything from network hardware to applications for software developers, PC LAN administrators, and end users.
Usability is a central theme in the design and development of NIS products. Similar to Nielsen [8], we use a broad definition of usability, including learnability, efficiency, memorability, few errors, and user satisfaction. We consider the usability of all aspects of NIS products with which users may interact, and we consider the system's utility to be a precondition of its usability.
Usability work is generally well integrated into the development of individual NIS products, and members of the NIS Usability Team work at all stages of product development, from product requirements determination through product design, development, and release [2]. Nevertheless, in spite of persistent usability work, users were not satisfied with the usability of NIS products. They reported that the products were complex and somewhat inconsistent. Using and managing computer networks required too much knowledge of the underlying technology.
The NIS Usability Team began to look for additional approaches to ensuring usability across complex product sets. There is a large professional literature offering excellent advice on effective customer engagement, participatory design, usability engineering, and user interface design processes [11, 12]. These processes form the foundation for product usability. They are generally well-accepted and widely practiced in Digital [15].
One shortcoming of most usability methods is that they apply best during the development of individual software applications. As Nielsen [8, p. 71] says, "Usability cannot be seen in isolation from the broader corporate product development context where one-shot projects are fairly rare. Indeed, usability applies to the development of entire product families and extended projects where products are released in several versions over time."
What approach might help bring usability to a complex family of products like that produced by NIS? Advice on achieving system-wide usability falls into three categories.
First, many people recommend process mandates or "institutionalizing" good usability practices within an organization's product development. The idea is that what works for one product should be done on all products; the result will be overall usability. For example, Bellcore has a user-centered design policy focused on the methods developers will use. It states, "All producers of software products will apply user-centered design methods: task analysis, usability objectives, early prototyping, usability testing, and iterative design." [9, p. 493] Software development organizations in Digital have applied usability methods relatively effectively, and usability is represented in both formal and informal development practices [2,15].
A second general approach is to provide tools that allow individual developers to incorporate usability more easily and develop more consistent user interfaces. For example, Gould, Boies, and Lewis [5] describe a method of rapid prototyping that builds in guideline compliance. Other examples of this general approach are style guides [13], user interface standards, and terminology dictionaries [8]. The NIS group also recognizes the importance of consistency in achieving overall usability. As a parallel effort to the work described here, the NIS User Interface Style Guide was developed. It is now in use throughout NIS.
A third approach to system-wide usability is to find a unifying theme or conceptual model that provides developers with a common set of expectations about how final products should behave. Mayhew [7, p. 80] defines a conceptual model as "the general conceptual framework through which the functionality is presented." Conceptual models are best if they match users' internal representation or mental models. Following the conceptual model will then unify the interfaces of the separate components.
An explicit metaphor can help users understand the conceptual model by providing a mapping or analogy between a familiar domain, such as "the desktop," and the new domain, such as "networked computers" [1].
The third general approach, defining a conceptual model, is the one we considered most promising for addressing NIS system-wide usability issues. We set out to understand users' mental models and expectations for networked computing, intending to construct a conceptual model, and possibly an explicit metaphor, to guide design choices.
In analyzing the network complexity problem, we realized that how usable a product set will be depends strongly on the system architecture that underlies the products.
By "system architecture" we mean the top-level design of software solutions. System architecture is often described using the analogy of blueprints for a house. The analogy we like better compares system architecture to city planning, - "there are multiple, relatively independent structures connected into a network over which people, goods (information), and services (utilities) flow. The structures are far from uniform and are at different points in their life cycle." The architecture activity is a "coordination activity" rather than a design blueprint [6, pp. xii-xiii].
System architectures typically persist far longer than the specific products designed with them. For example, the PATHWORKS architecture that was established in 1986 was replaced in 1995, but in that time span there were five major releases of the products based on that architecture.
At Digital, "technical directors" are responsible for system architecture. During the design of network products, technical directors make decisions that impact virtually every area. Starting with requirements gathering, technical directors look beyond the specific request the customer articulates and try to determine the underlying problem or opportunity. They use their understanding of how the technologies are evolving within the market to project which problems or opportunities will arise.
Once an opportunity has been identified, technical directors evaluate the wide spectrum of possible solutions to the problem. Each solution brings with it a series of tradeoffs that must be evaluated on a number of dimensions simultaneously in order to select an optimal solution. If the system architecture does not address usability in any form, the product components, each on a different schedule, and with different constraints, will be locally optimized around assumptions of the individual component developers. But the system as a whole will be chaotic and inconsistent.
Based on this understanding of the broad scope and long life of system architecture, we recognized that the NIS architecture should provide a rationale for making decisions that trade off usability against other considerations in product development. We expected that representing NIS-specific usability principles in the form of a conceptual model would form the basis of our "usability architecture," and that it would be a useful tool for technical directors as they made architectural decisions.
In the spring of 1994, Betsy Comstock, with support of several development managers and the NIS Usability Team, recruited a cross-functional team of seven senior contributors to serve on the "Usability Architecture Team." The team consisted of four engineers and technical directors, including Bill Duane, two usability specialists, and a product business manager. Members were chosen for their passion around usability issues and their broad influence on product development.
From the beginning, it was clear that the Usability Architecture Team shared a strong sense of the usability problems faced by NIS. We readily agreed on many anecdotes from our own experiences, from customers reports, and from internal development practices. We agreed that we needed a high-leverage approach to ensuring product usability, and we described the need with words like, "whole system," "top down," "strategic," and "usability architecture." The team crafted a mission statement containing the goal, "to develop, communicate, and ensure implementation of a usability architecture for future end user and system administration tasks." We expected to create a new conceptual model for computing in the network integration space, perhaps the next "desktop metaphor."
Here's how the Usability Architecture Team summarized the characteristics of the conceptual model we were seeking: The conceptual model is a set of design principles based on users' tasks and thought processes. When the conceptual model is used as the cornerstone of development, users will clearly experience an appropriate organization of functionality and a demonstrable reduction in the time it takes to learn and use the product family. The conceptual model aids high-level business and development decision-making by specifying how product components must appear to users to function and interrelate. The conceptual model is simple to communicate. It works like a "magnet" to provide a sense of development direction and to pull components toward a coherent whole for users.
Two beliefs about our work were particularly salient - it needed to be based on users' tasks and thoughts, and it would influence product development over time, rather than all at once. Our primary criterion for success was that product development would move "in the right direction."
The Usability Architecture Team recognized that it was crucial to understand our own beliefs (as developers) about our customers and their requirements for our products. Therefore, at an early meeting, we conducted an extensive and humbling brainstorm session in which we discovered a number of "tensions" between the ideal and what we were actually doing. Here are a few examples:
The developers' assumptions contributed to the team's work by making us aware of the magnitude of the changes we were trying to cause, and our own roles in perpetuating certain less-effective beliefs and practices.
The bulk of the work of the Usability Architecture Team can be characterized as immersion in user data, followed by a series of steps to abstract and distill the data. The beginning steps were very close to the users' statements, and the later steps were closer to users' core values.
We did not generate new data for this work. Instead, we collected and "re-used" customer data that NIS had recently collected for other purposes. Most of the data came from direct customer and user studies conducted within the previous year, including usability tests, field test reports, quality assurance reports from recently-developed NIS products, and transcripts from three large sets of visits to customer sites to understand their work. Other sources were published reports by industry analysts, press articles, and Digital Equipment Computer Users Society reports.
The team scoured the data, picking out what we called "customer assumptions," ideas and expectations in customer terms about what their networked environment should be like. We did not mean features or technologies, but rather how people wanted to think about and use their networked environment in the future. We found 231 customer assumptions such as the following:
From these we created 24 affinity groupings. They encapsulated the Usability Architecture Team's shared understanding of what we heard from customers and users about networked computing. Here are a few examples:
The next stage in the usability architecture work involved identifying core themes in what we were learning from the customer assumptions and our own assumptions. We expected to use the themes as "building blocks" for the conceptual model. Themes such as the following emerged:
Next, each team member chose metaphors for networked computing that they thought represented significant aspects of the customer assumptions and themes. Some of the metaphors we considered were the US highway system ("See the USA in your Chevrolet"), cable TV boxes and remote control devices, building a house with a general contractor, Federal Express, town government, musical scores and an orchestra, Amish barn-raising, family life in Eastern and native American cultures, baseball teams, and mitochondria in human cells.
The metaphor exercises were enjoyable, but the team was beginning to see that an explicit metaphor was not what we needed. Each metaphor had major flaws, and we couldn't imagine designing a system with these metaphors in mind. We realized that most NIS products were used in environments where other end-user products set the metaphor. Our products should not call attention to themselves with their own metaphor, but invisibly blend into the fabric of the users' work. A visible conceptual model no longer seemed appropriate.
We made one additional step toward finding a conceptual model. We felt that each metaphor captured something of the essence of what networked computing should be like. Taken together, ideal network computing and all the metaphors shared significant characteristics. We identified 19 such characteristics that we called "essences." Here are some examples:
The previous steps occurred during about three months of weekly meetings. Frustration with the slow pace was rising. The team agreed, "We are done brainstorming." We had exhausted our motivation for the current direction, but we had not arrived at a conceptual model or metaphor for our products, and it didn't look like we would. Nevertheless, the team strongly agreed that we had distilled something very important. If it wasn't a conceptual model we needed, then what was it?
We consulted with Dennis Wixon, Program Manager for Usability at Digital, who reviewed the work we had done and listened as we said we felt that a conceptual model was not emerging. We described how we were steeped in a set of beliefs about what our products needed to be to maximize usability, and that we had a sense of the higher-order principles that should guide product design. We felt we had uncovered something beyond users' expectations, something like users' values, or the ethical principles the products should embody. But we didn't see how to encapsulate our learning, to impel action, to fill the role we had intended for the conceptual model.
Dennis said, "Perhaps you need to think of this knowledge in a different form. I think you are making a declaration."
The sense of a "declaration" is that it brings about a new state, or new behavior. Winograd and Flores [14, p. 59], defining the categories of speech act, say, "Declarations bring about the correspondence between the propositional content of the speech act and reality, as illustrated by the example of pronouncing a couple married." The definition we like best is what Whiteside [10, p. 91] calls a transformational proclamation: "...an expression in language that creates the possibility of a new future by virtue of its being expressed powerfully within a befitting context."
This was the breakthrough we needed! It wasn't a conceptual model, but rather a declaration. We were ready to declare that our organization would move its products in a direction in accordance with the customer values and principles we'd uncovered.
We modeled the Declaration of System Usability on the United States Declaration of Independence [3]. This was a convenient, memorable vehicle for our declaration, and it provided a format. We began with the "truths," the essential characteristics of the users, their environments, and Digital's NIS business. We then abstracted six separate declaration points, listed in the second half of Figure 1.
The first declaration point states that we will integrate people's tasks and intentions, not underlying technology. This point should be familiar to usability professionals [c.f., 4]. It states the primacy of a task-based approach to system design that was clearly requested by our customers.
The second declaration point states that we will recognize the primacy of the person. People are not represented in today's networking; only their accounts or access points to the system are represented. One of the major sources of network complexity is that people use multiple machines, multiple accounts, multiple passwords, and multiple roles in their organizations. We believe that network system use and management will be simpler if the system "understands" the concept of a person, regardless of where they are and what resources they are trying to use.
The third declaration point recognizes that people use networked computers to work with other people. We pledge to facilitate group operations in the broadest sense, allowing groups of people to be treated as single entities, without requiring individuals to separately manage the groups. Similar to the second declaration point, we believe that network system use and management will be simpler if the system "understands" the concept of groups.
The fourth declaration point relates to the first. It recognizes that taking a task-based approach means not requiring knowledge of the underlying network technology.
The fifth declaration point establishes a new way to think about change in network environments. It recognizes that in most network environments, change is feared and avoided because it causes so many problems. We pledge to build products that recognize that change is the norm, in fact, often initiated for positive reasons, and we pledge to hide changes that are irrelevant to users' tasks.
The final declaration point recognizes the importance of cross-component user interface consistency and establishes the basis for the use of the NIS User Interface Style Guide described previously.
Since it is critical that the NIS technical directors support the Declaration at a fundamental level, we decided that its ongoing ownership did not belong with the team that created it, but rather with the technical directors themselves. We revised the Declaration until the following people were willing to sign it: every member of the Usability Architecture Team, every technical director, key development managers, and representatives of product management and marketing. Over the course of 3 months and dozens of meetings, the Declaration went through 10 major drafts. The final Declaration of System Usability was printed on parchment-like paper, made available in both Postscript and World-Wide Web formats on Digital's internal network, and distributed with each copy of the NIS User Interface Style Guide.
The Declaration has had a significant impact on the design and development of NIS products. Here are a few specific examples from the new architecture, of which the first product is the recently-released PATHWORKS V6.0:
Today, when multiple network operating systems (NOSs) are present in the same PC LAN environment, administrators use separate applications to perform the same task on different NOSs. For example, if an administrator needs to modify the password to the accounts of a particular user, the administrator will use a different tool to change the password for each NOS. This requires the administrator to know what type of NOS is supporting the various user accounts, and to do the same task in different ways in each of the NOS environments.
Based on the first declaration point, PATHWORKS re-aligned the administrative model of account management with the task the administrator is performing. The administrator can perform account maintenance without seeing the boundaries between the various NOS environments. For example, to manage passwords, the administrator need only select "account maintenance" for a particular user and then select "change password." The system performs all the NOS specific functions needed to change the password. This task-based redesign of the management approach was driven by the Declaration.
A second example is the design of PATHWORKS Single Login, the ability for one password to give a person access to all their network resources. We were having difficulty deciding how far we should go to address the security issues in heterogeneous NOS environments. One option was a complex and comprehensive security solution including access control lists and encryption key administration policies. Another option was to abandon the entire strategy for Single Login. What was missing was guidance to help us bound what we wanted to achieve.
As the complete system architecture became better defined, we used the second declaration point to bound the design for Single Login. As a result, PATHWORKS decided to address the ease-of-use and location-independence problems associated with logging in to NOS environments, and to preserve the security from the existing NOS environments rather than create a new security solution.
Perhaps one of the most important impacts of the Declaration was on the design of Directory Assistant, our underlying distributed directory, and how it integrates with the Microsoft Windows and Windows 95 operating systems. When introducing a new technology, there is a pull to directly expose the new technology to the user. As PATHWORKS worked through the various alternatives for the distributed directory solution, we relied heavily on the Declaration to help prune the space of options for the system design. In particular, the third, fifth, and sixth declaration points had impact in this area.
PATHWORKS decided to develop a directory solution that was not visible directly to the user, but rather enhanced existing services to make them more robust, more scaleable, and NOS independent. PATHWORKS built the concepts of teams and groups into the system architecture from the start so that we can now address large and small collections of people and network resources as a single item. PATHWORKS also designed a solution that used existing distributed directories in the customer environment rather than require the customer to deploy a new directory. Lastly, PATHWORKS leveraged the fact that distributed directories create a level of indirection between the end user and the administrator so that administrators can change the underlying NOS environment and distributed services without impacting end users.
A final example from a different perspective is the way in which NIS technical directors present PATHWORKS V6.0 to customers. Initial presentations focused on the new technologies in the product and example applications. From an engineering point of view, the attempt was to educate customers on the benefits of the new technologies. The typical presentation assumed that customers would map the technologies into their environment and see the value for themselves. This approach did not work.
Based on the fourth declaration point, the presentation was inverted. It began with customer configurations and problems, and contrasted the way in which the customers are dealing with the problems now against the way in which they would deal with the problems when PATHWORKS was deployed. The scope was then expanded to give brief examples of how the same approach works for other problems. All discussion of the technical mechanisms used to effect this change in the work environment was deleted from the talk, and a set of backup slides was produced to address technical details in case questions arose. The presentation is now extremely well received, and the backup slides are rarely used.
As we have illustrated, the Declaration provides a clear user-centered product focus and everyday guidance for NIS technical directors as they consider questions such as: Are our products being developed in the "right" direction? Do the components interact properly? Does our system architecture support ("afford") usability? In support of the notion that the Declaration is for architects is our observation that it doesn't mean much to line engineers, many of whom consider it "gimmicky," not specific enough to their work tasks, and not clear enough about what the exact implications are for their work.
In addition to focusing the work of technical directors, the Declaration has subtly altered the work of usability specialists in NIS. It has established the role of usability in decisions about product family directions. And it has shifted the emphasis of our work toward the early conceptual stages of product development.
The success of the Declaration lies in its conceptual value to the participants -- it helps organize our experience in a meaningful way, somewhat like a theory does. Different from a theory, however, the Declaration also impels action, changing both products and behavior.
In summary, we believe that a fundamental determinant of the usability of a set of products is its underlying system architecture. But system architects and other technical leaders have few tools or examples of ways that usability can be embedded in system design at the architecture level. We created a powerful declaration, driven by user data, as a new way to embed usability in system architecture.
The method we used to generate the Declaration contains elements of all the categories of textbook advice described above. Like the methods suited to the development of specific products, the Declaration is tied to users' real work and expectations. Like usability mandates, the Declaration sets a policy for product development, based on product content and values rather than usability processes. Like other tools for developers, the Declaration functions as a tool for system architects to use in their high-level decision-making. And, like a conceptual model, the Declaration identifies principles in users' expectations that must be represented in the products.
What is perhaps most different about the Declaration is the level at which it operates. As described in the previous section, the Declaration provides a clear product focus and everyday guidance for NIS technical directors.
There were many aspects of the Usability Architecture Team that we believe made it function exceptionally well and that we'd recommend to others who consider a similar venture. First, the team was very diverse. About the only thing shared by all members was a passion for usability.
Second, the team became immersed in vast amounts of customer data from diverse sources in our business. We believe in the Declaration partly because it is so well-grounded and we spent so much time digesting the data.
Third, we continued abstracting and reconsidering the data long enough to allow the important ideas to emerge. This was very difficult, especially considering our busy, deliverable-driven schedules outside the team. But as we began to see that there was value in what we were finding, our impatience turned into recognition that we were generating a mind shift, a change in attitudes and beliefs. Change like this takes time.
And finally, we generated the support of the technical directors and managers. Without their support, the Declaration would not influence product development.
In searching for new ways to improve product development and enhance usability, we invite usability professionals to explore the form and level of the recommendations they make about usability. We urge other organizations to search for and share ways that fundamental user values can be embedded in the architectures of the systems we build.
The Declaration depended on gracious, thoughtful contributions of many people. Jean Proulx, NIS Vice President, sponsored the Usability Architecture Team and ensured its committed membership. John Adams, Networking Technical Director, sponsored the Declaration in the technical community. Michelle Chambers, Dennis Giokas, Neil Murray, Marlene Steger, Colin Strutt, and Sarah Webber were crucial to the work of the team. Minette Beabes helped define the team's process. Rick Frankosky and Nancy Clark headed the style guide effort. All the signers of the Declaration contributed wise comments in mapping it to their work. Pat Billingsley contributed insights based on her "usability policy statements." George Casaday, Michael Good, Tom Graefe, Deborah Mayhew, Anne Parshall, Mary Beth Raven, Colin Strutt, Dennis Wixon, and Sarah Webber made valuable comments on an earlier draft of this paper. And most significantly, Dennis Wixon enabled our breakthrough with the concept of a declaration and inspired the work described in this paper.