Short Papers

CHI 96 Guide To Successful Short Papers Submissions

based on "An Insider's Look at the CHI Papers Review Process" by Mary Beth Rosson, CHI'95 Papers Chair; the CHI'94 "Guide to Sucessful Submissions" and the INTERCHI'93 "Guide for Sucessful Submissions"

Introduction

Short papers should present late-breaking results and ongoing work in HCI. Accepted short papers are presented in 8 to 10 minutes lecture style format and are published in the CHI Conference Companion. Short papers are 2 pages, formatted according to the standard Conference Proceedings format.

Short Papers must be received in hardcopy form not later than 5:00 p.m (17:00) local time at receiving address on Friday, December 15, 1995.

Fax or email submissions will not be included in the review process.

In keeping with the conference theme of Common Ground , CHI 96 especially encourages work that

integrates different views of the field
explores ideas which can not explicitly be put into a single subfield
develops frameworks of working together and combining results of the different fields represented in the CHI community
combines CHI approaches with approaches from non-CHI disciplines
brings users and other people with an interest or a concern for the system and/or the users work together to discover and explore their common ground.

Short papers are valuable opportunity to showcase both research and practice in HCI problems, needs, and solutions without the demands of formal (longer) paper structure.

All submissions will be reviewed using a high standard of content and presentation. You should have something new and significant to say, you should state it very clearly - in particular because of the restricted space available - and you should support your statements.

Short papers of all types will be evaluated on the basis of

originality
importance of the contribution
soundness of rationale or demonstration
quality of written and visual presentation
adequate citation of the most relevant literature

All short papers describe the context, contribution, content and consequences of the work with adequate focus on the problem you address.

Context (subject area and the perspectives of you and your intended audience).

domain of HCI: the task being done, the class of users doing it and the technology being used
author perspective(s): e.g. researcher, developer, user, manager
audience perspective(s): e.g. researcher, developer, user, managers

Contribution (the relationship of this work to similar work in the field)

background and related work: who has studied this before and from what perspectives
lessons learned: how does this go beyond what has been done before, and from what perspectives
innovation: what are the new ideas of the this work

Content (your central mesage and why you and the audience ought to believe it)

claims: what is the question or issue that you are addressing
description: what was done and by which method
justification: given the authors perspective, why should the audience believe the authors claim; this support for your claims is a central part of your submission; the authors perspective will determine the established standards for these justifications (e.g., experimental psychology requires statistical proof, implementations require resource usage statistics, field experience might use videotape analysis or questionnaires, design ideas require some form of usability testing)

Consequences (the practical implications of the audience believing the content)

action: what should the audience do differently if and when they have accepted your message
directions: what are the directions for future work based on this work (new questions, next studies, new experiences)

Due to the two page restriction avoid too general statements and too long introductory discussions. Be as precise as possible to show the value of the work presented. Do not try to describe everything, it is sometimes better to focus on specific and most important parts of the solution.

Some typical mistakes reported by reviewers of submissions from previous conferences include:

the work was not finished, the outcome was unknown or still in doubt, so the claim was not supported
the work did not compare and contrast the author's work with important work as recorded in the published literature of the human computer interaction field
the author did not demonstrate a good understanding of the state- of-the-art as documented in the literature (e.g. CHI proceedings from previous years) and in industrial products (e.g. a "new" idea should not already be available in some two-year old commercial product)
the work did not draw conclusions or focus the lessons learned for the reader
the work did not state the research results, but merely provides background information and discussion on the importance of the topic
the author made unsupported claims; the conclusions were beyond the results of the work reported
a practice and experience oriented work did not describe the lessons learned; the more general the lessons learned the more important for the conference; the lessons learned must advance over existing knowledge in the field the work was commercial; the problem is the promotion of a product where there is no need to do so the work contained too many unexplained or unnecessary technical terms; the HCI field includes many areas of work, thereforedefine terms from a subfield for the overall audience

A general piece of advice is to have the submission reviewed by somebody outside the group that did the work. If they have some problems with it, then the reviewers will propably not understand it either.

A Short Look At The Review Process

There are several phases in the ultimate decision made concerning your short paper from the reviewers viewpoint. Let's look at each of them and see what you might do to help or hurt yourself.

Assignment

The first important thing that happens is that your short paper gets assigned to some set of reviewers. Your short paper can stand or fall as a function of the reviews it gets, both with respect to what the reviewers say, and how well they say it. What can you do about this?

Assignments will be determined mostly by the topic keywords you select. Each reviewer has been asked to indicate keywords s/he wishes to cover. We will match their keywords against the ones you specify. You can do nothing about the reviewers' self-classification, but you should think carefully about the keywords you provide. Don't go hog-wild, or your short paper will match no reviewer well, and every reviewer so-so. Pick a set that BEST characterizes the way you want your work reviewed.

A secondary input into assignments will be title and abstract -- in cases when the keywords aren't enough to produce a decision, we will browse this information to get a better idea of what the short paper is about. So make sure that the title is revealing, and that the abstract provides a concise and accurate summary of the short paper content. Remember that we anticipate about 300 submissions. We cannot promise to read each one in full before we assign it to reviewers.

Our goal is to get your short paper reviewed by at least 5 reviewers. Generally, it seems that more reviews is better, it gives a more balanced picture of the short paper.

Finally, just in case you are unlucky and your short paper gets assigned to reviewers who don't know the area (sometimes you may feel that you are the only person IN your area!), you should try to make sure that any competent CHI expert can at least grasp the main point of your short paper. Remember, you can't count on CHI attendees to have taken your well-crafted tutorial before coming to your presentation!

Reviewing

Reviewers write and submit their reviews electronically. We have found that this increases the quality of the reviews, encouraging more explanatory material to substantiate the reviewers' opinions.

Before you write your short paper, look at the form reviewers will be filling out when they evaluate it (see the corresponding file which we will provide in this directory). Try to imagine how a reviewer might answer each question concerning your short paper: Do you like the result? If not, how can you fix the short paper to get a better response?

Reviewers are asked to summarize the short paper. Can your short paper be summarized? Is it easy to extract this summary from it? The best place to influence this review question is in the abstract; if you can't write a concise summary of your short paper, how can you expect a reviewer who may not be familiar with the work to do so?

Another question is, "What is new and significant about the work you report. The whole point of having conferences like CHI is to provide a forum for sharing leading-edge ideas and techniques. This doesn't mean that you need to invent an entirely new way of doing or thinking about something. Instead you might offer instead an interesting new twist on some well-established principle, finding or technique. But when you write your short paper, make sure you can answer this question for yourself, and make sure you convey this to your readers.

Other categories emphasize soundness. This means you need to provide a rationale for the work you've done -- whether that consists of arguments for why data was collected or analyzed in a particular way, why a system was designed and/or implemented the way it was, why a theoretical argument makes sense, or why some method is appropriate for particular tasks. Connecting with prior work is key here, as you must make clear how you are building on but going beyond what has already been done. (Of course, given the page limit, you must also be careful not to spend too much time discussing the work of others -- no one said this would be easy!) If this rationale isn't there, reviewers will have to construct their own. This is risky. They may get it wrong. Worse, they may be unwilling to develop it at all and just dismiss the work.

Finally, some categories emphasize the extent to which your work will be useful to the CHI community. It's not enough to just make a good argument, you need to persuade the reviewers that they (and the CHI community at large) should CARE.

The usefulness criterion is especially important for experience oriented work, because these short papers often do not present new technology or concepts, but rather reflect on experiences applying some existing idea or method. For these short papers, you must persuade reviewers that your "lessons learned" will be useful to other practitioners. If you just describe a process you went through, reviewers won't grasp what you learned. If your reflections describe things your reviewers think practitioners already know, they're not going to care very much. If you describe some interesting insights but don't make clear how you arrived at these insights, your reviewers might be intrigued but they won't be convinced that practitioners can apply what you've learned.

Reviewers are also going to care whether your work is likely to stimulate further work in the field. Will it excite people who hear or read about it? Will they go home with great new ideas about how to enhance their own work? This doesn't mean that accepted short papers must be flashy or entertaining. But if presentation of your material is going to have people snoring in their chairs, then CHI is probably not the best venue for it. The best short papers are those that capture the imagination of attendees and promote a background of conversations.

Writing quality is important. Reviewers have expectations about how technical papers should be written. They usually expect to see an introduction to the problem, some method or argument applied to the problem, and some discussion of what conclusion to take away. Relevant related work should be discussed. Make sure your short introduction makes clear who the intended audience is, e.g., software engineers, graphics designers, HF engineers, and write your arguments accordingly. Do not use overly flowery or complex sentence constructions; reviewers will not be happy if they have trouble understanding your prose! Remember that English may not be the narrative language of some of the reviewers and readers.Remember also that we will be judging whether to accept your short paper as-is, so leave yourself plenty of time to write and then refine the short paper.

Reviewers are asked to assign a numerical rating summarizing their overall recommendation concerning your short paper. The discussion questions are intended to prepare the reviewer to make this judgement, and you should see a good correspondence in the comments provided and this numerical rating. However, reviewers will vary in how they react to different aspects of the short paper -- some will respond mostly to writing quality, some mostly to originality, and so on. That is why we have each short paper evaluated by multiple reviewers.

Conclusion

We wish you good luck in submitting your submission in time. We hope you will have great fun in presenting your work and enjoy to contribute to the Common Ground of CHI96! If you have any questions do not hesitate to contact us .