UE Process / Article 6. May 2008, HCI Vistas Vol-IV Author: Pushkaraj Mirajkar

In most of the product/ service based organizations, technical writers are based offshore. They collaborately work with the product development team and create documents like User Manuals, Setup Guides, Quick-Start Guides, In-built help in software’s, Online Documentation, which accompany the final product.

Technical writers usually create such documents which contribute to assist the user for any product they use.

Documentation is more of an “After Thought” process; and in many respects documentation tries to compensate for the lack of intuition in a product.

A critical point to use these technical documents is when the users face a Cognitive Friction (as mentioned by Alan Cooper in his popural book The Inmates Are Running The Asylum), while interacting with the product and they are now forced to refer to these documents to achieve their tasks.

Challenges faced offshore

Technical writers are involved at a very late stage in the product-development life cycle due to which the quality of the documentation suffers.

1. Time bound deliverables

Due to late involvement; technical writers are bounded to create documents with a quick turn around time.

2. Closely associated with the development team hence they are

a. Dictated by the product development team

b. Biased and tend to use technical jargons

c. More focused on the functions and features rather than the user’s tasks

3. No access to the actual users

Technical writers usually don’t have an in-dept knowledge about the target audience for whom they are catering to hence the documentation-

a. Does not speak the user’s language

b. Reflects a lot of cultural issues

Technical writers can overcome these challenges by

• Getting the organization to have Technical Writing as a planned activity
• Involving them in the early stage of the product life cycle
• By creating User Focused Usable documents, which is the most critical activity Testing documents of a product with actual users is key to create Usable documents.

Conducting a Usability Test for Technical Documentation

As Technical Writer’s are based offshore and they need to connect with the actual users worldwide to conduct a Usability Tests for their documents. Tools and Technology can be used to bridge this gap. They can either test their documents-

• Remotely (offshore) OR
• One-on-One with actual users

Remote Usability Testing

• Using WebEx a web/video conferencing tool documents could be shared online with the representative users. Phone/ Skype or any other chat messenger could be used as a conversation medium along with WebEx.
• Another medium to test the users is installing Morae software on the client’s machine and invite users for the test.

Advantages of Morae

• Using a Webcam, users’ expressions can be captured
• Users actions can be recorded and monitored

One-on-One testing with actual users

This is the best way to test your documents with the actual users, as you can physically see users stumble using the products and refer to the documents. A scenario-based testing can be conducted. E.g. for a cell phone, a user could be using his handset while he travelling, or office etc. We could use these real life scenarios and test the efficacy of the technical documents. Using any of the methods above, a Usability Test can be conducted with the actual/representative users. Mentioned below are the details of how to conduct a Usability Test.

With whom should you conduct your usability test?

Usability testing of documents is conducted with user’s who are likely to purchase and use your product. So for e.g. in a library system, you must test your documents with librarians.

Why do you do Usability Testing?

The objective of usability testing is to identify specific concerns and goals the user might face while using the documents for the product being developed. In a typical Usability Test you want to-

1. Choose 3-4 critical tasks
   The documents which you create could be very elaborate and testing the entire document is not feasible. You can choose 4 to 5 most critical tasks which are important from a user’s perspective and where YOU think the user would likely refer to the document while performing the task.
2. Create a Test Plan
   In usability testing the most critical step is to develop a plan for the test. The plan should reflect what you are going to test with the user and what kind of data you are looking for. Below is a table which can help you design a test plan.

Purpose of Testing	What are the concerns, questions, and goals in the test on which you would focus?
No. of Participants	How many users would you test?
Scenarios	At what instances the user would refer to the document while performing the task
Pre-test and Post-test Questions	What will you ask the user at the beginning and end of the test session?
Data to be collected	What would you want to measure?

3. Benchmark the Test

Why should you benchmark a test?

You must know what would be a reasonable and realistic time estimate, errors etc. that a user will tolerate before they abandon or get frustrated with their task.

What will you benchmark?

• Ability to complete tasks
• Time taken to complete a task successfully while referring to the document
• No. of errors the users encounter while referring to the document
• Level of satisfaction while referring to the document

How do we set benchmarks for the test?
Typical benchmarks are set on the users Task and his Goals.

E.g. Task: By referring to a user manual how can a user send an email by using his
newly purchased mobile phone.

Goal: 85% of all participants should be able to complete this task within 2 minute
(this is your benchmark assumption)

You could also study the competitor’s product by using the same usability benchmarks and get a comparative idea.

4. Conduct Test on the document

• How many Tests should you conduct and How Many Participants should you test?
  The number of participants is frequently dictated by the budget and time allotted for the test. The best results come from testing no more than 5 users and running as many small tests as you can afford.

If the no. of users exceeds beyond 5 your observations become repetitive hence there is no more value added to your test results.

• Conducting the test
  You will now conduct the test with the target users by narrating to them the scenarios and asking them to perform the task. In this process you must ask the users to Think-Aloud while they perform their tasks. You can record the data by either video/audio recording of the test session or by taking notes.

Points to keep in mind while conducting the test?

• Making the participant feel comfortable
  The most important thing to remember is to take good care of participants. Make them comfortable.
• Telling the participant that you are testing the document not them.
  Users are usually intimidated with the term Testing. So participants need to be informed that you are testing the Document and not them. In fact you can approach it as help us evaluate the document.
• Staying Neutral with the participant
  You conduct a test to listen and watch your users, not to demonstrate, not to train nor to teach; You must not say either negative or positive things about the document. You should only encourage the participant to think aloud. If the participant asks a question, reply by saying something like “What do you think?” or “I’m interested to know what you would do.”
• Taking good notes
  If an additional note-taker can be present at the test, they should capture what the participant does in myriad details as possible. When participants do not take one of the expected pathways, it is most useful to note what exactly they would do.

What do you gain by following this process?

• Learn about the discrepancies in your documentation
• Apply lessons learnt in the testing to the rest of the documentation

Key points

• Usable product documents can be planned by the organization by having the Technical Writers involved in the very early stages in the product development cycle
• It is important that technical writers test their documents with actual users so that they can have a higher confidence level that the documents which accompany the products are €œeasy-to-use€ and which speak the users language.
• They also need to sit remotely from the development team, which would help them write the users language rather than influencing the document with technical jargon

Other ways of contributing usability into the product

While Technical Writers study the product for which they plan to create a document, they could also note down points which they think could be discrepancies in the product such as

• Terminologies which the user may find difficult to understand
• Any content which is too elaborate and difficult for the user to read
• Error messages that are difficult to understand particularly if they are expressed in technical jargon rather than in plain English
• Headers/ Labels that may not be intuitive to the content
• These points could be sent to the client or to the development team in a documented memo that could be used in the present or a later version of the product.

References

1. From the actor to the script: A Human Factors Case Study Iyengar, J. Human Factors in Practice, 1999
2. Why you need to test with 5 users? Jakob Nielsen’s Alert box, March 19, 2000
3. Usability.gov
4. The art of usability benchmarking
5. How many users do you need to test?

Pushkaraj Mirajkar, a Computer Engineer by profession, started his career into the space of web, and later moved into the world of creativity. He is a certified professional in User-Centered analysis, design and Usability testing from the Ohio University, USA.

In the field of Usability and User Interfaces, he has catered to various challenging domains including e-Commerce, Health, Lifesciences, Collaborative space, Finance etc. This involved product analysis, UI Design and usability testing. He has gained valuable skills in understanding designs for ease of use, and related ergonomic studies.

© Copyright