A Multi-dimensional Approach to the Study of Online Annotation

Presenters:
Rick Kopak, Assistant Professor, School of Library, Archival and Information Studies, UBC
Chia-Ning Chiang, PhD Candidate, School of Library, Archival and Information Studies, UBC
Thursday, July 12, 2007
9:40 AM - 10:40 AM in SFUHC, Canfor Policy Room

Abstract

Presentation (Powerpoint/PPS file)

Rick Kopak’s major research interest is in human information interaction focusing on how semantic or meaning cues derived from the use of information can be used to improve navigation in distributed information environments. He is currently investigating the role of annotation and typed linking in providing useful navigational aids in discursive digital texts.

Chia-Ning Chiang's research focus is on mediated user-interface design in human-computer interaction.

A Field Study of Online Annotation

Rick Kopak and Chia-Ning Chiang study online annotation using a multi-dimensional approach. They carried out a field study and analyzed the resulting data to improve system design.

Their study seeks to increase critical engagement of the users in the online reading environment. Using the Open Journal System (OJS) as the reading environment, they have a prototype of reading tools built on the platform. With the tools, they are aiming for the users to perform active reading, in which they are not only reading but also writing. Readers of printed books or papers have been doing this naturally, for example when they annotate on the margin of the books.

Annotation and Linking

Aim: to explore behavior, cognition, and social dimensions
Multi-dimensional: form, function, and role (audience)
Context: to create a digital annotation environment to investigate the dimensions

Prototype of Annotation and Linking in OJS


The audience had a chance to see some screen captures of the prototype. The annotation area, where users type in their annotations, is on the right side of the article’s text (the margin). The audience also saw a demonstration on how it works. A user can highlight an area of the text, select from a drop down menu the type of link to create (called a link type), and then typing a note on the margin. A user can also create a link between two articles in OJS, which points exactly to the text of interest.



Evaluation of the Study

The 15 participants of the study, comprising of graduate students and faculty members, were asked to do general (go through each article and see what they are about) and directed reading task (looking for a specific aspect/meaning from the article).

The study uses multi-dimensional approach to investigate online annotation:

• the relative value (combining frequency of use and user rating) of online annotation according to combinations of form (behaviour) and function (cognitive state) in relation to the prospective audience for the annotations (i.e. private, work group, and larger public)
• the patterns of behavior and cognitive states of readers while annotating

Results

The study looks at a number of aspects:

• Happy highlighters or meager markers - result is all over the map, no pattern of distribution
• Text annotation by task (do people create notes more or less when the tasks are different) - no pattern discovered when comparing results of general vs directed reading
• What was highlighted - general reading vs directed reading
• Links and link typing - general reading vs directed reading

Feedback and Wish List from the Study Participants

About difficulties encountered when using the tools:

• highlighting is easy but the next step is not too obvious
• note-writing widgets wanted

About links and link typing:

• need more obvious link creation and representation
• need to maintain coherence when linking back and forth between articles
• not clear what the link type means

About sharing annotations:

• selectively share (when it is meaningful or helpful to others, they want to share the annotation)
• enable revision
• enable others to comment (to encourage dialogue and give information via linking)
• search capability (search public annotations by topics/functions or by section of the article)

Wish list:

• Toggle between viewing a clean copy or an annotated copy
• Turn on/off annotation easily
• Distinguish own annotations from other people's annotation when in public view mode
• Highlight (icons to mark the text)
• Note-writing (a place for writing long notes, drag and drop text, content description)
• Linking (one to many links, live link to non-OJS Internet resources)

Future Directions

• Separate workspace: personal and shared
• Support reading tasks: skip complex function for general reading, more complex ones for directed reading
• Integration with reading tools: look-up terms, check Internet sources

Discussion

Among the questions asked by the audience, one person put forward an interesting idea of adding a feature to the system so that users can look at and follow one person’s annotations on all articles that the person have read (a la del.icio.us).

Commentary

Online annotation and linking enable scholarly communications to take the most advantage of the online format. With this kind of reading tools available, readers of online scholarly journals can find out what other readers think of a particular journal article and can even follow the discourse around a certain scholarly enquiry in a new way made possible by the online reading environment. They become actively engaged with their readings as the tools allow them to "write on the margins" and link the reading materials in ways that make sense to them.

In the spirit of increasing the circulation of knowledge, the result of the study would continue the tradition of scholarly publishing that Henry Oldenburg has started with his journal the Philosophical Transactions.

Reading Research Online: Ways of Enhancing Scholarly Reading Environments

Presenter: Peter Arthur, University of British Columbia
Thursday, July 12, 2007
9:40 AM - 10:40 AM in SFUHC, Canfor Policy Room

Abstract

Peter Arthur is Director, Centre for Teaching & Learning, UBC Okanagan and currently pursuing his doctorate with John Willinsky at UBC. His research is assessing what contextual elements of an online “Reading Tool” support the quality of reading for readers of online research.

Review of the Literature

Peter Arthur presented a review of the literature on online scholarly reading, ranging from 1987 to 2007. His aim was to come up with recommendations to create an online reading environment that is conducive to maximum knowledge acquisition.

During the time, technology has progressed, but some problems still exist with reading online. Some people would have no problem with reading on screen and even prefer this way of reading, but some people would prefer to print the online reading material first and then read it on paper.

Hypertext and System Experience

As online reading materials are hypertext, he presented the advantages and disadvantages of hypertext. Users of hypertext have the advantage of having greater control over the reading materials compared to users of print materials. However, users of hypertext can experience disorientation (lost in hyperspace, navigation problem, etc). They are also faced with cognitive load because they have to make decisions as they read through, such as whether or not to search online for further information on the reading specifics.

Online reading presents challenges for novices, or readers with low domain knowledge. On the other hand, expert online readers (high domain knowledge) are better able to acquire knowledge. System experience is a factor causing the difference.

Reading can be linear or nonlinear. In the print environment, reading is linear as users read from start to finish. It is different in the online environment, where reading hypermedia and hypertext is nonlinear as users encounter many potential pathways and have greater user control.

What the Research Says

Research about navigational aids has been trying to enhance the online reading environment by the following means:

• Hierarchical map – helps readers understanding the structure of the online material
• Preview links – shows a snapshot of the destination website (pop-ups) menu when you hover over the link
• Searching tools – helps readers looking for more details on the reading

Research has also shown useful features for readers in the online reading environment:

• Consistency - for example, links are always underscored and blue, so reader can intuitively tell the links apart from the text
• Breadcrumbs - showing reader where they are
• Visited links – on the browser, links will change colour after they are clicked (or visited)

Design Considerations

Peter then brought up some design considerations for the online reading environment:

Who is the target audience? Who are going to be the readers?

It matters because the way novices read online would be different from the way expert readers do. Novices require different tools or support than experts do.

Experts (high domain knowledge) would want:

• greater level of control
• finding specific information rather than general or basic information

Novice (low domain knowledge) would want:

• hierarchical maps
• hierarchical topology
• not too much linearity (this is causing too much cognitive load and disorientation)

Discussions

Peter’s presentation was followed by some interesting discussions with the audience. Asked about usability testing on the reading tools in the Open Journal System (OJS), he mentioned Mia Quint’s study with students as participants, in which half of them are given access to reading tools and the other half are not. The study is still in progress, so the result is not available yet.

Another person asked about research regarding orientation for novice readers, but Peter did not recall any study on teaching novices how to navigate or orient themselves. He mentioned that in Mia Quint’s study, the students are not taught about how to navigate the online reading environment they are using.

A question about deep reading versus scanning when people read online also came up. However, Peter did not recall any empirical study on this matter, but acknowledged existing discourse on it.

Finally, someone in the audience from Australia put forward a comment that some users do not even know what reading tools are, so customization of the reading tools are necessary. To this end, Peter was in agreement with the suggestion.

Commentary

Designing an online reading environment that works well for its users is complicated. Design considerations vary from one type of reading materials to another. Reading novels online for pleasure would definitely be different from reading scholarly papers online. Peter Arthur focuses his study on the scholarly reading environment and attempts to produce research-based recommendations for designing systems that allow maximum knowledge acquisition.

As stated in the Budapest Open Access Initiative, the Internet has allowed electronic distribution of scholarly journals and the open access initiative is making them freely available online. Thus, readers of these journals are likely to read research online. Peter’s study contributes to making the online reading experience better for them.

His distinction of novice and expert readers with regards to design requirements takes into account that opening access to research and scholarship means widening the readership to include not only scholars but also readers with little or no prior knowledge of the reading materials, as suggested by John Willinsky.

Understanding both types of readership and accommodating their needs into the system design would create a better system experience, which will retain and increase readership. In turn, this will increase the circulation of knowledge, continuing the spirit of scholarly publishing that Henry Oldenburg has started with his journal the Philosophical Transactions.