How do screen reader users use search interfaces differently than sighted users searching visually
Graphical User Interfaces afford a visual information seeking model, which may not be very suitable for non-visual or auditory modalities when using assistive technology. Yet, there is a lack of research to understand behavior in complex web searches when using screen readers.
1. This independent student research was conducted with the help of the Gavin Lew fund at DePaul University under the advisement of Dr. Cynthia Putnam.
2. Recently, this research was selected to appear as a late breaking publication at ACM CHI 2020.
NOTE: Collaboration organized by the Steans Center at DePaul University to explore Service design for community service - Coursework for Design Ethnography, group of 3 graduate students.
People conduct complex information searches at all times, aiding decision making and problem solving in their day to day tasks. As search interfaces become more complex and visually directed, potential accessibility concerns hinder information seeking when using screen readers.
Most research in accessibility highlights usability concerns and browsing behavior within the context of "navigability of web pages". It is unclear how screen reader users interact adaptively with the search environment in an auditory interaction to maximize gain of desired information.
need for research, and our questions
Need for better understanding of a non-visual foraging model
Information foraging theory states that "people will modify their strategies, or structure of the environment, to maximize their rate of gaining valuable information". The decision to pursue a source of information (scent) is based on its perceived cost of access and value.
Many users who are legally blind use assistive technology such as screen readers. Screen reader users use keystroke commands to select HTML page elements to receive auditory feedback. The nature of non-visual web access is sequential and auditory.
There are increased costs (time, effort and decision making to access information).
Cognitive overload from excessive auditory information.
Multiple usability and accessibility issues due to missing semantic HTML tags
Prior studies place screen reader users' adaptive browsing strategies in context of navigability of websites. We sought to understand:
What information behaviors do screen reader users employ to maximize information access?
How is this information behavior different from sighted people using the visual web for search?
research plan and methodology
We recruited ten sighted web users and ten screen-reader users
Ten sighted and ten screen reader users (evenly distributed between the ages of 18-65) were recruited from the university, and an institute offering computer classes for people with vision impairment. All participants conducted a variety of information seeking tasks regularly as part of their work or academics, and reported their extent and frequency of internet use in a screener. Non-sighted participants were screened for less than 20/200 visual acuity, and did not use screen magnification. View some recruitment considerations in the recruitment documents below.
Data collection using task-based systematic observations
In a 40-minute session, participants completed a scenario based task, where they were asked to gather information on graduate schools in psychology (towards a potential discussion with their academic advisor). They recorded desired information about schools in a text document while conducting their search. Participants' information search behavior was captured in the form of screen recordings.
Data logging from screen recordings of participant behavior in tasks
Participants' screen recordings were transcribed in the form of logs that captured behavior around query formulation, search result exploration and link following to reach desired information. These transcription logs captured the following instances (and their attributes), as they occured in the search activity :-
Type of query - informational, navigational or transactional
Any accessibility concern in query formulation
Any adaptive behavior in query formulation
Search result exploration
# results explored
Method of exploration - headings, descriptions, url
Any accessibility concern in search result exploration
Any adaptive behavior in search result exploration
Any revisit to a set of search results
Capturing links followed from a search result
Adaptive browsing within the webpage
Reasons for site leaving
Any revisits to the webpage
CAPTION: we transcribed and coded screen recordings of participant activity
Using our data, we created web behavior diagrams to capture participants' actions in context of the entire search session, and conducted quantitative analysis of querying and link following.
CAPTION: This web behavior diagram shows one participant completing the search task using a screen reader. like many other participants, tina anchored her search process on the information scent gained from the first search result exploration
sighted participants - extensive querying (5)
sighted participants - extensive link following (5)
findings | information seeking behaviors
Sighted participants either demonstrated extensive querying or extensive link following
Extensive Querying - 5 sighted participants used a combination of multiple queries to reach desired information
Extensive Link following - 5 sighted participants followed links to reach desired information
Both these groups used methods that were comfortable to their browsing style to reach desired information
caption: these are web behavior diagrams of 2 sighted participants. on the left - extreme link following, on the right, extreme querying
Screen reader users demonstrated neither behaviors. Instead, they adaptively maximized gain from their initial searches by returning to that page. In order to explain this, let's discuss search result exploration behavior of initial searches.
Non sighted participants employed adaptive navigation methods for exploring search results.
On an initial scanning navigation using Headings, if the search results were relevant, they conducted an exhaustive exploration.
Exhaustive exploration included reading relevant/ skipping irrelevant headings, reading descriptions and sometimes url for relevant search result headings.
caption: in screen reader use, exhaustive search result exploration included one pass of heading navigation, and a second pass of skipping irrelevant results, and detailed exploration of relevant results
Why might screen reader users have demonstrated less querying or link following?
Search result exploration is a costly process, and perhaps, that cost of exploration wasn't justified for multiple search result explorations. Scanning navigation was common when exploring results of navigational searches.
They chose a careful selection of high-scent links or combined few informational and navigational queries
Perhaps, extensive link following led to a loss of sense of context in the browsing process
adapted mental models | Learned behaviors
screen reader users maximized gain from their initial searches. they returned to certain scent-heavy pages to use the scent gained to further their information search
extensive querying or link following by sighted participants
inferences for systems and interactions
Through these findings and visualizations, we can argue that adaptive behavior is exercised not only in light of inaccessible content on standalone web pages to escape. Instead, screen reader users illustrated learned behaviors with respect to an understanding of how the system, and the interaction works.
How might we aid different search result exploration methods in screen reader use?
Informational searches included both scanning + comprehensive exploration techniques
Navigational searches included only scanning exploration
Screen reader shortcuts to aid both types of navigation
How might we improve information scent in informational searches - varied results?
Transcoding solutions could explore highlighting relevant keywords in Headings, and descriptions, highlight relevant meta information(such as order of results), to allow for better scanning on a revisit.
Bookarking relevant results to return to on a revisit of search results page.
How might we improve information scent in navigational searches - results leading to a web address?
Transcoding solutions can clarify key differences between similar search results (keywords) to avoid following undesired links.
Quick scanning of a gist of keywords in heading navigation