Understanding Web Search Behavior for Analytical Tasks Joann Keyton - - PowerPoint PPT Presentation

Understanding Web Search Behavior for Analytical Tasks

Joann Keyton and Grant Harned Department of Communication April 20, 2016

Across Analyst Domains

• Online searching is key to managing

uncertainty in fulfilling task demands

• Seeded discovery problems
• instructions given to find content
• Unknown-unknown problems
• instructions given to identify factors, activities,

events that may present as anomalies, abnormalities and future threats

Uncertainty Management

• Uncertainty reduction theory
• (Berger & Calabrese, 1975)
• Anxiety-uncertainty management theory
• (Gudykunst, 1985)
• Uncertainty management theory
• (Brashers, 2001; Brashers et al., 2000)
• Uncertainty = inability to predict one’s

attitudes, behavior, and feelings

Why tasks are uncertain

• Information is often fragmented and

distributed among sources and locations

• The best information may be among the many

(and nonobvious) sources of online information

• Analysts work on complex problems for which

inferences about short- and long-term future events are made

LAS Update

Anticipatory Intelligence Tradecraft

• Anticipatory Intelligence = approach to

assessing the future

• Covers the three broad tasks
• Exploring the distant future for weak signals of

change in major conditions

• Assessing how dynamic issues may evolve into

trends and impact US interests

• Analyzing the probability that specific events will
• ccur

How this translates to online searching

• Users have some control or potential to

manage the breadth and depth of the information-acquisition process

• Information seekers have the opportunity to

choose

• search terms
• websites and pages viewed
• when a search begins and concludes

Parterning with LAS

Searching behavior is influenced by biases and heuristics

• least amount of effort
• confirmation bias
• strategies for finding and managing the

volume

• technical and authoritative evaluations =

assessments of authority and accuracy

Parterning with LAS

Research Questions

• 1. Given an unknown-unknown task as

motivation, how do individuals vary in their

• nline searching activity?
• 2. What is the distribution of dwell time?
• 3. What is the relationship between the number
• f sites visited and the number of sentences

participants produced in their written reports?

• 4. What is the relationship between online search

behavior and confidence in decision choice?

Parterning with LAS

Dwell Time

• Length of time a visitor spends on a page
• time per page is indicator of greater levels of

systematic processing

• most significant indicator of document relevance
• Web users read quickly, the distribution of

dwell time is based on a model of system failure

Parterning with LAS

Weibull distributions

• Positive and negative aging Weibull

distribution

• the longer the component has been in service, the

more likely it is to fail/the less likely it is to fail

• 99% of Web pages have a negative aging

effect

• People use a 10 second view to make a

decision if the page is useful/relevant

• At 30 second mark, users tend to stay

Parterning with LAS

Method in a

• 51 experimental sessions
• 70 min
• Pretest
• Training search
• Task 1: search and write draft
• Discussion: talk about process or content
• Task 2: search and finalize report

Parterning with LAS

Participant search behavior in T1

• M keywords per search during = 5.28
• SD = 3.04; min/max = 1/19
• Total of 2,586 searches
• M = 19.98 searches per user
• Used 13,654 keywords
• M = 104.49 keywords per user
• 99.92% participants used Google as search

engine

Parterning with LAS

Parterning with LAS

Domain Type M Time per User Total Web Pages Visited

Search Engine 185.30 2557 Presidential Candidate-General 260.19 1233 General Information 174.51 1071 Candidate 49.14 400 Political News: Nonpartisan 28.57 173 News 29.19 169 Political News: Conservative 42.76 135 Social Media 23.87 113 Political News: Liberal 23.67 89 Government 17.11 95 Political Party 3.88 56 Images 2.68 49 Political News: Libertarian 6.85 48

RQ 1: Where did they go?

RQ 2: Dwell time distribution

• Weibull analysis on the six most frequently

used domain types

• political candidate sites β = 0.73
• social media β = 0.75
• candidate portals β = 0.77
• general information β = 0.85
• search engine β = 0.85
• news β = 0.91

Parterning with LAS

Parterning with LAS

Interpreting dwell statistic

• Shape parameter in the range 0 < β < 1
• user will leave a particular web page monotonically

decreases with time

• negative aging phenomenon
• pages relating to political information across a wide

range of domain types

• Rate of web page abandonment decreases more

slowly for news and information sites than for social media and political candidate sites

Parterning with LAS

RQ 3: Number of sites visited and number of sentences?

• Participants wrote a total of 1302 sentences in

the draft reports

• M = 10.25, SD = 4.18, min/max = 3/27 per

participant

• number of URLs visited = proxy for

information gained

• more sentences written = proxy for cognitive

load.

• r = -.22, p < .05; search and writing trade-off

Parterning with LAS

RQ 4: Online search behavior and confidence?

• No relationship between number of URLs

visited and confidence in the decision as reported in the report

• r = .04, ns

Parterning with LAS

Discussion

• Participants could manage uncertainty by how
• they structured their online searching
• created and reported their inferences in writing
• r both
• Not one participant found the Federal Election

Commission Form 2 website

• Cognitive misers
• only 18% users view 2nd page of search results,

3% viewed 3rd, and 1.5% viewed beyond

Parterning with LAS

Takeaways 1

• Rate of web page abandonment decreases more slowly

for news and information sites than for social media and political candidate sites

• According to Liu et al. (2010), participants were more harshly

screening candidate portals, social media, and political candidate

• Overwhelming presence of Google as a search tool
• Google as an authority?
• The lack of reliance on the FEC website is (insert your

favorite word)

Parterning with LAS

Takeaways 2

• Search strategy/query is contingent to some degree
• n the type of task a participant is asked to perform
• Participants rarely visit beyond the first page of

search results

• 18% viewed 1st page
• 3% viewed 3rd page
• 1.5% viewed 4th or beyond.
• Confidence seems somewhat arbitrary in non-expert

users

Parterning with LAS

And there’s much more to do

• Linguistic analysis
• Draft and final reports
• Discussions
• Interaction analysis
• Discussions
• Sequential analyses of types of talk

Parterning with LAS