Modelling in fm uenza-like illness using online search Vasileios - - PowerPoint PPT Presentation

@lampos lampos.net

www

Modelling infmuenza-like illness using

• nline search

Vasileios Lampos

Computer Science, UCL

Mapping online search to fmu estimates

2004 2005 2006 Year 2007 2008 2 4 6 8 10 ILI percentage 12

• Complement traditional syndromic surveillance
• timeliness
• broader demographic coverage, larger cohort
• broader geographic coverage
• not afgected by closure days
• lower cost
• Applicable to locations that lacl an established

healthcare system

Why estimate fmu rates from online search?

Google Flu Trends — discontinued

— popularising an established idea

Ginsberg et al. (2009); Eysenbach (2006); Polgreen et al. (2008)

Google Flu Trends — why did it fail?

• Jan. '09
• Jan. '10
• Jan. '11
• Jan. '12
• Jan. '13

0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09

ILI rates

CDC Google Flu Trends

rsv — 25%

fmu symptoms — 18%

benzonatate — 6% symptoms of pneumonia — 6%

upper respiratory infection — 4%

ILI rateβ0β1 ⨉ Q, where Q is the average query frequency

Google Flu Trends — why did it fail?

• non-ideal query selection, model simplicity
• inappropriate evaluation (less than 1 fmu season!)
• Jan. '09
• Jan. '10
• Jan. '11
• Jan. '12
• Jan. '13

0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09

ILI rates

CDC Google Flu Trends

• Treat single search queries as distinct variables
• Model nonlinearities

1 2 3 4 5 6

Frequency of query 'how long is flu contagious'

• 7

0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08

ILI rates Raw data Linear fit

frequency of “how long is fmu contagious” ILI rate

Multivariate, nonlinear, generative models

• Treat single search queries as distinct variables
• Model nonlinearities
• Model groups of queries that share common temporal

patuerns Gaussian Processes (GPs)
 — distribution over functions that can explain the data
 — allow some room for model interpretability
 — can model uncertainty

Multivariate, nonlinear, generative models

Correcting the defjciencies of Google Flu Trends

• Jan. '09
• Jan. '10
• Jan. '11
• Jan. '12
• Jan. '13

0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09

ILI rates

CDC Gaussian Process

• 42% mean absolute error reduction compared to

Google Flu Trends

• .95 Pearson correlation (previously .89) with CDC

Modelling uncertainty

• Jan. '09
• Jan. '10
• Jan. '11
• Jan. '12
• Jan. '13

0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09

ILI rates

CDC Gaussian Process

Combining GPs with autoregression (AR)

• Jan. '10
• Jan. '11
• Jan. '12
• Jan. '13

0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09

ILI rates

CDC Gaussian Process with AR

• 1 week delay in incorporating historical CDC estimates
• 27% mean absolute error reduction over GFT with AR
• 52% mean absolute error reduction over GP without AR
• .99 Pearson correlation with CDC

• Select search queries based on their semantic

similarity to the topic of fmu

• Make this possible by using word embeddings, i.e.

word representations in a common vector space
 — learn them using a corpus of 215 million tweets

Qvery selection based on meaning

• Select search queries based on their semantic

similarity to the topic of fmu

• Make this possible by using word embeddings, i.e.

word representations in a common vector space
 — learn them using a corpus of 215 million tweets

Analogy: A (is to) → B what X (is to) → ? Rome → Italy London → [UK, Denmark, Sweden] go → went do → [did, doing, happened] Messi → football Lebron → [basketball, bball, NBA] Elvis → Presley Aretha → [Franklin, Ruffjn, Vandross] UK → Brexit Greece → [Grexit, Syriza, Tsipras] UK → Farage USA → [Trump, Farrage, Putin]

Qvery selection based on meaning

• Select search queries based on their semantic

similarity to the topic of fmu

• Make this possible by using word embeddings, i.e.

word representations in a common vector space
 — learn them using a corpus of 215 million tweets

• Combine temporal correlation with semantic

similarity (hybrid similarity) for optimal feature selection

Qvery selection based on meaning

2013 2014 2015 5 10 15 20 25 30 35

ILI rates

RCGP (England) Correlation-based feature selection

Qvery selection based on meaning — Results

• prof. surname (70%)

name surname (27%) heating oil (21%) name surname recipes (21%) blood game (12.3%) swine fmu vaccine side efgects (7.2%)

Examples of spurious selected queries

Qvery selection based on meaning — Results

2013 2014 2015 5 10 15 20 25 30 35

ILI rates

RCGP (England) Hybrid feature selection

• 12.3% performance improvement
• .913 Pearson correlation with RCGP ILI rates

i-sense fmu (Flu Detector)

! fludetector.cs.ucl.ac.uk

@isenseflu

i-sense fmu (Flu Detector)

! fludetector.cs.ucl.ac.uk

• daily fmu estimates for England, publicly accessible
• transferred to Public Health England (PHE)
• its estimates have been included in the two most

recent annual fmu reports of PHE (gov.uk/

government/statistics/annual-flu-reports)

• open source, github.com/UCL/fludetector-flask
• credit to David Guzman for constantly refjning it

@isenseflu

Forecasting fmu rates — Ongoing work

mean absolute error = 2.56 (cases per 100,000) r = .901

• Jan. '15
• Jan. '16
• Jan. '17
• Jan. '18

10 20 30 40 50

ILI rates (per 100,000)

RCGP (England) 3-weeks ahead forecasts (preliminary model)

led by Simon Moura

Forecasting fmu rates (US) — Ongoing work

mean absolute error = 0.33% r = .927

• Jan. '15
• Jan. '16
• Jan. '17
• Jan. '18

1 2 3 4 5 6 7 8

ILI rates (%)

CDC (US) 3-weeks ahead forecasts (preliminary model)

led by Simon Moura

Multi-task learning for fmu

Multi-task learning (MTL) vs. single-task learning (STL)

• learns models jointly instead of independently
• for related tasks is performing betuer than STL solutions
• provides good performance with fewer training samples

Flu models with MTL

• limit performance loss under sporadic training data
• improve accuracy
• of regional models within a country
• across difgerent countries

Modelling fmu across US regions with MTL

surveil-

• -fold. Firstly,

various ge- countries — can to assist eillance

• data. We ex-

multi-task cess, and

• formulations. We use

eriments on health and indicate national mod- absolute reduced

CA OR WA MT ID NV UT AZ WY CO NM TX OK KS NE SD ND MN WI IA IL MO AR LA MS AL GA FL SC NC TN KY IN MI OH WV VA MD DE PA NJ NY CT MA VT NH ME RI AK HI

Region 1 Region 2 Region 3 Region 4 Region 5 Region 6 Region 7 Region 8 Region 9 Region 10

Train 10 US regional models for fmu jointly

MTL across US and US regions

Pearson correlation mean absolute error 0.44 0.88 0.51 0.85 single-task learning multi-task learning

Performance for US — 1 year of training data

MTL across US and US regions

Pearson correlation mean absolute error 0.47 0.87 0.54 0.84 single-task learning multi-task learning

Performance for US regions — 1 year of training data

MTL across US and US regions

Pearson correlation mean absolute error 0.48 0.82 0.59 0.77 single-task learning multi-task learning

Performance for US regions — 1 year of training data
 50% of the data lost

MTL across US and England

Pearson correlation mean absolute error 0.59 0.88 0.98 0.85 single-task learning multi-task learning

Performance for England — 1 year of training data

Why estimate fmu rates from online search?

• oxymoron: healthcare data is


required for training the models!

• Complement traditional syndromic surveillance
• timeliness
• broader demographic coverage, larger cohort
• broader geographic coverage
• not afgected by closure days
• lower cost
• Applicable to locations that lacl an established

healthcare system

Transfer learning for fmu modelling

Main task

• train a model for a source location where historical

syndromic surveillance data is available

• transfer it to a target location where syndromic surveillance

data is not available or, in our experiments, ignored Transfer learning steps 1. Learn a regression model for a source location 2. Map search queries from the source to the target domain 3. Transfer the source regression weights to the target domain

Mapping source to target queries

• Direct translation does not work
• Two similarity components
• Semantic similarity (meaning) using cross-lingual

word embedding representations (Θs)

• Temporal similarity based on their frequency time

series (Θc)

• Joint similarity: Θ = γΘs + (1−γ)Θc , γ ∈ [0,1]

Source: US, Target: France

2008 2009 2010 2011 2012 2013 2014 2015 2016

• 1

1 2 3 4 5

ILI rates (z-scored)

US FR

How similar are their fmu rates?

Source: US, Target: France

MAE = 61.5, r = .835 MAE = 46.8, r = .956 MAE = 34.1, r = .959

Source: US, Target: Australia

How similar are their fmu rates?

2008 2009 2010 2011 2012 2013 2014 2015 2016

• 1

1 2 3 4 5

ILI rates (z-scored)

US AU

Source: US, Target: Australia

MAE = 42.6, r = .7 MAE = 30.3, r = .915 MAE = 22, r = .921

Conclusions

We have shown that we can

• estimate fmu rates from online search
• right modelling approach
• right query selection approach
• utilise multi-task learning to improve models
• transfer models when healthcare data is not available

Future work within i-sense includes

• forecasting fmu rates
• translation of our research to public health solutions

Aclnowledgements

Collaborators: Ingemar J. Cox, Elad Yom-Tov, Richard Pebody, Bin Zou, Andrew Miller, Moritz Wagner, Simon Moura Organisations: Microsofu Research, Google, Royal College of General Practitioners (RCGP), Public Health England (PHE) Funding: EPSRC IRC “i-sense”

@lampos lampos.net

www

References

Key papers from our group

• Lampos, Miller, Crossan and Stefansen (2015). Advances in Nowcasting Infmuenza-like

Illness Rates using Search Qvery Logs. Scientifjc Reports, 5(12760).

• Lampos, Zou and Cox (2017). Enhancing Feature Selection Using Word Embeddings: Tie

Case of Flu Surveillance. Proc. of the 26th International Conference on World Wide Web, pp. 695–704.

• Zou, Lampos and Cox (2018). Multi-Task Learning Improves Disease Models from Web
• Search. Proc. of the 2018 World Wide Web Conference, pp. 87–96.
• Zou, Lampos and Cox (2019). Transfer Learning for Unsupervised Infmuenza-like Illness

Models from Online Search Data. Proc. of the 2019 World Wide Web Conference, pp. 2505–2516.

Other papers mentioned in the slides

• Ginsberg, Mohebbi, Patel, Brammer, Smolinski and Brilliant (2009). Detecting

Infmuenza Epidemics using Search Engine Qvery Data. Nature, 457(7232):1012–1014.

• Eysenbach (2006). Infodemiology: tracking fmu-related searches on the web for

syndromic surveillance. Proc. of AMIA Annual Symposium, pp. 244–248.

• Polgreen, Chen, Pennock, Nelson and Weinstein (2008). Using Internet Searches for

Infmuenza Surveillance. Clinical Infectious Diseases, 47(11):1443–1448.