TextMed: A Multi-Agent System with Reinforcement Learning Agents - - PowerPoint PPT Presentation

TextMed: A Multi-Agent System with Reinforcement Learning Agents for Biomedical Text Mining

Michael Camara Janyl Jumadinova Oliver Bonham-Carter September 9, 2015

Big Data

Biomedical Research

◮ PubMed: U.S. National Library of Medicine free search engine ◮ 24 million records (abstracts and citations) ◮ Annual growth rate of 4%

Text Mining

◮ Text summarization ◮ Document retrieval ◮ Document classification

Text Mining

◮ Text summarization ◮ Document retrieval ◮ Document classification ◮ Information extraction

Preprocessing: Lister

• 1. Lister downloads and decompresses data
• 2. Keyword used to obtain relevant abstracts
• 3. Abstracts divided into datasets
• 4. Agent assigned to each dataset

Preprocessing: Abstract Creation

• 1. Lister downloads and decompresses data
• 2. Keyword used to obtain relevant abstracts
• 3. Abstracts divided into datasets
• 4. Agent assigned to each dataset

Preprocessing: Dataset Creation

• 1. Lister downloads and decompresses data
• 2. Keyword used to obtain relevant abstracts
• 3. Abstracts divided into datasets
• 4. Agent assigned to each dataset

Preprocessing: Agent Allocation

• 1. Lister downloads and decompresses data
• 2. Keyword used to obtain relevant abstracts
• 3. Abstracts divided into datasets
• 4. Agent assigned to each dataset

TextMed: Parsing

• 1. Scan through each document with keyword

TextMed: MeSH Keyword List

• 2. Obtain keyword from MeSH (Medical Subject Heading) list

TextMed: Match Found?

• 3. Iterate through list until match found

TextMed: SentiStrength

• 4. Perform sentiment analysis on keyword match

TextMed: SentiStrength (cont.)

Sentiment Analysis Example: ”The penicillin successfully treated the condition, but the patient complained

• f severe side effects afterwards.”

TextMed: SentiStrength (cont.)

Sentiment Analysis Example: ”The penicillin successfully [+3] treated the condition, but the patient complained

• f severe [-2] side effects afterwards.”

Sentiment Score = [+3] + [-2] = 1

TextMed: Reinforcement Learning

• 5. Perform reinforcement learning:

TextMed: Reinforcement Learning (cont.)

• 1. Give command
• 2. Dog performs an action
• 3. Give treat if action matches

command

• 4. Dog tries to maximize treats

TextMed: Reinforcement Learning (cont.)

• 1. Provide list of possible actions
• 2. Agent performs an action
• 3. Agent receives reward based on how

sentiment changes Rk =

N

• i=d

|gsk − lsk,d| N

• 4. Agent tries to optimize reward for

next time

TextMed: Continue Parsing

• 6. Continue parsing all keywords, then begin next document

TextMed: Multiple Agents

• 7. Multiple agents working simultaneously

Experimental Setup

◮ Three primary datasets used for experiments ◮ Each dataset obtained using different keywords with Lister

program and PubMed database

◮ Similar pattern of results for each

Alzheimer’s Dataset: Reward Data

◮ Smaller reward = more optimal, less sentiment fluctuation ◮ Initially high reward, becomes smaller over time

Alzheimer’s Dataset: Local Sentiment vs Global Sentiment

◮ Sentiment before learning ◮ Highly variable throughout all

documents

◮ Sentiment after learning ◮ Variable at beginning,

stabilizes near end

Proximity Parameter

Keyword = penicillin. The penicillin successfully treated the condition, but the patient complained of severe side effects afterwards.

Proximity Parameter

Keyword = penicillin. Proximity = 1: The penicillin successfully [+3] treated the condition, but the patient complained of severe side effects afterwards. Sentiment Score = [+3] + 0 = 3

Proximity Parameter

Keyword = penicillin. Proximity = 2: The penicillin successfully [+3] treated the condition, but the patient complained of severe side effects afterwards. Sentiment Score = [+3] + 0 = 3

Proximity Parameter

Keyword = penicillin. Proximity = 3: The penicillin successfully [+3] treated the condition, but the patient complained of severe side effects afterwards. Sentiment Score = [+3] + 0 = 3

Proximity Parameter

Keyword = penicillin. Proximity = 13: The penicillin successfully [+3] treated the condition, but the patient complained of severe [-2] side effects afterwards. Sentiment Score = [+3] + [-2] = 1

Alzheimer’s: Proximity/Reward Heatmap

Future Work

◮ Optimize SentiStrength for biomedical texts ◮ Modify reinforcement learning algorithm ◮ Incorporate data from multiple databases ◮ Incorporate data from medical records ◮ Compare to other systems

Thank You:

◮ Professor Jumadinova ◮ Oliver Bonham-Carter ◮ Dr. Michael Thelwall ◮ Dr. Barbara Lotze Research Fellowship Fund