Pattern Markup-Language Pattern Markup-Language A tool for - PowerPoint PPT Presentation

Pattern Markup-Language Pattern Markup-Language A tool for simplifying data extraction A tool for simplifying data extraction from semi-structured sources from semi-structured sources , Jonathan Baker, Hilton Campbell , Jonathan Baker, Hilton Campbell Jordan Crabtree, David W. Embley Jordan Crabtree, David W. Embley

Many Sites with Genealogical Many Sites with Genealogical Data Data Pattern Markup Language 2 Pattern Markup Language 2

Pattern Markup Language 3 Pattern Markup Language 3

Pattern Markup Language 4 Pattern Markup Language 4

Structural Patterns Structural Patterns Pattern Markup Language 5 Pattern Markup Language 5

Pattern Markup Language 6 Pattern Markup Language 6

Pattern Markup Language 7 Pattern Markup Language 7

Pattern Markup Language 8 Pattern Markup Language 8

Pattern Markup Language 9 Pattern Markup Language 9

Programmer Defined Programmer Defined Regular Expressions Regular Expressions Regular Expression A Pattern Markup Language 10 Pattern Markup Language 10

Programmer Defined Programmer Defined Regular Expressions Regular Expressions Regular Expression B Pattern Markup Language 11 Pattern Markup Language 11

Programmer Defined Programmer Defined Regular Expressions Regular Expressions Regular Expression C Pattern Markup Language 12 Pattern Markup Language 12

Which Relationships Which Relationships Found ? Found ? Death Date Birth Date Given Name Aliases Pattern Markup Language 13 Pattern Markup Language 13

Simple Schema Simple Schema Represents Relationships Represents Relationships Person Birth Death Names Date Date Given Aliases Pattern Markup Language 14 Pattern Markup Language 14

Combine Schema and Combine Schema and Regular Expressions Regular Expressions Person Birth Death Names Date Date Given Aliases Regular Expression A Regular Expression B Regular Expression D Regular Expression C Tree Represented by XML = Tree Represented by XML = PatML PatML Pattern Markup Language 15 Pattern Markup Language 15

Person Birth Death Names Date Date Given Aliases Regular Expression A Regular Expression B Regular Expression C Regular Expression D Pattern Markup Language 16 Pattern Markup Language 16

Person Birth Death Names Date Date Given Aliases Regular Expression A Regular Expression B Regular Expression C Regular Expression D Pattern Markup Language 17 Pattern Markup Language 17

Person Birth Death Names Date Date Given Aliases Regular Expression A Regular Expression B Regular Expression C Regular Expression D Pattern Markup Language 18 Pattern Markup Language 18

Person Birth Death Names Date Date Given Aliases Regular Expression A Regular Expression B Regular Expression C Regular Expression D Pattern Markup Language 19 Pattern Markup Language 19

PatML Generation Tools Person Birth Death Names Date Date Given Aliases Regular Expression A Regular Expression B Regular Expression C Regular Expression D Schema Generator Establishes relationships Pattern Markup Language 20 Pattern Markup Language 20

PatML Generation Tools Person Birth Death Names Date Date Given Aliases Regular Expression A Regular Expression B Regular Expression C Regular Expression D PatML Editor Helps write the regular expressions and establish which facts they match Pattern Markup Language 21 Pattern Markup Language 21

Pattern Markup Language 22 Pattern Markup Language 22

Using PatML Editor Using PatML Editor  Get your schema file Get your schema file  Browse for sample page Browse for sample page  Add nodes Add nodes  Add expressions Add expressions  See the highlights in source See the highlights in source  Adjust Adjust Pattern Markup Language 23 Pattern Markup Language 23

PatML Editor PatML Editor Tree representing Text area with Interface Interface PatML structure sample page source Browser with Pattern Markup Language 24 Pattern Markup Language 24 rendered sample page

Pattern Markup Language 25 Pattern Markup Language 25

Fast and Versatile Fast and Versatile  Regular sites can be integrated Regular sites can be integrated in hours in hours  Adaptable to any type of Adaptable to any type of information information Pattern Markup Language 26 Pattern Markup Language 26

Implementation to Date Implementation to Date  Genesis uses PatML files to search a variety Genesis uses PatML files to search a variety of sites of sites  Searches TNG, Retrospect-GDS, Family Searches TNG, Retrospect-GDS, Family Search, GedCom and Kansas Gunslingers Search, GedCom and Kansas Gunslingers  Standardizes information for a common Standardizes information for a common datamodel datamodel  Simultaneously searches other sites (in Simultaneously searches other sites (in different formats) for people with similar different formats) for people with similar information information Pattern Markup Language 27 Pattern Markup Language 27

Results Results Pattern Markup Language 28 Pattern Markup Language 28

Results Results  Produced PatML that correctly extracts Produced PatML that correctly extracts data from TNG, RGDS, GedCom Sites, data from TNG, RGDS, GedCom Sites, and Kansas Gunslingers and Kansas Gunslingers  User Interface allows for improved User Interface allows for improved debugging environment debugging environment  ~1/10 coding time with PatML ~1/10 coding time with PatML generation tools compared to similarly generation tools compared to similarly functioning hand coded parsers functioning hand coded parsers Pattern Markup Language 29 Pattern Markup Language 29

Limitations Limitations Sites must be recognizable with Sites must be recognizable with  regular expressions regular expressions  Even regular sites have page to Even regular sites have page to page HTML variations page HTML variations Programmer error with regular Programmer error with regular  expressions expressions Regular expression operations can be Regular expression operations can be  slow slow Pattern Markup Language 30 Pattern Markup Language 30

Future work Future work  Automatic regular expression Automatic regular expression generation generation  Parsing links to extract data on Parsing links to extract data on connected pages connected pages  Use in other applications and fields Use in other applications and fields  XPath approaches XPath approaches Pattern Markup Language 31 Pattern Markup Language 31