an evaluation of string similarity measures on pricelists
play

An evaluation of string similarity measures on pricelists of - PowerPoint PPT Presentation

Apr 13, 2023 •6 likes •1.16k views

An evaluation of string similarity measures on pricelists of computer components R. Jirouek, V. Kratochvl, T. Kroupa, R. Lnnika, M. Studen, J. Vomlel, P. Hampl, and H. Hamplov Institute of Information Theory and Automation Academy

  1. The string edit distance measure Example R 1 W I N D O W S T E R M R 2 k = 8 W I N T R M N L R = ” T” Length ( R ) = 2 Similarity ( R 1 , R 2 ) = 2 + 3 + 2 + 2

  2. The string edit distance measure Example R 1 W I N D O W S T E R M R 2 k = 9 W I N T R M N L Similarity ( R 1 , R 2 ) = 2 + 3 + 2 + 2

  3. The string edit distance measure Example R 1 W I N D O W S T E R M R 2 k = 10 W I N T R M N L Similarity ( R 1 , R 2 ) = 2 + 3 + 2 + 2

  4. The string edit distance measure Example R 1 W I N D O W S T E R M R 2 k = 11 W I N T R M N L Similarity ( R 1 , R 2 ) = 2 + 3 + 2 + 2

  5. The string edit distance measure Example R 1 W I N D O W S T E R M R 2 k = 11 W I N T R M N L R = ”RM” Length ( R ) = 2 Similarity ( R 1 , R 2 ) = 2 + 3 + 2 + 2 + 2

  6. The string edit distance measure Example R 1 W I N D O W S T E R M R 2 W I N T R M N L Similarity ( R 1 , R 2 ) = 2 + 3 + 2 + 2 + 2 = 11

  7. A vector based method • Every string is encoded as a vector of real numbers whose components are formed by weights of individual tokens (groups of characters) presented in the string.

  8. A vector based method • Every string is encoded as a vector of real numbers whose components are formed by weights of individual tokens (groups of characters) presented in the string. • The string is divided into tokens by special characters - tokens separators (e.g., space, comma, semicolon, etc.)

  9. A vector based method • Every string is encoded as a vector of real numbers whose components are formed by weights of individual tokens (groups of characters) presented in the string. • The string is divided into tokens by special characters - tokens separators (e.g., space, comma, semicolon, etc.) • A popular method for computing the weights is the TF-IDF method.

  10. A vector based method • Every string is encoded as a vector of real numbers whose components are formed by weights of individual tokens (groups of characters) presented in the string. • The string is divided into tokens by special characters - tokens separators (e.g., space, comma, semicolon, etc.) • A popular method for computing the weights is the TF-IDF method. • Let n ( x , S ) be the number of occurrences of token x in string S (often, it is 0 and 1),

  11. A vector based method • Every string is encoded as a vector of real numbers whose components are formed by weights of individual tokens (groups of characters) presented in the string. • The string is divided into tokens by special characters - tokens separators (e.g., space, comma, semicolon, etc.) • A popular method for computing the weights is the TF-IDF method. • Let n ( x , S ) be the number of occurrences of token x in string S (often, it is 0 and 1), • n ( S ) be the total number of tokens in string S ,

  12. A vector based method • Every string is encoded as a vector of real numbers whose components are formed by weights of individual tokens (groups of characters) presented in the string. • The string is divided into tokens by special characters - tokens separators (e.g., space, comma, semicolon, etc.) • A popular method for computing the weights is the TF-IDF method. • Let n ( x , S ) be the number of occurrences of token x in string S (often, it is 0 and 1), • n ( S ) be the total number of tokens in string S , • m be the total number of all strings in the data, and

  13. A vector based method • Every string is encoded as a vector of real numbers whose components are formed by weights of individual tokens (groups of characters) presented in the string. • The string is divided into tokens by special characters - tokens separators (e.g., space, comma, semicolon, etc.) • A popular method for computing the weights is the TF-IDF method. • Let n ( x , S ) be the number of occurrences of token x in string S (often, it is 0 and 1), • n ( S ) be the total number of tokens in string S , • m be the total number of all strings in the data, and • m ( x ) be the number of strings containing token x .

  14. A vector based method • Every string is encoded as a vector of real numbers whose components are formed by weights of individual tokens (groups of characters) presented in the string. • The string is divided into tokens by special characters - tokens separators (e.g., space, comma, semicolon, etc.) • A popular method for computing the weights is the TF-IDF method. • Let n ( x , S ) be the number of occurrences of token x in string S (often, it is 0 and 1), • n ( S ) be the total number of tokens in string S , • m be the total number of all strings in the data, and • m ( x ) be the number of strings containing token x . • The weight of a token x in string S is defined as n ( x , S ) m w ( x , S ) = n ( S ) log m ( x ) .

  15. A vector based method • Let d be the total number of different tokens in the entire data.

  16. A vector based method • Let d be the total number of different tokens in the entire data. • Then w ( S ) = ( w ( x 1 , S ) , . . . w ( x d , S )) T is a vector that characterizes string S .

  17. A vector based method • Let d be the total number of different tokens in the entire data. • Then w ( S ) = ( w ( x 1 , S ) , . . . w ( x d , S )) T is a vector that characterizes string S . • By v ( S ) we will denote the normalized weight vector w ( S ) v ( S ) = �� d i = 1 w ( x i , S ) 2

  18. A vector based method • Let d be the total number of different tokens in the entire data. • Then w ( S ) = ( w ( x 1 , S ) , . . . w ( x d , S )) T is a vector that characterizes string S . • By v ( S ) we will denote the normalized weight vector w ( S ) v ( S ) = �� d i = 1 w ( x i , S ) 2 • Similarity of two strings S 1 and S 2 is then computed as the scalar product of normalized weight vectors v ( S 1 ) and v ( S 2 ) d � v ( S 1 ) T · v ( S 2 ) . Sim 3 ( S 1 , S 2 ) = v ( x i , S 1 ) · v ( x i , S 2 ) = i = 1

  19. A vector based method • Let d be the total number of different tokens in the entire data. • Then w ( S ) = ( w ( x 1 , S ) , . . . w ( x d , S )) T is a vector that characterizes string S . • By v ( S ) we will denote the normalized weight vector w ( S ) v ( S ) = �� d i = 1 w ( x i , S ) 2 • Similarity of two strings S 1 and S 2 is then computed as the scalar product of normalized weight vectors v ( S 1 ) and v ( S 2 ) d � v ( S 1 ) T · v ( S 2 ) . Sim 3 ( S 1 , S 2 ) = v ( x i , S 1 ) · v ( x i , S 2 ) = i = 1 • Note that since both vectors are sparse the computation of the scalar product can be efficiently implemented.

  20. The vector based method Example S 1 toner magenta pro clp-510/510n, az 5000 stran

  21. The vector based method Example S 1 toner magenta pro clp-510/510n, az 5000 stran S 2 samsung toner magenta pro clp510/n (5000str )

  22. The vector based method Example S 1 toner magenta pro clp-510/510n, az 5000 stran S 2 samsung toner magenta pro clp510/n (5000str ) • For simplicity assume tokens from these two strings only: toner, magenta, pro, clp, 510, 510n, az, 5000, stran, samsung, clp510, n, 5000str

  23. The vector based method Example S 1 toner magenta pro clp-510/510n, az 5000 stran S 2 samsung toner magenta pro clp510/n (5000str ) • For simplicity assume tokens from these two strings only: toner, magenta, pro, clp, 510, 510n, az, 5000, stran, samsung, clp510, n, 5000str • w ( toner , S 1 ) = 1 9 log 36478 274 = 0 . 236

  24. The vector based method Example S 1 toner magenta pro clp-510/510n, az 5000 stran S 2 samsung toner magenta pro clp510/n (5000str ) • For simplicity assume tokens from these two strings only: toner, magenta, pro, clp, 510, 510n, az, 5000, stran, samsung, clp510, n, 5000str • w ( toner , S 1 ) = 1 9 log 36478 274 = 0 . 236 • w ( toner , S 2 ) = 1 7 log 36478 274 = 0 . 303

  25. The vector based method Example S 1 toner magenta pro clp-510/510n, az 5000 stran S 2 samsung toner magenta pro clp510/n (5000str ) • For simplicity assume tokens from these two strings only: toner, magenta, pro, clp, 510, 510n, az, 5000, stran, samsung, clp510, n, 5000str • w ( toner , S 1 ) = 1 9 log 36478 274 = 0 . 236 • w ( toner , S 2 ) = 1 7 log 36478 274 = 0 . 303 • w ( magenta , S 1 ) = 1 9 log 36478 = 0 . 310 59

  26. The vector based method Example S 1 toner magenta pro clp-510/510n, az 5000 stran S 2 samsung toner magenta pro clp510/n (5000str ) • For simplicity assume tokens from these two strings only: toner, magenta, pro, clp, 510, 510n, az, 5000, stran, samsung, clp510, n, 5000str • w ( toner , S 1 ) = 1 9 log 36478 274 = 0 . 236 • w ( toner , S 2 ) = 1 7 log 36478 274 = 0 . 303 • w ( magenta , S 1 ) = 1 9 log 36478 = 0 . 310 59 • w ( magenta , S 2 ) = 1 7 log 36478 = 0 . 399 59

  27. The vector based method Example S 1 toner magenta pro clp-510/510n, az 5000 stran S 2 samsung toner magenta pro clp510/n (5000str ) • For simplicity assume tokens from these two strings only: toner, magenta, pro, clp, 510, 510n, az, 5000, stran, samsung, clp510, n, 5000str • w ( toner , S 1 ) = 1 9 log 36478 274 = 0 . 236 • w ( toner , S 2 ) = 1 7 log 36478 274 = 0 . 303 • w ( magenta , S 1 ) = 1 9 log 36478 = 0 . 310 59 • w ( magenta , S 2 ) = 1 7 log 36478 = 0 . 399 59 • w ( S 1 ) = (0.236, 0.310, 0.285, 0.420, 0.235, 0.345, 0.034, 0.121, 0.097, 0.000, 0.000, 0.000, 0.000)

  28. The vector based method Example S 1 toner magenta pro clp-510/510n, az 5000 stran S 2 samsung toner magenta pro clp510/n (5000str ) • For simplicity assume tokens from these two strings only: toner, magenta, pro, clp, 510, 510n, az, 5000, stran, samsung, clp510, n, 5000str • w ( toner , S 1 ) = 1 9 log 36478 274 = 0 . 236 • w ( toner , S 2 ) = 1 7 log 36478 274 = 0 . 303 • w ( magenta , S 1 ) = 1 9 log 36478 = 0 . 310 59 • w ( magenta , S 2 ) = 1 7 log 36478 = 0 . 399 59 • w ( S 1 ) = (0.236, 0.310, 0.285, 0.420, 0.235, 0.345, 0.034, 0.121, 0.097, 0.000, 0.000, 0.000, 0.000) • w ( S 2 ) = (0.303, 0.399, 0.366, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.056, 0.451, 0.023, 0.456)

  29. The vector based method Example S 1 toner magenta pro clp-510/510n, az 5000 stran S 2 samsung toner magenta pro clp510/n (5000str ) i = 1 w ( x i , S 1 ) 2 = w ( S 1 ) w ( S 1 ) • v ( S 1 ) = 0 . 780 �� d

  30. The vector based method Example S 1 toner magenta pro clp-510/510n, az 5000 stran S 2 samsung toner magenta pro clp510/n (5000str ) i = 1 w ( x i , S 1 ) 2 = w ( S 1 ) w ( S 1 ) • v ( S 1 ) = 0 . 780 �� d i = 1 w ( x i , S 2 ) 2 = w ( S 2 ) w ( S 2 ) • v ( S 2 ) = 0 . 794 �� d

  31. The vector based method Example S 1 toner magenta pro clp-510/510n, az 5000 stran S 2 samsung toner magenta pro clp510/n (5000str ) i = 1 w ( x i , S 1 ) 2 = w ( S 1 ) w ( S 1 ) • v ( S 1 ) = 0 . 780 �� d i = 1 w ( x i , S 2 ) 2 = w ( S 2 ) w ( S 2 ) • v ( S 2 ) = 0 . 794 �� d • v ( S 1 ) = (0.302, 0.397, 0.365, 0.538, 0.301, 0.442, 0.044, 0.155, 0.124, 0.000, 0.000, 0.000, 0.000)

  32. The vector based method Example S 1 toner magenta pro clp-510/510n, az 5000 stran S 2 samsung toner magenta pro clp510/n (5000str ) i = 1 w ( x i , S 1 ) 2 = w ( S 1 ) w ( S 1 ) • v ( S 1 ) = 0 . 780 �� d i = 1 w ( x i , S 2 ) 2 = w ( S 2 ) w ( S 2 ) • v ( S 2 ) = 0 . 794 �� d • v ( S 1 ) = (0.302, 0.397, 0.365, 0.538, 0.301, 0.442, 0.044, 0.155, 0.124, 0.000, 0.000, 0.000, 0.000) • v ( S 2 ) = (0.339, 0.446, 0.409, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.063, 0.504, 0.026, 0.510)

  33. The vector based method Example S 1 toner magenta pro clp-510/510n, az 5000 stran S 2 samsung toner magenta pro clp510/n (5000str ) i = 1 w ( x i , S 1 ) 2 = w ( S 1 ) w ( S 1 ) • v ( S 1 ) = 0 . 780 �� d i = 1 w ( x i , S 2 ) 2 = w ( S 2 ) w ( S 2 ) • v ( S 2 ) = 0 . 794 �� d • v ( S 1 ) = (0.302, 0.397, 0.365, 0.538, 0.301, 0.442, 0.044, 0.155, 0.124, 0.000, 0.000, 0.000, 0.000) • v ( S 2 ) = (0.339, 0.446, 0.409, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.063, 0.504, 0.026, 0.510) v ( S 1 ) T · v ( S 2 ) Sim 3 ( S 1 , S 2 ) = = 0 . 302 · 0 . 339 + 0 . 397 · 0 . 446 + 0 . 365 · 0 . 409 = 0 . 429

  34. A linear combination of methods • Each method uses a different approach for finding equivalent components.

  35. A linear combination of methods • Each method uses a different approach for finding equivalent components. • Therefore one can hope that their combination can provide better results.

  36. A linear combination of methods • Each method uses a different approach for finding equivalent components. • Therefore one can hope that their combination can provide better results. • We have tested linear combinations of

  37. A linear combination of methods • Each method uses a different approach for finding equivalent components. • Therefore one can hope that their combination can provide better results. • We have tested linear combinations of • the fulltext search Sim 1 ,

  38. A linear combination of methods • Each method uses a different approach for finding equivalent components. • Therefore one can hope that their combination can provide better results. • We have tested linear combinations of • the fulltext search Sim 1 , • string similarity Sim 2 , and

  39. A linear combination of methods • Each method uses a different approach for finding equivalent components. • Therefore one can hope that their combination can provide better results. • We have tested linear combinations of • the fulltext search Sim 1 , • string similarity Sim 2 , and • the vector based method Sim 3

  40. A linear combination of methods • Each method uses a different approach for finding equivalent components. • Therefore one can hope that their combination can provide better results. • We have tested linear combinations of • the fulltext search Sim 1 , • string similarity Sim 2 , and • the vector based method Sim 3 Sim 4 ( S 1 , S 2 ) = c 1 · Sim 1 ( S 1 , S 2 )+ c 2 · Sim 2 ( S 1 , S 2 )+ c 3 · Sim 3 ( S 1 , S 2 )

  41. A linear combination of methods • Each method uses a different approach for finding equivalent components. • Therefore one can hope that their combination can provide better results. • We have tested linear combinations of • the fulltext search Sim 1 , • string similarity Sim 2 , and • the vector based method Sim 3 Sim 4 ( S 1 , S 2 ) = c 1 · Sim 1 ( S 1 , S 2 )+ c 2 · Sim 2 ( S 1 , S 2 )+ c 3 · Sim 3 ( S 1 , S 2 ) where c = ( c 1 , c 2 , c 3 ) was set to ( 0 . 3 , 1 , 1 ) , ( 0 , 1 , 1 ) , and ( 0 , 1 , 2 ) .

  42. Experiments • We selected two pricelists of computer components from two different suppliers.

  43. Experiments • We selected two pricelists of computer components from two different suppliers. • They contained together 64566 components.

  44. Experiments • We selected two pricelists of computer components from two different suppliers. • They contained together 64566 components. • From these two pricelists we selected only those components that were given a part number in both pricelists - we have got 7060 different part numbers.

  45. Experiments • We selected two pricelists of computer components from two different suppliers. • They contained together 64566 components. • From these two pricelists we selected only those components that were given a part number in both pricelists - we have got 7060 different part numbers. • From these we randomly selected 500 part numbers.

  46. Experiments • We selected two pricelists of computer components from two different suppliers. • They contained together 64566 components. • From these two pricelists we selected only those components that were given a part number in both pricelists - we have got 7060 different part numbers. • From these we randomly selected 500 part numbers. • These part numbers defined our test pairs of components.

  47. Experiments • We selected two pricelists of computer components from two different suppliers. • They contained together 64566 components. • From these two pricelists we selected only those components that were given a part number in both pricelists - we have got 7060 different part numbers. • From these we randomly selected 500 part numbers. • These part numbers defined our test pairs of components. • For each of 500 components from the first pricelist we used the tested methods to find k ( k = 1 , 2 , . . . , 15) most similar components in the (complete) second pricelist.

  48. Experiments • We selected two pricelists of computer components from two different suppliers. • They contained together 64566 components. • From these two pricelists we selected only those components that were given a part number in both pricelists - we have got 7060 different part numbers. • From these we randomly selected 500 part numbers. • These part numbers defined our test pairs of components. • For each of 500 components from the first pricelist we used the tested methods to find k ( k = 1 , 2 , . . . , 15) most similar components in the (complete) second pricelist. • Then we checked whether the component with the same part number is among those k selected ones.

  49. Experiments • We selected two pricelists of computer components from two different suppliers. • They contained together 64566 components. • From these two pricelists we selected only those components that were given a part number in both pricelists - we have got 7060 different part numbers. • From these we randomly selected 500 part numbers. • These part numbers defined our test pairs of components. • For each of 500 components from the first pricelist we used the tested methods to find k ( k = 1 , 2 , . . . , 15) most similar components in the (complete) second pricelist. • Then we checked whether the component with the same part number is among those k selected ones. • We counted the number of these cases and computed the relative success rate for each method with respect to k .

  50. Results of experiments

  51. Examples of unmatched components Example (Acer server) AAG320 PD 940 (3.2 GHz, 2x 2MB, 800 MHz FSB), 1x 512 MB DDR2 533/16x DVD-ROM Acer Altos G320-PD940 3.2GHz/2x2MB,800F/512MB/DVD/noHDD/noKB

  52. Examples of unmatched components Example (Acer server) AAG320 PD 940 (3.2 GHz, 2x 2MB, 800 MHz FSB), 1x 512 MB DDR2 533/16x DVD-ROM Acer Altos G320-PD940 3.2GHz/2x2MB,800F/512MB/DVD/noHDD/noKB • Acer Altos is abbreviated to AA .

  53. Examples of unmatched components Example (Acer server) AAG320 PD 940 (3.2 GHz, 2x 2MB, 800 MHz FSB), 1x 512 MB DDR2 533/16x DVD-ROM Acer Altos G320-PD940 3.2GHz/2x2MB,800F/512MB/DVD/noHDD/noKB • Acer Altos is abbreviated to AA . • Different token separators (comma, space, slash, dash, braces) are used.

  54. Examples of unmatched components Example (Acer server) AAG320 PD 940 (3.2 GHz, 2x 2MB, 800 MHz FSB), 1x 512 MB DDR2 533/16x DVD-ROM Acer Altos G320-PD940 3.2GHz/2x2MB,800F/512MB/DVD/noHDD/noKB • Acer Altos is abbreviated to AA . • Different token separators (comma, space, slash, dash, braces) are used. • Whether a symbol is a separator depends on its context.

  55. Examples of unmatched components Example (Acer server) AAG320 PD 940 (3.2 GHz, 2x 2MB, 800 MHz FSB), 1x 512 MB DDR2 533/16x DVD-ROM Acer Altos G320-PD940 3.2GHz/2x2MB,800F/512MB/DVD/noHDD/noKB • Acer Altos is abbreviated to AA . • Different token separators (comma, space, slash, dash, braces) are used. • Whether a symbol is a separator depends on its context. • For example, the space symbol is a separator between PD940 and 3.2 GHz but “ 3.2 GHz ” should be one token.

  56. Examples of unmatched components Example (Ink cartridge) Ink. náplň No. 84 pro DesignJet 10PS/20PS/50PS C5016A Black ink Cartridge pro DSJ x0ps

  57. Examples of unmatched components Example (Ink cartridge) Ink. náplň No. 84 pro DesignJet 10PS/20PS/50PS C5016A Black ink Cartridge pro DSJ x0ps • Cartridge is náplň in Czech,

  58. Examples of unmatched components Example (Ink cartridge) Ink. náplň No. 84 pro DesignJet 10PS/20PS/50PS C5016A Black ink Cartridge pro DSJ x0ps • Cartridge is náplň in Czech, • 10PS/20PS/50PS is abbreviated to x0ps , and

  59. Examples of unmatched components Example (Ink cartridge) Ink. náplň No. 84 pro DesignJet 10PS/20PS/50PS C5016A Black ink Cartridge pro DSJ x0ps • Cartridge is náplň in Czech, • 10PS/20PS/50PS is abbreviated to x0ps , and • DesignJet is abbreviated to DSJ .

  60. Examples of unmatched components Example (Cable) Kabel Pure AV Blue series Firewire 4pin/6pin, 1.8m PureAV kabel FireWire, 4/6 kolíků - 1,8 m - Řada Blue

  61. Examples of unmatched components Example (Cable) Kabel Pure AV Blue series Firewire 4pin/6pin, 1.8m PureAV kabel FireWire, 4/6 kolíků - 1,8 m - Řada Blue • series is Řada in Czech,

  62. Examples of unmatched components Example (Cable) Kabel Pure AV Blue series Firewire 4pin/6pin, 1.8m PureAV kabel FireWire, 4/6 kolíků - 1,8 m - Řada Blue • series is Řada in Czech, • 4pin/6pin corresponds to 4/6 kolíků since pin is kolík in Czech, and

  63. Examples of unmatched components Example (Cable) Kabel Pure AV Blue series Firewire 4pin/6pin, 1.8m PureAV kabel FireWire, 4/6 kolíků - 1,8 m - Řada Blue • series is Řada in Czech, • 4pin/6pin corresponds to 4/6 kolíků since pin is kolík in Czech, and • 1.8m corresponds to 1,8m .

  64. Examples of unmatched components Example (Mail antispam and antivirus) SYMANTEC BRIGHTMAIL ANTISPAM + ANTIV 6.0 SUBS + GOLD MAINT 1YR IN VALUE BAND F(5 Sym. Bright.Antispam + Antivirus 6.0 IN F(500-999) + 1YR GM

  65. Examples of unmatched components Example (Mail antispam and antivirus) SYMANTEC BRIGHTMAIL ANTISPAM + ANTIV 6.0 SUBS + GOLD MAINT 1YR IN VALUE BAND F(5 Sym. Bright.Antispam + Antivirus 6.0 IN F(500-999) + 1YR GM • Sym. Bright.Antispam + Antivirus corresponds to SYMANTEC BRIGHTMAIL ANTISPAM + ANTIV and

  66. Examples of unmatched components Example (Mail antispam and antivirus) SYMANTEC BRIGHTMAIL ANTISPAM + ANTIV 6.0 SUBS + GOLD MAINT 1YR IN VALUE BAND F(5 Sym. Bright.Antispam + Antivirus 6.0 IN F(500-999) + 1YR GM • Sym. Bright.Antispam + Antivirus corresponds to SYMANTEC BRIGHTMAIL ANTISPAM + ANTIV and • GM is an abbreviation for GOLD MAINT .

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

(Dis-)Similarity Measures for Description Logics Representation Claudia dAmato Computer

(Dis-)Similarity Measures for Description Logics Representation Claudia dAmato Computer

(Dis-)Similarity Measures for Description Logics Representation Claudia dAmato Computer Science Department University of Bari Poznan, 22 June 2011 Similarity Measures: Related Work (Dis-)Similarity measures for DLs Influence of DLs

1.21k views • 66 slides
Similarity Measures There are an enormous number of ways in which we can measure similarity

Similarity Measures There are an enormous number of ways in which we can measure similarity

Similarity Measures There are an enormous number of ways in which we can measure similarity They vary depending on whether the items we are interested in analysing come from one sample or two; are qualitative or quantitative; binary,

355 views • 7 slides
A Study of Hybrid Similarity Measures for Semantic Relation Extraction Alexander Panchenko and

A Study of Hybrid Similarity Measures for Semantic Relation Extraction Alexander Panchenko and

Introduction Methodology Evaluation Results Conclusion A Study of Hybrid Similarity Measures for Semantic Relation Extraction Alexander Panchenko and Olga Morozova , Center for Natural Language Processing (CENTAL), Universit e catholique

420 views • 41 slides
Towards an Efficient Combination of Similarity Measures for Semantic Relation Extraction

Towards an Efficient Combination of Similarity Measures for Semantic Relation Extraction

Introduction Methodology Evaluation Results Conclusion and Further Research Towards an Efficient Combination of Similarity Measures for Semantic Relation Extraction Alexander Panchenko alexander.panchenko@student.uclouvain.be Universit

1.12k views • 54 slides
N = * log tfidf f i , k i , k df Similarity

N = * log tfidf f i , k i , k df Similarity

Outline An Overview of Limitations of Database systems (Motivation for Information Retrieval IR systems) Information Retrieval Indexing Nov. 10, 2009 Similarity Measures Evaluation Maryam Karimzadehgan Other IR

329 views • 16 slides
Analysis of Similarity Measures between Short Text for the NTCIR-12 Short Text Conversation Task

Analysis of Similarity Measures between Short Text for the NTCIR-12 Short Text Conversation Task

Proceedings of the 12th NTCIR Conference on Evaluation of Information Access Technologies, June 7-10, 2016 Tokyo Japan Analysis of Similarity Measures between Short Text for the NTCIR-12 Short Text Conversation Task Kozo Chikai Yuki Arase

336 views • 8 slides
B ed -Tree: An All-Purpose Index Structure for String Similarity Search Based on Edit Distance

B ed -Tree: An All-Purpose Index Structure for String Similarity Search Based on Edit Distance

B ed -Tree: An All-Purpose Index Structure for String Similarity Search Based on Edit Distance Zhenjie Zhang, Marios Hadjieleftheriou, Beng Chin Ooi, Divesh Srivastava Outline Motivation and B ed -Tree Framework String Orders

395 views • 28 slides
Similarity and Correction of Strings and Trees : Towards a Correction of XML Documents Agata

Similarity and Correction of Strings and Trees : Towards a Correction of XML Documents Agata

Similarity and Correction of Strings and Trees : Towards a Correction of XML Documents Agata SAVARY Universit-Franois Rabelais de Tours, Campus de Blois, Laboratoire dInformatique Seminarium IPIPAN, 24 kwietnia, 2006 String-to-string

516 views • 33 slides
Topological measures of similarity Erin Wolf Chambers Saint Louis University

Topological measures of similarity Erin Wolf Chambers Saint Louis University

Topological measures of similarity Erin Wolf Chambers Saint Louis University erin.chambers@slu.edu Erin Chambers Topological measures of similarity Motivation: Measuring Similarity Between Curves How can we tell when two cycles or curves are

1.46k views • 129 slides
Generalized similarity measures for text data. Hubert Wagner (IST Austria) Joint work with

Generalized similarity measures for text data. Hubert Wagner (IST Austria) Joint work with

Generalized similarity measures for text data. Hubert Wagner (IST Austria) Joint work with Herbert Edelsbrunner GETCO 2015, Aalborg April 9, 2015 Plan Shape of data. Text as a point-cloud. Log-transform and similarity measure.

1.29k views • 69 slides
Investigating semantic similarity measures across the Gene Ontology: the relationship between

Investigating semantic similarity measures across the Gene Ontology: the relationship between

Investigating semantic similarity measures across the Gene Ontology: the relationship between sequence and annotation by P . W. Lord, R. D. Stevens, A. Brass and C. A. Goble Bioinformatics 19(10) 12751283

360 views • 22 slides
9. Evaluation Outline 9.1. Cranfield Paradigm & TREC 9.2. Non-Traditional Measures 9.3.

9. Evaluation Outline 9.1. Cranfield Paradigm & TREC 9.2. Non-Traditional Measures 9.3.

9. Evaluation Outline 9.1. Cranfield Paradigm & TREC 9.2. Non-Traditional Measures 9.3. Incomplete Judgments 9.4. Low-Cost Evaluation 9.5. Crowdsourcing 9.6. Online Evaluation Advanced Topics in Information Retrieval / Evaluation 2

573 views • 40 slides
Vladimir Polyakov Vladimir Polyakov NEW APPROACHES APPROACHES TO TO NEW LANGUAGE SIMILARITY

Vladimir Polyakov Vladimir Polyakov NEW APPROACHES APPROACHES TO TO NEW LANGUAGE SIMILARITY

Vladimir Polyakov Vladimir Polyakov NEW APPROACHES APPROACHES TO TO NEW LANGUAGE SIMILARITY LANGUAGE SIMILARITY MEASURES MEASURES The Swadesh Centenary Conference, Leipzig, January 17-18, 2009 . Introduction in the DB JM 1 . Introduction

917 views • 46 slides
Paraphrase Recognition Using Machine Learning to Combine Similarity Measures Prodromos

Paraphrase Recognition Using Machine Learning to Combine Similarity Measures Prodromos

Paraphrase Recognition Using Machine Learning to Combine Similarity Measures Prodromos Malakasiotis Department of Informatics Athens University of Economics and Business Paraphrase Recognition Given a pair of phrases, sentences, or

419 views • 19 slides
Lexical Semantics: Similarity Measures and Clustering Friday Monday Thursday Wednesday Tuesday

Lexical Semantics: Similarity Measures and Clustering Friday Monday Thursday Wednesday Tuesday

Beyond Dead Parrots Automatically constricted clusters of semantically similar words (Charniak, 1997): Lexical Semantics: Similarity Measures and Clustering Friday Monday Thursday Wednesday Tuesday Saturday Sunday People guys folks fellows

486 views • 16 slides
Semantic Similarity MultiJEDI ERC 259234 Semantic Similarity Semantic Similarity Mostly

Semantic Similarity MultiJEDI ERC 259234 Semantic Similarity Semantic Similarity Mostly

SemEval 2014 Task-3 Cross-Level Semantic Similarity MultiJEDI ERC 259234 Semantic Similarity Semantic Similarity Mostly focused on similar types of lexical items Semantic Similarity What if we have different types of inputs? CLSS:

971 views • 47 slides
Xerographic Copiers Xerographic Copiers After jams, the powder sometimes wipes off After

Xerographic Copiers Xerographic Copiers After jams, the powder sometimes wipes off After

Xerographic Copiers 1 Xerographic Copiers 2 Observations About Copiers Observations About Copiers Copies consist of colored powder stuck to paper Copies consist of colored powder stuck to paper Xerographic Copiers Xerographic Copiers

273 views • 3 slides
Introduction to spatial data Working with Geospatial Data in R What is spatial data? Data

Introduction to spatial data Working with Geospatial Data in R What is spatial data? Data

Working with Geospatial Data in R Introduction to spatial data Working with Geospatial Data in R What is spatial data? Data is associated with locations Location described by coordinates + a coordinate reference system (CRS)

628 views • 23 slides
Software Engineering I (02161) Week 10 Assoc. Prof. Hubert Baumeister Informatics and

Software Engineering I (02161) Week 10 Assoc. Prof. Hubert Baumeister Informatics and

Software Engineering I (02161) Week 10 Assoc. Prof. Hubert Baumeister Informatics and Mathematical Modelling Technical University of Denmark Spring 2011 2011 H. Baumeister (IMM) c Software Engineering I (02161) Spring 2011 10 / 69

448 views • 42 slides
Categorical Default Approaches to Uncertainty Factor Development in y p Chemical Risk

Categorical Default Approaches to Uncertainty Factor Development in y p Chemical Risk

Categorical Default Approaches to Uncertainty Factor Development in y p Chemical Risk Assessment John C. Lipscomb PhD, DABT, F.A.T.S. National Center for Environmental Assessment Assessment, Office of Research and Development, U. S.

207 views • 9 slides
SC State Fiscal Accountability Authority Division of Procurement Services Statewide IT Contract

SC State Fiscal Accountability Authority Division of Procurement Services Statewide IT Contract

SC State Fiscal Accountability Authority Division of Procurement Services Statewide IT Contract Updates South Carolina Procurement Code Changes Mike Spicer Stacy Adams http://www.procurement.sc.gov Statewide IT Term Contract Updates

224 views • 11 slides
pharmacy Chris Ellett, Transformation Director PharmacyForum April 2018, Marbella @CJEllett77

pharmacy Chris Ellett, Transformation Director PharmacyForum April 2018, Marbella @CJEllett77

The consumer future of pharmacy Chris Ellett, Transformation Director PharmacyForum April 2018, Marbella @CJEllett77 Hello Content 1. Consumers changing business 2. How does Pharmacy and Healthcare stack up? 3. What is this digital thing?

1.4k views • 106 slides
Sustainability and the Health 325 bed Tertiary Medical Center 3 Regional Hospitals of

Sustainability and the Health 325 bed Tertiary Medical Center 3 Regional Hospitals of

About us Integrated Delivery System Approximately 6,500 Total Employees 795 providers employed / 505 medical staff 59 clinic locations Sustainability and the Health 325 bed Tertiary Medical Center 3 Regional Hospitals of

79 views • 5 slides
IN INTR TRODUCTION ODUCTION The NRP and NRP jr is an NTU outreach programme designed for

IN INTR TRODUCTION ODUCTION The NRP and NRP jr is an NTU outreach programme designed for

IN INTR TRODUCTION ODUCTION The NRP and NRP jr is an NTU outreach programme designed for students from selected Secondary Schools, and all IP (Integrated Programme) Schools and Junior Colleges NRP projects will only be offered to

395 views • 17 slides

More recommend