Elastic Search - Aditi Choksi (EW18455) Elastic Search Search - - PowerPoint PPT Presentation

Elastic Search

• Aditi Choksi (EW18455)

Elastic Search

• Search engine
• Distributed search
• Full text Search
• Near real time search

• Size of data being

generated and stored has grown exponentially

• ver the past few

decades.

Evolution of Data

Vertical Scaling – increase machine size Horizontal Scaling – add more machines

Need for Distributed Data Systems

• Elastic search sends a query to every

node / machine and then collects and combines the results from them to return to the user.

Elastic Search Cluster

Shard Shard Shard Shard 2 Shard 3 Shard 3 Shard 1 Shard 2 Shard 1

Lucene Index

Segment

Documents 3 1 2, 3 2 1, 2, 3 2 2,3 1 3

dictionary postings

Inverted Indexes

Term count Frequency choice 1 coming 1 contours 2 fury 1 is 3

• urs

1 the 2 winter 1 yours 1

Documents 3 1 2, 3 2 1, 2, 3 2 2,3 1 3

dictionary postings

Inverted Indexes

Term count Frequency choice 1 coming 1 contours 2 fury 1 is 3

• urs

1 the 2 winter 1 yours 1

2 2, 3

• Wild card searches are difficult
• These are unindexed queries
• So searching somethings like *our* requires going

through all the terms of the index.

Wild Card Queries

Term count Frequency choice 1 coming 1 contours 2 fury 1 is 3

• urs

1 the 2 winter 1 yours 1

• Can you think of a way to make queries like *ours

efficient? What kind of index can we create?

Term count Frequency choice 1 coming 1 contours 2 fury 1 is 3

• urs

1 the 2 winter 1 yours 1

Question

• Can you think of a way to make queries like *ours

efficient? What kind of index can we create?

• Reverse Indexing:

*ours → sruo*

• search(our*) union search(sruo*)

Term count Reversed word choice eciohc coming gnimoc contours sroutnoc fury yruf is si

• urs

srou the eht winter retniw yours sruoy

Question

Shard Shard Shard Shard 2 Shard 3 Shard 3

Shard 1 Shard 2 Shard 1

Bottom up

• Indexes are immutable,

segments are merged and that’s when obsolete entries are cleaned

References

• [1]Reaz Ahmed, R. Boutaba, 2011 “A Survey of Distributed Search Techniques in Large Scale

Distributed Systems”, IEEE Communications Surveys and Tutorials

• [2]Enrico Nardelli, Fabio Barillari, 2015, “Distributed Searching of Multi-dimensional Data”
• [3] ShaoHua Liu ; Xing Xue, 2016, Distributed Database Query Based on Improved Genetic Algorithm,

3rd International Conference on Information Science and Control Engineering

• [4] Clinton Gourmley, Zachary Tong, 2015, ElasticSearch: The Definitive Guide
• https://www.youtube.com/watch?v=lWKEphKIG8U

Thanks ☺