Neural Information Retrieval Wassila Lalouani 1 Plan Neural - - PowerPoint PPT Presentation
Neural Information Retrieval Wassila Lalouani 1 Plan Neural network architectures Neural IR tasks Neural IR architecture Feature Representations Neural IR query auto completion Neural IR query suggestion Neural IR document
1
2
3
w1 w2 wd x1 x2 xd 1 y
f1
Perceptron Neural net
INPUT 32x32
Convolutions Subsampling Convolutions
C1: feature maps 6@28x28
Subsampling
S2: f. maps 6@14x14 S4: f. maps 16@5x5 C5: layer 120 C3: f. maps 16@10x10 F6: layer 84
Full connection Full connection Gaussian connections
OUTPUT 10
LeNet
4
ü What are the neural IR task?
Examples extracted from [2]
5
ü Learning a matching function using traditional feature based representation
query and document ü learning representations of text to deal with vocabulary mismatch ü Learning end-to end IR.
Figure from [2]
6
q The vector dimensions are not hand-crafted features q latent space that preserve the properties and the relationships between the items.
Examples extracted from [1]
7
“seattle”>
§ Next query suggestion § <“things to do in seattle”, “space needle”> § CLSM is a deep neural network with a convolutional and pooling layers. § CLSM projects a variable-length text into a fixed-length vector. § Each word in the input text is first hashed to a letter trigram vector. § For each word, the convolutional layer extracts contextual features based on its immediate neighbours within the window size. § A max pooling layer combines the output of the convolutional layer into a fixed-length feature vector. § The output layer produces the final vector representation of the entire query.
Paper [7]
8
Auto-Completion
§ QAC suggest queries according to observed queries in the logs. § Select the set of candidates given partial user query § Compute the frequently of the query suffixes based on historical of the search. § The rank the candidat suffix based on n-gram feature and CLSM.
ꭕ recommend completion for rare query prefixes. ꭕ Generic mining and efficient suffix ranking. ꭕ N-gram suffix generate synthetic suggestion candidates that have never been observed in the historical. ꭕ N-gram outperform CLSM suggestion and no no semantic evaluation. n-gram frequency features depend on the number of words in the suffix and the frequency of the grams. CLSM model uses prefix-suffix pairs of the dataset to train CLSM. Normalize the prefix and suffix by removing the end-term The CLSM model projects both the prefix and the suffix to a common vector space to rank the candidate according to prefix-suffix cosine similarity. CLSM output the clsmsim feature based on the cosine similarities between pair of suffix prefix.
Paper [3]
9
Auto-Completion ꭕ CLSM and ngram perform better than each separately. ꭕ The approach perform better than baseline for unseen and rate suffix.
10
§ CLSM represent query reformulations as vectors that map semantically and syntactically similar query closer in the embedding space. § CLSM provide semantical mapping of query to the embedding space. § The input in this architecture are pair of queries formulated during same users session. § Non-contextual features: prefix length, suggestion length, the vowels,… § Contextual features: n-gram similarity between the suggestion candidate and the previous queries from the same user session. § CLSM: The cosine similarity between the CLSM vectors corresponding to the suggestion candidate and a maximum of previous 10 queries from the same session
Consecutive queries in user sessions ꭕ recommend suggestion based on semantically embedding. ꭕ Lack of context-aware ranking model.
Paper [4]
11
ü The supervised ranking models using contextual features (CLSM) show large improvements on the short prefixes and handle efficiently the ambiguity. ü Across all CLSM based features out-perform the baseline ranking model. mean reciprocal rank
12
ü The relevance of a document based on exact matches of query terms in the body text. ü The short vertical lines correspond to exact matches between pairs of query and document terms. ü The query term matches in the relevant documents are more localized and clustered. ü Does not learn embedding. Feature representation Neural model
Paper [5]
13
§ Distributed representation: § latent semantic analysis learn distributed representations of text § Match the query against the document in the latent semantic space. § Deep neural network to matches the query and the document using learned distributed representations. § n-graph based representation of each term in the query and document. § The matching is based
Hadamard product between the embedded document matrix and the extended query embedding § Duet Model: § The join training will provide the robustness to learn both embeddings that are combined with similarity checking layer. ü Traditional IR based on statistical proprieties offer outstanding results for matching query terms and the document ü distributed representations complement the model with semantically equivalence matching.
Paper [5]
14
ü All duet runs outperformed local and distributed, non-neural and neural baselines. normalized discounted cumulative gain
15
16
17
§ https://www.researchgate.net/figure/Artificial-neural-network-architecture- ANN-i-h-1-h-2-h-n-o_fig1_321259051
§ Craswell, N., Croft, W.B., de Rijke, M. et al. Neural information retrieval: introduction to the special issue. Inf Retrieval J 21, 107–110 (2018). https://doi.org/10.1007/s10791-017-9323-9 NEURAL MODELS FOR INFORMATION RETRIEVAL. § Bhaskar Mitra and Nick Craswell. 2015. Query Auto-Completion for Rare Prefixes. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management (CIKM ’15). Association for Computing Machinery, New York, NY, USA, 1755–1758. § Bhaskar Mitra. 2015. Exploring Session Context using Distributed Representations of Queries and Reformulations. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’15). Association for Computing Machinery, New York, NY, USA, 3–12. § Bhaskar Mitra, Fernando Diaz, and Nick Craswell. 2017. Learning to Match using Local and Distributed Representations of Text for Web Search. In Proceedings of the 26th International Conference on World Wide Web (WWW ’17). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 1291–1299. § Bhaskar Mitra and Nick Craswell. 2017. Neural Models for Information Retrieval. CoRR abs/1705.01509 (2017). arXiv:1705.01509. § Shen, Yelong et al. “A Convolutional Latent Semantic Model for Web Search.” (2014).
18