AnomalyDAE: Dual Autoencoder for Anomaly Detection on Attributed - - PowerPoint PPT Presentation

• 1-

AnomalyDAE: Dual Autoencoder for Anomaly Detection on Attributed Networks

Haoyi Fan 1, Fengbin Zhang 1, Zuoyong Li 2

Harbin University of Science and Technology 1 Minjiang University 2 isfanhy@hrbust.edu.cn

• 2-

Background

https://webfoundation.org/2019/09/raising-the-bar-for-internet-access- introducing-meaningful-connectivity/ https://cointelegraph.com/news/south-africas-standard-bank-to-launch- permissioned-blockchain-for-overseas-exchange-services https://www.sciencedirect.com/science/article/pii/S0969212610003035

World Wide Web Biology Network Finance Transaction Network Social Network

• 3-

Background

Latent Space

Anomaly Anomaly

Anomaly Detection on the Attributed Network

• 4-

Background

Different types of anomalies

Structure-inconsistent Attribute-consistent Structure-consistent Attribute-inconsistent Structure-inconsistent Attribute-inconsistent Different neighbors contribute differently for anomaly detection

• Challenges:
• The cross-modality interactions between

the network structure and node attribute

• Neighbor-attention aware anomaly

measuring

• 5-

Background

Numerous attributed network based anomaly detection methods have been proposed…

LOF Breunig et al. 2000 SCAN Xu et al. 2007 FocusCO Perozzi et al. 2014 AMEN Perozzi et al. 2016 Radar Li et al. 2017 ANOMALOUS Peng et al. 2018 Dominant Ding et al. 2019 …

Network Structure

LOF SCAN AMEN Radar

Node Attribute

ANOMALOUS Dominant

• Deep representation learning

framework on graph?

FocusCO

• The cross-modality interactions

between the network structure and node attribute?

• 6-

Problem Statement

Problem

Given 𝓗 = 𝓦, 𝓕, 𝐘 , learn a score function 𝑔: 𝓦𝑗 ↦ 𝑧𝑗 ∈ ℝ, to classify sample 𝑦𝑗 based

• n the threshold 𝜇 :

𝑧𝑗 = {1, 𝑗𝑔 𝑔(𝓦𝑗) ≥ 𝜇, 0, 𝑝𝑢ℎ𝑓𝑠𝑥𝑗𝑡𝑓. where 𝑧𝑗 denotes the label of sample 𝑦𝑗, with 0 being the normal class and 1 the anomalous class.

Notations

𝓗 : Attributed network 𝓦 : Set of nodes in network. 𝓕 : Set of edges in network. 𝑁 : Number of nodes. 𝑂 : Dimension of attribute. 𝐁 ∈ ℝ𝑁×𝑁 : Adjacency matrix

• f a network.

𝐘 ∈ ℝ𝑁×𝑂 : Attribute matrix of all nodes.

• 7-

Method

AnomalyDAE

Structure Autoencoder Attribute Autoencoder

• 8-

Method

AnomalyDAE

Structure-level and attribute-level anomaly score

• 9-

Method

AnomalyDAE

Attribute Generation Node Attention

• 10-

Method

Neighbor-attention Mechanism in Structure Autoencoder

Initial feature transformation:

𝐚

~𝓦 = 𝜏(𝐘𝐗𝓦(𝟐) + b𝓦(𝟐))

Importance weights:

𝑓𝑗,𝑘 = 𝑏𝑢𝑢𝑜 𝐚

~ 𝑗 𝓦, 𝐚 ~ 𝑘 𝓦

= 𝜏(aT ⋅ [𝐗𝓦(𝟑)𝐚

~ 𝑗 𝓦||𝐗𝓦(𝟑)𝐚 ~ 𝑘 𝓦])

Normalization:

𝛿𝑗,𝑘 = ൯ exp(𝑓𝑗,𝑘 ቁ σ𝑙∈𝒪𝑗 exp (𝑓𝑗,𝑙

Neighbor-attention aware feature aggregation:

𝐚𝑗

𝓦 = σ𝑙∈𝒪𝑗 𝛿𝑗,𝑙 ⋅ 𝐚 ~ 𝑙 𝓦

𝐘, 𝐁 𝐚𝓦 𝐚

~𝓦

• 11-

Method

Attribute reconstruction:

𝐘

^

= 𝐚𝓦(𝐚𝓑) T 𝐚𝓦 𝐚𝓑

Cross-modality Interactions Capturing in Attribute Autoencoder

• 12-

Method

Loss and Anomaly Score

ℒ𝑠𝑓𝑑 = 𝛽||(𝐁 − 𝐁

^

) ⊙ 𝜾||𝐺

2 + (1 − 𝛽)||(𝐘 − 𝐘 ^

) ⊙ 𝜽||𝐺

2

𝜾𝑗,𝑘 = {1 𝑗𝑔 𝐁𝑗,𝑘 = 0, 𝜄 𝑝𝑢ℎ𝑓𝑠𝑥𝑗𝑡𝑓., 𝜽𝑗,𝑘 = {1 𝑗𝑔 𝐘𝑗,𝑘 = 0, 𝜃 𝑝𝑢ℎ𝑓𝑠𝑥𝑗𝑡𝑓. 𝑇𝑑𝑝𝑠𝑓 = 𝛽||(𝐁 − 𝐁

^

) ⊙ 𝜾||𝐺

2 + (1 − 𝛽)||(𝐘 − 𝐘 ^

) ⊙ 𝜽||𝐺

2

Loss Function: Anomaly Score:

Structure-level Anomaly Measure Attribute-level Anomaly Measure

• 13-

Method

Loss and Anomaly Score

ℒ𝑠𝑓𝑑 = 𝛽||(𝐁 − 𝐁

^

) ⊙ 𝜾||𝐺

2 + (1 − 𝛽)||(𝐘 − 𝐘 ^

) ⊙ 𝜽||𝐺

2

𝜾𝑗,𝑘 = {1 𝑗𝑔 𝐁𝑗,𝑘 = 0, 𝜄 𝑝𝑢ℎ𝑓𝑠𝑥𝑗𝑡𝑓., 𝜽𝑗,𝑘 = {1 𝑗𝑔 𝐘𝑗,𝑘 = 0, 𝜃 𝑝𝑢ℎ𝑓𝑠𝑥𝑗𝑡𝑓. 𝑇𝑑𝑝𝑠𝑓 = 𝛽||(𝐁 − 𝐁

^

) ⊙ 𝜾||𝐺

2 + (1 − 𝛽)||(𝐘 − 𝐘 ^

) ⊙ 𝜽||𝐺

2

Loss Function: Anomaly Score: 𝑧𝑗 = {1, 𝑗𝑔 𝑔 𝓦𝑗 ≥ 𝜇, 0, 𝑝𝑢ℎ𝑓𝑠𝑥𝑗𝑡𝑓. 𝜇=Distribution(𝑇𝑑𝑝𝑠𝑓) Solution for Problem:

• 14-

Experiment

Datasets Baselines Evaluation Metric

AUC (Area Under a receiver operating characteristic Curve) LOF Breunig et al. 2000 SCAN Xu et al. 2007 AMEN Perozzi et al. 2016 Radar Li et al. 2017 ANOMALOUS Peng et al. 2018 Dominant Ding et al. 2019 …

• 15-

Experiment

Results

At least 15.11% ~ 22.32% AUC improvement!

19.68% 22.32% 15.11%

• 16-

Experiment

Results

Robust and Effective!

• 17-

Conclusion

• Traditional machine learning based methods

perform poor for feature learning on large graph.

• Traditional deep graph model cannot effectively

capture the cross-modality interactions between the network structure and node attribute.

• We propose a deep joint representation learning

framework via a dual autoencoder to capture the complex cross-modality interactions between the network structure and node attribute.

• 18-

Reference

• [LOF] Breunig, Markus M., et al. "LOF: identifying density-based local
• utliers." KDD. 2000.
• [SCAN] Xu, Xiaowei, et al. "Scan: a structural clustering algorithm for

networks." KDD. 2007.

• [FocusCO] Perozzi, Bryan, et al. "Focused clustering and outlier detection in

large attributed graphs." KDD. 2014.

• [AMEN] Perozzi, Bryan, and Leman Akoglu. "Scalable anomaly ranking of

attributed neighborhoods." SIAM, 2016.

• [Radar] Li, Jundong, et al. "Radar: Residual Analysis for Anomaly Detection in

Attributed Networks." IJCAI. 2017.

• [GAT] Veličković, Petar, et al. "Graph attention networks." ICLR. 2018.
• [ANOMALOUS] Peng, Zhen, et al. "ANOMALOUS: A Joint Modeling Approach

for Anomaly Detection on Attributed Networks." IJCAI. 2018.

• [Dominant] Ding, Kaize, et al. "Deep anomaly detection on attributed

networks." SIAM, 2019.

• 19-

Thanks

Thanks for listening!

Contact: isfanhy@hrbust.edu.cn Home Page: https://haoyfan.github.io/