Challenges in Vessel Behavior and Anomaly Detection: From Classical - - PowerPoint PPT Presentation

CAI 2020 | Petry et al. | Challenges in Vessel Behavior and Anomaly Detection

Challenges in Vessel Behavior and Anomaly Detection: From Classical Machine Learning to Deep Learning

Lucas May Petry1, Amilcar Soares2, Vania Bogorny1, Bruno Brandoli2, Stan Matwin2

33rd Canadian Conference on Artificial Intelligence

1Programa de Pós-Graduação em Ciências da

Computação (PPGCC), Universidade Federal de Santa Catarina (UFSC), Florianópolis, Brazil

2Institute for Big Data Analytics, Dalhousie

University, Halifax, Canada

CAI 2020 | Petry et al. | Challenges in Vessel Behavior and Anomaly Detection

Outline

• Introduction
• Related Work
• Objective and Contributions
• Research Challenges
• Conclusion

2

CAI 2020 | Petry et al. | Challenges in Vessel Behavior and Anomaly Detection

Introduction

• Maritime transportation represents 90% of all

international trade volume [22]

• Expansion of maritime activities and development of the

Automatic Identification System (AIS)

• Development of maritime monitoring systems

○ Prevent vessel accidents ○ Detect illegal activities ○ Protect the marine fauna and flora

• High volume of data makes real-time monitoring more

challenging to maritime agents

○ Detect anomalies, changes of behavior, events

3

CAI 2020 | Petry et al. | Challenges in Vessel Behavior and Anomaly Detection

Introduction

4

CAI 2020 | Petry et al. | Challenges in Vessel Behavior and Anomaly Detection

Related Work

5

Work Main approach Unsupervised No data pretraining Multi-sensor data Event detection Terroso et al. [24] Algorithm/rules Yes Yes* Patroumpas et al. [16] Queries/rules Yes Varlamis et al. [25]** Algorithm/rules Yes Yes Wen et al. [19] Probability model Yes Lei [9]** Clustering Yes Yes Soares et al. [18] Queries/rules Yes Yes Anomaly detection Bomberger et al. [28] Neural network Yes Ristic et al. [26] Kernel Density Estimation Yes Riveiro et al. [27] Gaussian model Nguyen et al. [14] Neural network Yes Varlamis et al. [29] Clustering Yes *They only use the weather description (e.g. sunny, rainy) as a deciding factor about detected abnormal low speed behavior. **Only a single type of behavior/event is detected.

CAI 2020 | Petry et al. | Challenges in Vessel Behavior and Anomaly Detection

Objective and Contributions

Objective Present research gaps and challenges in machine learning for detecting different types of vessel behavior, considering several constraints imposed by real-time data streams and the maritime monitoring domain. Contributions

• Short survey of the state of the art
• Extensive discussion on major topics with opportunities
• f research on vessel behavior detection with machine

learning

6

CAI 2020 | Petry et al. | Challenges in Vessel Behavior and Anomaly Detection

Research Challenges

7

Highly-dimensional data points derived from multiple sources Provide means for interpreting detected behaviors Detection of behavior recurrence Detection of vessel behaviors Absent knowledge of behaviors or labels present in the data Limited or non-existent labeled data

Tasks Data Issues

CAI 2020 | Petry et al. | Challenges in Vessel Behavior and Anomaly Detection

Research Challenges

• Behavior detection

○ Concept drift techniques [1, 8, 17]

■ Limited to univariate data, lack of interpretability ■ They can only detect points of change

• Recurrent behaviors

○ Toeplitz Inverse Covariance-based Clustering (TICC) [6]

■ Markov Random Fields (MRFs) for representing clusters/behaviors

• MRFs may provide insight for behavior interpretation!

■ Number of behaviors should be known apriori ■ Assumes all data is available

8

CAI 2020 | Petry et al. | Challenges in Vessel Behavior and Anomaly Detection

Research Challenges

• Deep learning

○ No work has used it for vessel behavior detection ○ Convolutional Neural Networks (CNNs)

■ Satellite imagery can be expensive and make detecting certain behaviors very difficult or even impossible [34] ■ They have been used for trajectory classification/prediction based

• n movement features [3, 30]

■ Visual techniques can be used for achieving interpretability [21, 31]

• Big, yet limited data

○ High volume of data available, but lacks labels ○ Use knowledge of previous works as a ground truth ○ Transfer learning for learning from a few examples [32, 33] ○ Generative Adversarial Networks (GANs) for synthesizing new behavior data

9

CAI 2020 | Petry et al. | Challenges in Vessel Behavior and Anomaly Detection

Conclusion

• Maritime monitoring has experienced significant

progress in the last decade

• Existing works do not take full advantage of machine

learning techniques for vessel behavior detection

• We presented several research gaps in the field,

indicating opportunities for future works

• We hope to instigate the development of new

algorithms, methods, and tools for maritime monitoring

10

CAI 2020 | Petry et al. | Challenges in Vessel Behavior and Anomaly Detection

References

1. Bifet, A., Gavalda, R.: Learning from time-changing data with adaptive windowing. In: Proc. of the 2007 SIAM int. conf. on data mining. pp. 443–448. SIAM (2007) 2. Claramunt, C., Ray, C., Salmon, L., Camossi, E., Hadzagic, M., Jousselme, A., Andrienko, G., Andrienko, N., Theodoridis, Y., Vouros, G.: Maritime data integration and analysis: recent progress and research challenges. Adv. in Database Technology - EDBT 2017, 192–197 (2017) 3. Dabiri, S., Heaslip, K.: Inferring transportation modes from gps trajectories using a convolutional neural network. Transportation research part C: emerging technologies 86, 360–371 (2018) 4. Doshi-Velez, F., Kim, B.: Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 (2017) 5. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Adv. in neural inf. processing systems. pp. 2672–2680 (2014) 6. Hallac, D., Vare, S., Boyd, S., Leskovec, J.: Toeplitz inverse covariance-based clustering of multivariate time series data. In: Proc. of the 23rd ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining. pp. 215–223. ACM (2017) 7. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural

• networks. In: Adv. in neural inf. processing systems. pp. 1097–1105 (2012)

8. Lee, J., Magoules, F.: Detection of concept drift for learning from stream data. In: 14th Int. Conf. on High Performance Computing and Communication & 9th Int. Conf. on Embedded Software and

• Systems. pp. 241–245. IEEE (2012)

11

CAI 2020 | Petry et al. | Challenges in Vessel Behavior and Anomaly Detection

References

9. Lei, P.R.: Mining maritime traffic conflict trajectories from a massive ais data. Knowledge and Inf. Systems pp. 1–27 (2019) 10. Lipton, Z.C.: The mythos of model interpretability. arXiv preprint arXiv:1606.03490 (2016) 11. Lv, J., Li, Q., Sun, Q., Wang, X.: T-conv: a convolutional neural network for multi-scale taxi trajectory

• prediction. In: 2018 IEEE int. conf. on big data and smart computing (bigcomp). pp. 82–89. IEEE

(2018) 12. Manzoor, E., Lamba, H., Akoglu, L.: xstream: Outlier detection in feature-evolving data streams. In:

• Proc. of the 24th ACM SIGKDD Int. Conf. on Knowledge Discovery & Data Mining. pp. 1963–1972.

KDD ’18, ACM (2018) 13. Mohammadi, M., Al-Fuqaha, A.: Enabling cognitive smart cities using big data and machine learning: Approaches and challenges. IEEE Communications Magazine 56(2), 94–101 (2018) 14. Nguyen, D., Vadaine, R., Hajduch, G., Garello, R., Fablet, R.: A multi-task deep learning architecture for maritime surveillance using ais data streams. In: 5th Int. Conf. on Data Science and Advanced Analytics (DSAA). pp. 331–340. IEEE (2018) 15. Nyman, E.: Techno-optimism and ocean governance: New trends in maritime monitoring. Marine Policy 99, 30–33 (2019) 16. Patroumpas, K., Artikis, A., Katzouris, N., Vodas, M., Theodoridis, Y., Pelekis, N.: Event recognition for maritime surveillance. In: Adv. in Database Technology - EDBT. pp. 629–640 (2015)

12

CAI 2020 | Petry et al. | Challenges in Vessel Behavior and Anomaly Detection

References

17. Qahtan, A.A., Alharbi, B., Wang, S., Zhang, X.: A pca-based change detection framework for multidimensional data streams: Change detection in multidimensional data streams. In: Proc. of the 21th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining. pp. 935–944. ACM (2015) 18. Soares, A., Dividino, R., Abreu, F., Brousseau, M., Isenor, A.W., Webb, S., Matwin, S.: Crisis: Integrating ais and ocean data streams using semantic web standards for event detection. In: Int.

• Conf. on Military Communications and Inf. Systems (2019a)

19. Wen, Y., Zhang, Y., Huang, L., Zhou, C., Xiao, C., Zhang, F., Peng, X., Zhan, W., Sui, Z.: Semantic modelling of ship behavior in harbor based on ontology and dynamic bayesian network. ISPRS Int. Journal of Geo-Inf. 8(3), 107 (2019) 20. Zhao, L., Shi, G.: Maritime anomaly detection using density-based clustering and recurrent neural

• network. The Journal of Navigation 72(4), 894–916 (2019)

21. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative

• localization. In: Proc. of the IEEE conf. on computer vision and pattern recognition. pp. 2921–2929

(2016) 22. Vespe, M., Visentini, I., Bryan, K., and Braca, P.. Unsupervised learning of maritime traffic patterns for anomaly detection. In 9th IET Data Fusion & Target Tracking Conf. (DF&TT 2012): Algorithms & Applications, pages 1–5. IET, (2012) 23. Le Guillarme, N. and Lerouvreur, X.. Unsupervised extraction of knowledge from s-ais data for maritime situational awareness. In Proc. of the 16th Int. Conf. on Information Fusion, pages 2025–2032 (2013) 24.

13

CAI 2020 | Petry et al. | Challenges in Vessel Behavior and Anomaly Detection

References

24. Terroso-Saenz, F., Valdes-Vela, M., and Skarmeta-Gomez, A. F.. A complex event processing approach to detect abnormal behaviours in the marine environment. Information Systems Frontiers, 18(4):765–780 (2016) 25. Varlamis, I., Tserpes, K., and Sardianos, C.. Detecting search and rescue missions from ais data. In 2018 IEEE 34th International Conference on Data Engineering Workshops (ICDEW), pages 60–65. IEEE (2018) 26. Ristic, B., La Scala, B., Morelande, M., and Gordon, N.. Statistical analysis of motion patterns in ais data: Anomaly detection and motion prediction. In 2008 11th International Conference on Information Fusion, pages 1–7. IEEE (2008) 27. Riveiro, M., Falkman, G., and Ziemke, T.. Improving maritime anomaly detection and situation awareness through interactive visualization. In 2008 11th International Conference on Information Fusion, pages 1–8. IEEE (2008) 28. Bomberger, N. A., Rhodes, B. J., Seibert, M., and Waxman, A. M.. Associative learning of vessel motion patterns for maritime situation awareness. In 2006 9th International Conference on Information Fusion, pages 1–8. IEEE (2006) 29. Varlamis, I., Tserpes, K., Etemad, M., Soares, A., and Matwin, S.. A network abstraction of multi-vessel trajectory data for detecting anomalies. In EDBT/ICDT Workshops (2019) 30. Lv J., Li Q., Sun Q., and Wang X.. T-conv: a convolutional neural network for multi-scale taxi trajectory prediction. In 2018 IEEE international conference on big data and smart computing (bigcomp), pages 82–89. IEEE (2018)

14

CAI 2020 | Petry et al. | Challenges in Vessel Behavior and Anomaly Detection

References

31. Simonyan K., Vedaldi A., and Zisserman A.. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 (2013) 32. Vinyals O., Blundell C., Lillicrap T., Wierstra D., et al. Matching networks for one shot learning. In Advances in neural information processing systems, pages 3630–3638 (2016) 33. Snell J., Swersky K., and Zemel R.. Prototypical networks for few-shot learning. In Advances in Neural Information Processing Systems, pages 4077–4087 (2017) 34. Nyman, Elizabeth. Techno-optimism and ocean governance: New trends in maritime monitoring. Marine Policy, 99:30–33 (2019)

15

CAI 2020 | Petry et al. | Challenges in Vessel Behavior and Anomaly Detection

Challenges in Vessel Behavior and Anomaly Detection: From Classical Machine Learning to Deep Learning

Lucas May Petry lucas.petry@posgrad.ufsc.br

33rd Canadian Conference on Artificial Intelligence

Thank you!