Towards Assumption-free Unsupervised Domain Adaptation for Visual - PowerPoint PPT Presentation

Towards Assumption-free Unsupervised Domain Adaptation for Visual recognition 马锦华 数据科学与计算机学院 中山大学

Outline • Background and Motivations • Unsupervised Domain Adaptation - Estimating label information in target domain - Domain-shared Group-sparse Representation • Conclusion and Future Works

Background and Motivations • Traditional machine learning problem Training Machine Learning Data Model Training images from PIE dataset [1] Apply the learned Test Model Data Test images from PIE dataset [1] Result References: [1] T. Sim, et al. The cmu pose, illumination, and expression (pie) database. (AFGR 2002)

Background and Motivations • Dataset bias (generalization) problem: Training Test • Performance drops across datasets due to � � � � distributions mismatch, Training Dataset Test Dataset Source: Images of license plate are downloaded from Wikipedia

Background and Motivations • Basic Idea of Domain Adaptation Labeled training data (Source Domain) Domain Model* for target domain Adaptation Test data w or w/o labels (Model: (Target Domain) classifier/detector/estimator/…) • Good performance if many target labeled data are available Accuracy (%) of 1‐NN on test dataset Training Test 2000 target 1000 target 100 target No target Dataset Dataset data data data data Frontal faces Side faces 67.65 59.52 29.49 29.10 (PIE27) (PIE05) Face recognition accuracy (%) on test dataset Reference: T. Sim, et al. The cmu pose, illumination, and expression (pie) database. (AFGR 2002)

Background and Motivations • Labels are unavailable in test dataset in many applications - Large number of datasets (e.g., large-scale camera networks) - Difficulty of labeling (e.g., medical images) • Unsupervised Domain Adaptation Labeled training data Unsupervised (Source Domain) Model for target Domain domain Test data w/o labels Adaptation (Target Domain) 6

Background and Motivations • Applications Source images Target images Source images Target images Cross-dataset Object Recognition [1,2] Cross-domain Pose Estimation [3] Source image pairs Target image pairs Cross-domain Segmentation [5] Cross-dataset Re-identification [4] References: [1] K. Saenko, et al. Adversarial Discriminative Domain Adaptation. (CVPR 2017) [2] Baoyao Yang, Andy J. Ma, PC Yuen, Domain‐shared Group‐sparse Dictionary Learning for Unsupervised Domain Adaptation (AAAI 2018) [3] Baoyao Yang, Andy J. Ma, PC Yuen, Body Parts Synthesis for Cross‐Quality Pose Estimation (TCSVT 2018) [4] Andy J Ma, Pong C Yuen, Jiawei Li and Ping Li. Cross‐Domain Person Reidentification Using Domain Adaptation Ranking SVMs.(TIP 2015) [5] Yang Zhang, et al. Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes. (ICCV 2017)

Background and Motivations • Challenges in Unsupervised Domain Adaptation - Joint distributions alignment WITHOUT labeled data in target domain • Joint distributions mismatch � � : Source data � � : Source labels � ��� � , � � � � � � , � � � � : Target data � � : Target labels • Equal conditional distribution assumption � � � |� � � ��� � |� � � • Marginal distribution alignment � � � ~ ��� � �

Background and Motivations • Challenges in Unsupervised Domain Adaptation - Equal conditional distribution assumption NOT valid Training Dataset Test Dataset • Estimating labels in target domain for conditional distribution alignment � � � |� � ~ ��� � |� � � 9

Background and Motivations • Challenges in Unsupervised Domain Adaptation - Imbalanced unlabeled data in target domain • Example applications: person re-identification, object detection, pose estimation, etc. • In re-identification: # of negatives >> # of positives • Difficulty in estimating positive labels (minority class) accurately (a) Balanced data (b) Imbalanced data

Estimating Label Information • Estimating positive (minority-class) mean [1] • Estimating positive (minority-class) region [2] • Dynamic Graph matching [3] Reference 1. A J Ma, J W Li, P C Yuen and P Li, “Cross-domain person re-identification using domain adaptation ranking SVMs”, IEEE Transactions on Image Processing (TIP) , Vol. 24, No. 5, pp. 1599-1613, 2015. 2. J W Li, A J Ma and P C Yuen, “Semi-Supervised Region Metric Learning for Person Re- identification”, International Journal of Computer Vision (IJCV), In press, 2018 3. M Ye, A J Ma, L Zheng, J W Li, and P C Yuen, “Dynamic Label Graph Matching for Unsupervised Video Re-identification”, International Conference on Computer Vision (ICCV) , 2017. 11

Positive Mean Strategy • Following linear models, confidence score can � be calculated by weighted summation, i.e. • If positive data is available in target domain � � � � � � � �� � � �� � � (*) • Taking summation over positive samples in (*) � � � � � � � � � �� � � � (**) • (**) is necessary condition of (*) & only related to positive mean 12

Positive Mean Strategy • Estimator 1 using distribution relationship - If target prior probability �� � is known, � � �� � � � � � � � � � � �� �� � � : mean of unlabeled image pairs - � � � : mean of available negatives, estimation of � � - � � negative mean � � • Unreliable in practice � ≫ � � � - Estimation error is large, since � � � � � � � � � � � � � � � � � �� � � � � � � 13

Positive Mean Strategy • Estimator 2 using domain similarity � � � � � � � � � � � � � , - Assume that � � � � � � � � � � � � � � � � �� � � - Upper bound of estimation error is � � � � � � �� � � � � � � � � � � � � � � � � � � � � � � � � � � 14

Positive Mean Strategy • Estimator 3 using potential positive samples � � �� � � - Find potential positive samples by � � ,� � �� � �⋯�� � � � �� � max � � ��� ��� � � �� � �� � � ∝ exp � � �� � where � �� . 15

Positive Mean Strategy • Combining Estimated Means � and � � are employed for their reliability in � �� � �� - Only � imbalanced data � & � � are independent � be a random vector, � - � � � �� � �� � � �� � �� � � � � �� � � �� � � � � �� � , � �� � �� � � � �� � � - Solving the ML problem under Gaussian distribution assumption � � �� � � 1 � � � � � � � � �� � �� 16

Positive Mean Strategy • Adaptive Ranking SVMs - Asymmetric domain adaptation model on target domain � �� � � � � � - Adaptive Ranking SVMs 1 � � � μ� � � � � � � min 2 �,�,� � � � 1 � � � �� � � � �� � �� �� � � � �. �. θ� � � 1 � � � � 0, 0 � � � 1, ∀� � � 17

Experimental Results on Re-ID • CMC curves comparing RankSVM [BMCV’10] training on target or source domain & non- learning metric i-LIDS to CUHK i-LIDS to PRID i-LIDS to VIPeR 18

Experimental Results on Re-ID • CMC curves comparing existing Domain Adaptation methods, i.e. TCA [TNN’11] & GFS [ICCV’11]. i-LIDS to CUHK i-LIDS to PRID i-LIDS to VIPeR 19

Experimental Results • Accuracy (%) comparing existing Re-ID algorithms on VIPeR Method source Rank 1 Rank 5 Rank 10 Rank 20 PRID 44.94 64.15 71.33 77.48 Ours CUHK 47.47 66.84 72.67 78.94 i‐LIDS 45.35 66.16 73.43 78.77 19.87 38.89 49.37 65.73 SDALF [CVPR’10] ‐‐‐‐ CPS [BMCV’11] ‐‐‐‐ 21.84 44.00 57.21 71.00 eLDFV [ECCVW’12] ‐‐‐‐ 22.34 47.00 60.04 71.00 CIS [PAMI’13] ‐‐‐‐ 24.24 44.91 56.55 69.40 eSDC [CVPR’13] ‐‐‐‐ 26.74 50.70 62.37 76.36 20

Positive Region Strategy • Region representation - Regularized affine hull based positive region representation � ∑ � � � 1, � � • � � � ∑ � � � �� � � � - Region-to-point distance metric learning • Maximize the distance between the positive region and the negatives Positives Positive region Negatives Region‐to‐point distance 21

Positive Region Strategy • Positive region estimation without using positives - Positive neighbors close to the positives �� � � ∈ � � � ∃�, � � � �� � • � �� � � � - Positive region close to the region of positive neighbors �� � ∑ � � � ��� �� ∑ � � � 1, � � • � � � � � Positives Positive neighbors Estimated positive region Ground truth positive region Negatives Region‐to‐point distance 22