Food Recognition using Fusion of Classifiers based on CNNs Eduardo - - PowerPoint PPT Presentation
ICIAP 2017 Food Recognition using Fusion of Classifiers based on CNNs Eduardo Aguilar, Marc Bolaos, and Petia Radeva Presented by: Petia Radeva Contents The problem of automatic food analysis Convolutional Neural Network (CNN)
Presented by: Petia Radeva
Eduardo Aguilar, Marc Bolaños, and Petia Radeva
24 hours dietary recall Food frequency questionnaire
Automatic visual food recognition tools for dietary assessment.
19:28
Food Detection Food Recognition Food Localization Food Segmentation for portion estimation Food Ingredients Recognition
19:28 Mainly focused on weight reduction, not so healthy habits!
19:28
200 400 600 800 1000 FoodDB ImageNet 231 1000 50000 100000 150000 200000 231 1000 200000 500000 1000000 1500000 FoodDB ImageNet 150000 1400000 50000000 100000000 150000000 200000000 250000000 300000000 150000 1400000 28000000
Intra-class variations Inter-class similarities Mixed items Bad Labeled
19:28
19:28 Inception model
19:28
0.63 0.35 0.02
0.55
0.31 0.14
19:28
0.55 0.31 0.14 …………… 0.63 0.35 0.02 ……………
1 2 … K models 1 2 …. C classes
19:28
0.55 0.31 0.14 0.63 0.35 0.02
0.55 0.31 0.14 0.63 0.35 0.02
0.55 0.31 0.14 0.63 0.35 0.02
0.55 0.31 0.14 ……………. 0.63 0.35 0.02 …………….
0.51 0.34 0.15 ……………. 0.61 0.29 0.1 …………….
Mean DTi
19:28
0.55 0.31 0.14 0.63 0.35 0.02 ………….
0.55 0.31 0.14 0.63 0.35 0.02 …………
0.55 0.31 0.14 0.63 0.35 0.02 ………….
0.51 0.34 0.15 …………….. 0.61 0.29 0.1 ……………..
DT1 DT2 DTC
19:28
0.55 0.31 0.14 ……………….. 0.63 0.35 0.02 ………………..
0.51 0.34 0.15 ……………… 0.61 0.29 0.1 ………………
Euclidean distance-based similarity:
19:28
N(A, B) =1− 1 K *C
i, j (ai, j − b i, j )2
…................. ……………… 0.61 0.29 0.1 ……………… ………………… ……………….. 0.63 0.35 0.02 ………………..
C1 C2 C3 ….
S
1(A, B) = || A B||
|| A B||
||ς ||= 1 n Σµς (u
i )
where ||.|| is the relative cardinality of the fuzzy set.
Similarity based on symmetric distance:
where is the symmetric difference defined by the Hamming distances:::
S
2(A, B) =1− || A∇B||
A∇B
µA∇B =| µA(u)−µB(u) |
C1 C2 C3 ….
…................. ……………… 0.61 0.29 0.1 ……………… ………………… ……………….. 0.63 0.35 0.02 ………………..
C(A, B) = supu{µA B}
The consistensy index is defined as:
Classes: Bread, Dairy Products, Dessert, Egg, Fried food, Meat, Noodles_Pasta, Rice, Seafood, Soup, Vegetable_Fruit.
11 categories: meat, dessert, vegetales, pasta, rise, seafood, etc.
Dataset #Classes #Train #Val #Test Balanced Food-11 11 9,866 3,430 3,347
101 most frequent food in foodspotting.com: baby back ribs, chocolate cake, hot and sour soup, caesar salad, eggs benedict, etc.
Dataset #Classes #Train #Val #Test Balanced Food-101 101 75,750
X
#Classes 101 #Train 75,750 #Val
25,250 #Classes 11 #Train 9,866 #Val 3,430 #Test 3,347 Balanced
Food-11: Food-101:
Relation btw # of models and accuracy in Food101
80 82 84 86 ResNet50(1) ResNet50(2) ResNet50(3) InceptionV3(1) InceptionV3(2) InceptionV3(3)
Overall accuracy for different CNN models
85,5 86 86,5 87 87,5 88 88,5 89 c i1 i2 i3 n s1 s2 s3 6 models 5 models 4 models 3 models 2 models
19:28 Relation btw # of models and accuracy in Food11
89 90 91 92 93 94 ResNet50(1) ResNet50(2) GoogLeNet(2)
Accuracy of single models
93,4 93,6 93,8 94 94,2 94,4 94,6 94,8 95 c i1 i2 i3 i5 n s1 s2 s3 Series3 Series2 Series1
19:28
19:28 Food category and class recognition
proposed a CNNs Fusion approach that improves the classification performance with respect to using CNN models separately.