THE IMPACT OF ADVERSARIALS WITHIN CNN- BASED IMAGE CLASSIFICATION - - PowerPoint PPT Presentation
THE IMPACT OF ADVERSARIALS WITHIN CNN- BASED IMAGE CLASSIFICATION By Josue Flores PI: Zhigang Zhu CCNY STEM Communities This material is based upon work supported by the National Science Foundation under Grant No. 1832567. Any opinions,
By Josue Flores PI: Zhigang Zhu
This material is based upon work supported by the National Science Foundation under Grant No. 1832567. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
CCNY STEM Communities
focuses on imitating portions of the complex human vision system, enabling computers to identify/process
humans do.
understand visual data is to feed it a plethora of images, thousands, even millions, if possible that have been labeled, and then subject those photos to various software applications, or algorithms, that allow the computer to hunt down patterns in all the elements that relate to those labels.
Recognition, Healthcare, etc..
An artificial neural network (ANN) is a computational model based on the structure and functions of biological neural networks. ANNs are considered nonlinear statistical data modeling tools where the complex relationships between inputs and
modeled or patterns are found. ANNs have three layers that are
first layer consists
Those neurons send data on to the second layer, which in turn sends the
the third layer.
Convolutional Neural Networks (CNNs), also known as ConvNets, act as a Deep Learning algorithm that takes an input image, assign importance (learnable weights and biases) to various aspects/objects in the image and be able to differentiate one from the other. The pre-processing required in a ConvNet is much lower as compared to other classification algorithms. The architecture of a CNN is analogous to that of the connectivity pattern of neurons in the human brain and was inspired by the organization of the Visual Cortex. Most ConvNets can successfully comprehend the spatial and temporal dependencies in an image through the application of relevant filters. The architecture performs a better fitting to the image dataset due to the reduction in the number of parameters involved and reusability of weights.
These methods include: Fast Gradient Sign Method (FGSM) Basic Iterative Method (BIM) DeepFool Carlini & Wagner Adversarial examples can be contrived and applied in multiple ways towards Neural Networks, especially ConvNets. Adversarial examples are images that contain nuanced aspects of alteration to them, that incur confusion or failure within a deep neural network’s ability to accurately classify images or information. This is primarily done is by incorporating perturbations (visual changes) to a notable number of pixels in an image.
Google Colaboratory
models (MobileNet_V2, ResNet50_V2, Inception_V3)
mentioned earlier.
various datasets including:
FASHION_MNIST
Fig (a). MobileNet_V2 Training Results Fig (b). ResNet50_V2 Training Results Fig (c). Inception_V3 Training Results
Figures (a), (b), and (c) were all pretrained models based on CNN architectures that were trained with the CIFAR-10 dataset. They were all trained with 5 epochs being the standard number of iterations over the entire dataset.
tV2
k/sites/autocar.co.uk/files/styles/flexslider_full/public/slideshow_image/0- pininfarina-battista.jpg?itok=3UsQ0zMD')
image_raw = tf.io.read_file(image_path) image = tf.image.decode_image(image_raw) image = preprocess(image) image_probs = pretrained_model.predict(image) plt.figure() plt.imshow(image[0]*0.5+0.5) # To change [-1, 1] to [0,1] loss_object = tf.keras.losses.CategoricalCrossentropy() def create_adversarial_pattern(input_image, input_label): with tf.GradientTape() as tape: tape.watch(input_image) prediction = pretrained_model(input_image) loss = loss_object(input_label, prediction) # Get the gradients of the loss w.r.t to the input image. gradient = tape.gradient(loss, input_image) # Get the sign of the gradients to create the perturbation signed_grad = tf.sign(gradient) return signed_grad _, image_class, class_confidence = get_imagenet_label(image_probs) plt.title('{} : {:.2f}% Confidence'.format(image_class, class_confidence*100),color = 'magenta') plt.show() # Get the input label of the image. retriever_index = 91 label = tf.one_hot(retriever_index, image_probs.shape[-1]) label = tf.reshape(label, (1, image_probs.shape[-1])) perturbations = create_adversarial_pattern(image, label) plt.imshow(perturbations[0]*0.5+0.5); # To change [-1, 1] to [0,1] epsilons = [0, 0.01, 0.1, 0.15,0.30] descriptions = [('Epsilon = {:0.3f}'.format(eps) if eps else 'Input') for eps in epsilons] for i, eps in enumerate(epsilons): adv_x = image + eps*perturbations adv_x = tf.clip_by_value(adv_x, -1, 1) display_images(adv_x, descriptions[i])
Figure 2. This figure illustrates how applying noise/perturbation to an image can incur misclassification in the case of a sports car.
Figure 3. This figure illustrates how applying noise/perturbation to an image can incur misclassification in the case of a street sign.
CNN models reacted differently and accordingly to various attacks.
to impact Inception V3 the most compared to other CNN architectures. These attacks ultimately skewed the image classification process especially when the epsilon of the perturbation gradually increased.
intensity of epsilon increases, the adversarial attack becomes more easily discernable to the human vision. As such, this renders the surreptitious aspect of the attack ineffective.
The Data illustrates how CNNs despite their supernatural efficiency in image classification, remain vulnerable to many forms of interference that impede upon their function. Not to mention, this demonstrates that CNNs must continuously be refined along with the new technological advances that are released over time. This is depicted by the creation of adversarial defenses to foil these unsightly examples, whether they are contrived, naturally or unnaturally. Additionally, I plan to procure more knowledge relating to the defensive measures and examine their interactions with prominent adversarial attack systems.
the National Science Foundation Grant No. 1832567