Home
This Title All WIREs
WIREs RSS Feed
How to cite this WIREs title:
WIREs Data Mining Knowl Discov
Impact Factor: 2.541

A dynamic‐adversarial mining approach to the security of machine learning

Full article on Wiley Online Library:   HTML PDF

Can't access this content? Tell your librarian.

Operating in a dynamic real‐world environment requires a forward thinking and adversarial aware design for classifiers beyond fitting the model to the training data. In such scenarios, it is necessary to make classifiers such that they are: (a) harder to evade, (b) easier to detect changes in the data distribution over time, and (c) be able to retrain and recover from model degradation. While most works in the security of machine learning have concentrated on the evasion resistance problem (a), there is little work in the areas of reacting to attacks (b) and (c). Additionally, while streaming data research concentrates on the ability to react to changes to the data distribution, they often take an adversarial agnostic view of the security problem. This makes them vulnerable to adversarial activity, which is aimed toward evading the concept drift detection mechanism itself. In this paper, we analyze the security of machine learning from a dynamic and adversarial aware perspective. The existing techniques of restrictive one‐class classifier models, complex learning‐based ensemble models, and randomization‐based ensemble models are shown to be myopic as they approach security as a static task. These methodologies are ill suited for a dynamic environment, as they leak excessive information to an adversary who can subsequently launch attacks which are indistinguishable from the benign data. Based on empirical vulnerability analysis against a sophisticated adversary, a novel feature importance hiding approach for classifier design is proposed. The proposed design ensures that future attacks on classifiers can be detected and recovered from. The proposed work provides motivation, by serving as a blueprint, for future work in the area of dynamic‐adversarial mining, which combines lessons learned from streaming data mining, adversarial learning, and cybersecurity. This article is categorized under: Technologies > Machine Learning Technologies > Classification Fundamental Concepts of Data and Knowledge > Motivation and Emergence of Data Mining
Illustration of exploratory attacks on a machine learning‐based spam filtering system
[ Normal View | Magnified View ]
The field of dynamic‐adversarial mining will derive from the concepts of machine learning, stream data mining, and cybersecurity
[ Normal View | Magnified View ]
Prediction landscapes as perceived by probing on the defender models. (a) Defender model is given as C1 ∧ C2. (b) Defender model given by randomly selecting C1 ∨ C2 to perform prediction. (c) Defender model is given by C1 (trained on feature X1), while C2 is kept hidden to detect adversarial activity. Blindspot (Margin) B2 denotes region for adversarial uncertainty
[ Normal View | Magnified View ]
Comparison of effective attack rate for nonrobust models, robust models, randomization‐based models, and randomization models with adversaries capable of filtering low confidence samples
[ Normal View | Magnified View ]
Anchor points attacks against a randomized defender. Top—Naive adversary disregards randomness. Bottom—Adversary with confidence filtering, repeated probing of exploration samples is used to weed out samples with inconsistent feedback
[ Normal View | Magnified View ]
Prediction landscape for the randomized feature bagging ensemble. Blindspots are perceived to be obscured, while high confidence spaces remain consistent across repeated probing
[ Normal View | Magnified View ]
Illustration of prediction landscape using simple and robust models, on two‐dimensional synthetic data. Left—Initial training data of the defender. Center—L1‐regularized linear SVM model for the defender (nonrobust). Right—L2‐regularized linear SVM model for the defender (robust)
[ Normal View | Magnified View ]
Illustration of anchor points attacks on a synthetic two‐dimensional dataset, using a restrictive one‐class classifier and a generalized two‐class classifiers for the defender's model
[ Normal View | Magnified View ]
Illustration of prediction landscape of a one‐class classifier. Smaller area of the legitimate samples indicate the resilience against probing‐based attacks
[ Normal View | Magnified View ]
Reducing adversarial uncertainty via filtering in anchor points attacks. After the exploration phase in (b), the adversary trains models on individual feature subspaces (X1 and X2). It then aggregates this information to clean out samples in low confidence areas (C: X1 ≠ C: X2). The final set of filtered high confidence samples are shown in (d)
[ Normal View | Magnified View ]
Anchor points‐based evasion attack framework with a high confidence filter phase
[ Normal View | Magnified View ]
Illustration of adversarial margin. It is given by the region of space that leads to evasion of defender's classifier C, but does not lead to evasion of all the informative features
[ Normal View | Magnified View ]
Illustration of data nullification attacks on the space of legitimate samples (blue)
[ Normal View | Magnified View ]
The attack–defense cycle between adversaries and system designers (i.e., defenders)
[ Normal View | Magnified View ]
Illustration of anchor points (AP) attacks. (Left–Right): The defender's model from its training data. The exploration phase depicting the seed (blue) and the AP samples (purple). The exploitation attack phase samples (red) generated based on the AP (Sethi et al., )
[ Normal View | Magnified View ]
Impact of defender model on attack severity. In case of restrictive models, attacks are harder to carry out, but will lead to inseparability from benign traffic
[ Normal View | Magnified View ]
Goals of proactive and reactive securities
[ Normal View | Magnified View ]

Browse by Topic

Fundamental Concepts of Data and Knowledge > Motivation and Emergence of Data Mining
Technologies > Classification
Technologies > Machine Learning

Access to this WIREs title is by subscription only.

Recommend to Your
Librarian Now!

The latest WIREs articles in your inbox

Sign Up for Article Alerts