-
Notifications
You must be signed in to change notification settings - Fork 26
/
Copy pathindex.html
191 lines (171 loc) · 10.9 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
<!DOCTYPE html>
<html lang="en">
<head>
<title>adversarial.js – Intro</title>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="Break neural networks in your browser.">
<link rel="shortcut icon" href="data/favicon.ico" />
<link rel="stylesheet" href="css/normalize.css">
<link rel="stylesheet" href="css/skeleton.css">
<link rel="stylesheet" href="css/style.css">
<script async src="https://www.googletagmanager.com/gtag/js?id=G-C02BRW1FMK"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'G-C02BRW1FMK');
</script>
</head>
<body>
<div class="container">
<div id="header" class="row">
<div id="logo" class="one-half column"><a href="."><h1>adversarial.js</h1></a></div>
<div id="nav" class="one-half column">
<a href="." class="active">Intro</a> · <a href="examples.html">Examples</a> · <a href="faq.html">FAQ</a> · <a href="api.html">API</a> · <a href="https://github.com/kennysong/adversarial.js" target="_blank">GitHub</a>
</div>
</div>
<div id="chrome-banner" class="warning-banner">⚠️ It looks like you're not using Google Chrome. This demo <strong>may not work properly</strong>. If it breaks, please try in Google Chrome.</div>
<div id="mobile-banner" class="error-banner">⚠️ It looks like you're on a mobile device without 32-bit WebGL. The demo <strong>will not work properly</strong>. Please try on desktop Google Chrome.</div>
<div id="banner">Break neural networks in your browser.</div>
<div id="dashboard">
<div id="top-row" class="row border-bottom">
<div class="center">Everything runs client-side – there is no server! Try the demo:</div>
</div>
<div class="row border-bottom">
<span id="select-model-label">Select a model: </span>
<select id="select-model">
<option value="" selected disabled>Select model</option>
<option value="mnist">MNIST (digit recognition)</option>
<option value="gtsrb">GTSRB (street sign recognition)</option>
<option value="cifar">CIFAR-10 (object recognition, small)</option>
<option value="imagenet">ImageNet (object recognition, large)</option>
</select>
</div>
<div class="row">
<div class="one-half column border-right">
<div id="original-image-overlay" class="overlay"></div>
<h5 class="center">Original Image</h5>
</div>
<div class="one-half column">
<div id="adversarial-image-overlay" class="overlay"></div>
<h5 class="center">Adversarial Image</h5>
</div>
</div>
<div class="row border-bottom">
<div class="one-half column split border-right">
<div id="original-canvas-overlay" class="overlay"></div>
<div class="splitleft61">
<canvas id="original" class="tableimg" width="224" height="224"></canvas>
</div>
<div class="splitright39">
<button id="next-image">Next Image ⟳</button>
</div>
</div>
<div class="one-half column split">
<div id="adversarial-canvas-overlay" class="overlay"></div>
<div class="splitleft55">
<canvas id="adversarial" class="tableimg" width="224" height="224"></canvas>
<canvas id="adversarial-noise" class="tableimg" width="224" height="224"></canvas>
</div>
<div class="splitright45">
<label for="select-target">Turn this image into a:</label>
<select id="select-target" class="u-full-width" data-model="mnist">
<option value="0">0</option>
<option value="1">1</option>
<option value="2">2</option>
<option value="3">3</option>
<option value="4">4</option>
<option value="5">5</option>
<option value="6">6</option>
<option value="7">7</option>
<option value="8">8</option>
<option value="9">9</option>
</select>
<label for="select-attack">Select an attack:</label>
<select id="select-attack" class="u-full-width">
<option value="cw">Carlini & Wagner (strongest)</option>
<option value="jsma">Jacobian-based Saliency Map Attack (stronger)</option>
<option value="jsmaOnePixel">Jacobian-based Saliency Map Attack 1-Pixel (stronger)</option>
<option value="bimTargeted">Basic Iterative Method (stronger)</option>
<option value="fgsmTargeted">Fast Gradient Sign Method (weak)</option>
</select>
<div class="center">
<button id="generate-adv" class="button-primary">Generate</button>
<div id="latency-msg">This will take a few seconds.</div>
<div id="difference">Can you see the difference? <a id="view-noise">View noise.</a></div>
<div id="difference-noise">Noise is magnified. <a id="view-image">View image.</a></div>
</div>
</div>
</div>
</div>
<div class="row center" style="display:flex;"> <!-- The flex here keeps the child columns the same height -->
<div class="one-half column border-right">
<div id="original-prediction-overlay" class="overlay"></div>
<h5>Prediction</h5>
<button id="predict-original" class="button-primary">Run Neural Network</button>
<div id="prediction"></div>
<div id="prediction-status"></div>
</div>
<div class="one-half column">
<div id="adversarial-prediction-overlay" class="overlay"></div>
<h5>Prediction</h5>
<button id="predict-adv" class="button-primary" disabled>Click "Generate" First</button>
<div id="prediction-adv"></div>
<div id="prediction-adv-status"></div>
</div>
</div>
</div>
<hr>
<div class="faq">
<h4>What is the demo doing?</h4>
<p>Neural networks achieve superhuman performance in many areas, but they are easily fooled.</p>
<p>In the demo above, we can force neural networks to predict anything we want. By adding nearly-invisible noise to an image, we turn "1"s into "9"s, "Stop" signs into "120 km/hr" signs, and dogs into hot dogs.</p>
<p>These noisy images are called <a href="https://en.wikipedia.org/wiki/Adversarial_machine_learning#Specific_Attacks_Types">adversarial examples</a>. They break the integrity of machine learning systems, and the illusion of their superhuman performance.</p>
<h4>Why does this matter?</h4>
<p>Our world is becoming increasingly automated, yet these systems have strange failure modes.</p>
<p>If machine learning systems are not properly defended, attackers could:</p>
<ul>
<li>Impersonate others in facial recognition systems</li>
<li>Force autonomous vehicles to misrecognize street signs & obstacles</li>
<li>Bypass content moderation and spam filters in social networks</li>
<li>Inject adversarial bytes into malware to bypass antivirus systems</li>
<li>Digitally alter numbers on a check in a mobile banking app</li>
<li>(and more)</li>
</ul>
<h4>Is this limited to image classification with neural networks?</h4>
<p>No. Adversarial examples exist for almost every machine learning task: <a href="https://nicholas.carlini.com/code/audio_adversarial_examples">speech recognition</a>, <a href="https://arxiv.org/pdf/1804.07998.pdf">text classification</a>, <a href="https://dl.acm.org/doi/10.1145/3308558.3313533">fraud detection</a>, <a href="https://www.ericswallace.com/imitation">machine translation</a>, <a href="https://adversarialpolicies.github.io/">reinforcement learning</a>, ....</p>
<p>Moreover, all machine learning models (not just neural networks) are vulnerable. In fact, simpler models such as logistic regression are <a href="https://arxiv.org/pdf/1412.6572.pdf">even more easily attacked</a>.</p>
<p>Finally – beyond adversarial examples – there are many more adversarial attack vectors, including data poisoning, model backdooring, data extraction, and model stealing.</p>
<h4>How do I defend against adversarial examples?</h4>
<p>There are several proposed defenses, including adversarial training and admission control.</p>
<p>However, no defense is universal and many have proven ineffective, so work with an expert to quantify your risks and invest in defenses appropriately.</p>
<p>(What happens if someone can make your system predict anything they want?).</p>
<h4>Where can I learn more?</h4>
<p>Here's a list of good resources, in rough order of approachability:</p>
<ul>
<li><a href="faq.html">The full FAQ</a></li>
<li><a href="examples.html">The directory of attacks</a> (try running locally and playing with various settings)</li>
<li><span class="badge">[Blog]</span> CleverHans – <a href="http://www.cleverhans.io/security/privacy/ml/2016/12/16/breaking-things-is-easy.html">start here</a></li>
<li><span class="badge">[Blog]</span> Gradient Science – <a href="https://gradientscience.org/intro_adversarial/">start here</a></li>
<li><span class="badge">[Tutorial]</span> <a href="https://adversarial-ml-tutorial.org/">Adversarial Robustness - Theory and Practice</a></li>
<li><span class="badge">[Paper]</span> <a href="https://arxiv.org/pdf/1611.03814.pdf">SoK: Towards the Science of Security and Privacy in Machine Learning</a></li>
<li><span class="badge">[Paper]</span> <a href="https://arxiv.org/pdf/1712.03141.pdf">Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning</a></li>
</ul>
<p>Last – feel free to <a href="mailto:[email protected]">email me questions</a>.</p>
<div id="banner-faq-link"><a href="faq.html">See the full FAQ.</a></div>
</div>
</div>
<!-- Hack to load a sample of ImageNet images that we can access in canvas -->
<img class="0" style="display:none;">
<img class="1" style="display:none;">
<img class="2" style="display:none;">
<img class="3" style="display:none;">
<img class="4" style="display:none;">
<img class="5" style="display:none;">
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/[email protected]/dist/tf.min.js" type="text/javascript"></script>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/[email protected]"></script>
<script src="js/intro.js" type="module"></script>
</body>
</html>