You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: beginner_source/bettertransformer_tutorial.rst
+3-3
Original file line number
Diff line number
Diff line change
@@ -127,14 +127,14 @@ We run the model on CPU, and collect profile information:
127
127
* The first run uses traditional ("slow path") execution.
128
128
* The second run enables BT fastpath execution by putting the model in inference mode using `model.eval()` and disables gradient collection with `torch.no_grad()`.
129
129
130
-
You can see a small improvement when the model is executing on CPU. Notice that the fastpath profile shows most of the execution time
130
+
You can see an improvement (whose magnitude will depend on the CPU model) when the model is executing on CPU. Notice that the fastpath profile shows most of the execution time
131
131
in the native `TransformerEncoderLayer` implementation `aten::_transformer_encoder_layer_fwd`.
132
132
133
133
.. code-block:: python
134
134
135
135
print("slow path:")
136
136
print("==========")
137
-
with torch.autograd.profiler.profile(use_cuda=True) as prof:
137
+
with torch.autograd.profiler.profile(use_cuda=False) as prof:
138
138
for i inrange(ITERATIONS):
139
139
output = model(model_input)
140
140
print(prof)
@@ -143,7 +143,7 @@ in the native `TransformerEncoderLayer` implementation `aten::_transformer_encod
143
143
144
144
print("fast path:")
145
145
print("==========")
146
-
with torch.autograd.profiler.profile(use_cuda=True) as prof:
146
+
with torch.autograd.profiler.profile(use_cuda=False) as prof:
Copy file name to clipboardExpand all lines: index.rst
+9-9
Original file line number
Diff line number
Diff line change
@@ -3,7 +3,7 @@ Welcome to PyTorch Tutorials
3
3
4
4
What's new in PyTorch tutorials?
5
5
6
-
* `Fast Transformer Inference with Better Transformer <https://pytorch.org/tutorials/intermediate/bettertransformer_tutorial.html?utm_source=whats_new_tutorials&utm_medium=bettertransformer>`__
6
+
* `Fast Transformer Inference with Better Transformer <https://pytorch.org/tutorials/beginner/bettertransformer_tutorial.html?utm_source=whats_new_tutorials&utm_medium=bettertransformer>`__
7
7
* `Introduction to TorchRec <https://pytorch.org/tutorials/intermediate/torchrec_tutorial.html?utm_source=whats_new_tutorials&utm_medium=torchrec>`__
8
8
* `Getting Started with Fully Sharded Data Parallel (FSDP) <https://pytorch.org/tutorials/intermediate/FSDP_tutorial.html?utm_source=whats_new_tutorials&utm_medium=FSDP>`__
9
9
* `Advanced model training with Fully Sharded Data Parallel (FSDP) <https://pytorch.org/tutorials/intermediate/FSDP_adavnced_tutorial.html?utm_source=whats_new_tutorials&utm_medium=FSDP_advanced>`__
@@ -214,6 +214,13 @@ What's new in PyTorch tutorials?
214
214
215
215
.. Text
216
216
217
+
.. customcarditem::
218
+
:header: Fast Transformer Inference with Better Transformer
219
+
:card_description: Deploy a PyTorch Transformer model using Better Transformer with high performance for inference
0 commit comments