Skip to content

Commit 49aec93

Browse files
Update channels_last memory format tutorial (#3434)
Co-authored-by: sekyondaMeta <127536312+sekyondaMeta@users.noreply.github.com>
1 parent 9e73898 commit 49aec93

File tree

1 file changed

+29
-5
lines changed

1 file changed

+29
-5
lines changed

intermediate_source/memory_format_tutorial.py

Lines changed: 29 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,28 @@
11
# -*- coding: utf-8 -*-
22
"""
3-
(beta) Channels Last Memory Format in PyTorch
3+
Channels Last Memory Format in PyTorch
44
*******************************************************
55
**Author**: `Vitaly Fedyunin <https://github.com/VitalyFedyunin>`_
66
7-
What is Channels Last
8-
---------------------
7+
.. grid:: 2
98
10-
Channels last memory format is an alternative way of ordering NCHW tensors in memory preserving dimensions ordering. Channels last tensors ordered in such a way that channels become the densest dimension (aka storing images pixel-per-pixel).
9+
.. grid-item-card:: :octicon:`mortar-board;1em;` What you will learn
10+
:class-card: card-prerequisites
11+
12+
* What is the channels last memory format in PyTorch?
13+
* How can it be used to improve performance on certain operators?
14+
15+
.. grid-item-card:: :octicon:`list-unordered;1em;` Prerequisites
16+
:class-card: card-prerequisites
17+
18+
* PyTorch v1.5.0
19+
* A CUDA-capable GPU
20+
21+
#########################################################################
22+
# Overview - What is channels last?
23+
# ---------------------------------
24+
25+
The channels last memory format is an alternative way of ordering NCHW tensors in memory preserving dimensions ordering. Channels last tensors ordered in such a way that channels become the densest dimension (aka storing images pixel-per-pixel).
1126
1227
For example, classic (contiguous) storage of NCHW tensor (in our case it is two 4x4 images with 3 color channels) look like this:
1328
@@ -19,7 +34,7 @@
1934
.. figure:: /_static/img/channels_last_memory_format.png
2035
:alt: channels_last_memory_format
2136
22-
Pytorch supports memory formats (and provides back compatibility with existing models including eager, JIT, and TorchScript) by utilizing existing strides structure.
37+
Pytorch supports memory formats by utilizing the existing strides structure.
2338
For example, 10x3x16x16 batch in Channels last format will have strides equal to (768, 1, 48, 3).
2439
"""
2540

@@ -387,3 +402,12 @@ def attribute(m):
387402
#
388403
# If you have feedback and/or suggestions for improvement, please let us
389404
# know by creating `an issue <https://github.com/pytorch/pytorch/issues>`_.
405+
406+
######################################################################
407+
# Conclusion
408+
# ----------
409+
#
410+
# This tutorial introduced the "channels last" memory format and demonstrated
411+
# how to use it for performance gains. For a practical example of accelerating
412+
# vision models using channels last, see the post
413+
# `here <https://pytorch.org/blog/accelerating-pytorch-vision-models-with-channels-last-on-cpu/>`_.

0 commit comments

Comments
 (0)