|
1 | 1 | # -*- coding: utf-8 -*-
|
2 | 2 | """
|
3 |
| -(beta) Channels Last Memory Format in PyTorch |
| 3 | +Channels Last Memory Format in PyTorch |
4 | 4 | *******************************************************
|
5 | 5 | **Author**: `Vitaly Fedyunin <https://github.com/VitalyFedyunin>`_
|
6 | 6 |
|
7 |
| -What is Channels Last |
8 |
| ---------------------- |
| 7 | +.. grid:: 2 |
9 | 8 |
|
10 |
| -Channels last memory format is an alternative way of ordering NCHW tensors in memory preserving dimensions ordering. Channels last tensors ordered in such a way that channels become the densest dimension (aka storing images pixel-per-pixel). |
| 9 | + .. grid-item-card:: :octicon:`mortar-board;1em;` What you will learn |
| 10 | + :class-card: card-prerequisites |
| 11 | +
|
| 12 | + * What is the channels last memory format in PyTorch? |
| 13 | + * How can it be used to improve performance on certain operators? |
| 14 | +
|
| 15 | + .. grid-item-card:: :octicon:`list-unordered;1em;` Prerequisites |
| 16 | + :class-card: card-prerequisites |
| 17 | +
|
| 18 | + * PyTorch v1.5.0 |
| 19 | + * A CUDA-capable GPU |
| 20 | +
|
| 21 | +######################################################################### |
| 22 | +# Overview - What is channels last? |
| 23 | +# --------------------------------- |
| 24 | +
|
| 25 | +The channels last memory format is an alternative way of ordering NCHW tensors in memory preserving dimensions ordering. Channels last tensors ordered in such a way that channels become the densest dimension (aka storing images pixel-per-pixel). |
11 | 26 |
|
12 | 27 | For example, classic (contiguous) storage of NCHW tensor (in our case it is two 4x4 images with 3 color channels) look like this:
|
13 | 28 |
|
|
19 | 34 | .. figure:: /_static/img/channels_last_memory_format.png
|
20 | 35 | :alt: channels_last_memory_format
|
21 | 36 |
|
22 |
| -Pytorch supports memory formats (and provides back compatibility with existing models including eager, JIT, and TorchScript) by utilizing existing strides structure. |
| 37 | +Pytorch supports memory formats by utilizing the existing strides structure. |
23 | 38 | For example, 10x3x16x16 batch in Channels last format will have strides equal to (768, 1, 48, 3).
|
24 | 39 | """
|
25 | 40 |
|
@@ -387,3 +402,12 @@ def attribute(m):
|
387 | 402 | #
|
388 | 403 | # If you have feedback and/or suggestions for improvement, please let us
|
389 | 404 | # know by creating `an issue <https://github.com/pytorch/pytorch/issues>`_.
|
| 405 | + |
| 406 | +###################################################################### |
| 407 | +# Conclusion |
| 408 | +# ---------- |
| 409 | +# |
| 410 | +# This tutorial introduced the "channels last" memory format and demonstrated |
| 411 | +# how to use it for performance gains. For a practical example of accelerating |
| 412 | +# vision models using channels last, see the post |
| 413 | +# `here <https://pytorch.org/blog/accelerating-pytorch-vision-models-with-channels-last-on-cpu/>`_. |
0 commit comments