Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
177 changes: 177 additions & 0 deletions docs/tutorials/computer_vision.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,177 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Blazingly fast Computer Vision Models"
]
},
{
"cell_type": "raw",
"metadata": {
"vscode": {
"languageId": "raw"
}
},
"source": [
"<a target=\"_blank\" href=\"https://colab.research.google.com/drive/1GkzxTQW-2yCKXc8omE6Sa4SxiETMi8yC?usp=sharing\\\">\n",
" <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
"</a>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This tutorial demonstrates how to use the `pruna` package to optimize any custom computer vision model. We will use the `vit_b_16` model as an example. Any execution times given below are measured on a T4 GPU."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 1. Loading the CV Model\n",
"\n",
"First, load your ViT model.\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import torchvision\n",
"\n",
"model = torchvision.models.vit_b_16(weights=\"ViT_B_16_Weights.DEFAULT\").cuda()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 2. Initializing the Smash Config\n",
"\n",
"Next, initialize the smash_config."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from pruna import SmashConfig\n",
"\n",
"# Initialize the SmashConfig\n",
"smash_config = SmashConfig([\"x_fast\"])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3. Smashing the Model\n",
"\n",
"Now, you can smash the model, which will take around 5 seconds."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from pruna import smash\n",
"\n",
"# Smash the model\n",
"smashed_model = smash(\n",
" model=model,\n",
" smash_config=smash_config,\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 4. Preparing the Input"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"from torchvision import transforms\n",
"\n",
"# Generating a random image\n",
"image = np.random.randint(0, 256, size=(224, 224, 3), dtype=np.uint8)\n",
"input_tensor = transforms.ToTensor()(image).unsqueeze(0).cuda()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 5. Running the Model\n",
"\n",
"After the model has been compiled, we run inference for a few iterations as warm-up. This will take around 8 seconds."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# run some warm-up iterations\n",
"for _ in range(5):\n",
" smashed_model(input_tensor)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Finally, run the model to transcribe the audio file with accelerated inference."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Display the result\n",
"smashed_model(input_tensor)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Wrap Up\n",
"\n",
"Congratulations! You have successfully smashed a CV model. You can now use the `pruna` package to optimize any custom CV model. The only parts that you should modify are step 1, 4 and 5 to fit your use case"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "pruna",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.10.15"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
40 changes: 37 additions & 3 deletions docs/tutorials/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -87,10 +87,44 @@ These tutorials will guide you through the process of using |pruna| to optimize

Learn how to use the ``target_modules`` parameter to target specific modules in your model.

.. grid-item-card:: Blazingly Fast Computer Vision

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as mentioned in the other PR, perhaps we can automatically generate this file to avoid missing tutorials? Anyhow, it might be better to just focus on is during a bigger documentation rewrite.

:text-align: center
:link: ./computer_vision.ipynb

Optimize any ``computer vision`` model with ``x_fast`` ``compilation``.

.. grid-item-card:: Recover Quality after Quantization
:text-align: center
:link: ./recovery.ipynb

Recover quality using ``text_to_image_perp`` after ``diffusers_int8`` ``quantization``.

.. grid-item-card:: Distribute across GPUs with Ring Attention
:text-align: center
:link: ./ring_attn.ipynb

Distribute your ``Flux`` model across multiple GPUs with ``ring_attn`` and ``torch_compile``.

.. toctree::
:hidden:
:maxdepth: 1
:caption: Pruna
:glob:

./*

image_generation
video_generation
llms
reasoning_llm
asr_tutorial
cv_cpu
diffusion_quantization_acceleration
evaluation_agent_cmmd
sana_diffusers_int8
flux2klein4b_tutorial
sd_deepcache
deploying_sana_tutorial
target_modules_quanto
portable_compilation
llm_quantization_compilation_acceleration
computer_vision
recovery
ring_attn
Loading