Roofline quantized conv3d/2d layer #3419
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/3419
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 New FailureAs of commit 30dc793 with merge base 73730e8 ( NEW FAILURE - The following job has failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
bot
added
the
CLA Signed
jainapurva
force-pushed
the
conv_roofline
branch
from
8fba23d to
79cdaec
Compare
| and not is_sm_at_least_100() | ||
| ) | ||
|
|
||
| if skip_conv_benchmarks: |
There was a problem hiding this comment.
I feel conditions seems a bit convoluted here
maybe:
if do_benchmarks:
if op_name in ("conv2d", "conv3d") and not is_sm_at_least_100():
print warning
else:
# can also move this part to a function to make it clearer
....
| r_speedup = None | ||
| # use roofline model to estimate gemm time using equivalent GEMM dims | ||
| r_bf16_gemm_time_s = float( | ||
| bf16_gemm_time_sympy.subs(M, gemm_M).subs(K, gemm_K).subs(N, gemm_N) |
There was a problem hiding this comment.
is the memory operations of conv the same as linear well?
ao/torchao/testing/training/roofline_utils.py
Line 332 in 0975a40
There was a problem hiding this comment.
As conv is an implicit gemm, I'm assuming the memory operations for gemm and conv should be same.
| print() | ||
| else: | ||
| # TODO: enable roofline analysis for conv | ||
| pass |
There was a problem hiding this comment.
if we share the same thing, we should remove this if/else branch and inline the code in if here I think
There was a problem hiding this comment.
I kept if/else just incase of an unexpected op_name. We can either make the code verify op_name in beginning to avoid any errors, or assume the input for op_name will always be correct
There was a problem hiding this comment.
we already verify that in L264 I think
|
|
||
| # real gemm benchmark time, also not added yet | ||
| # if enabled, also measured observed gemm time | ||
| # gemm benchmarks for conv not implemented, as conv uses implicit GEMM |
There was a problem hiding this comment.
we should run the conv ops I think?
No description provided.