The issue of "required_accum_size"

[ymn-lee]

First of all, thank you for providing such a helpful resource.

While simulating with Llama3-8b, I noticed a potential issue with the required_accum_size formula in Mapping.cc
The current code is
(inner_I * inner_J) * _config.precision
However, since the result of a 16-bit x 16-bit matrix multiplication should be stored as 32-bit,
I believe the correct formula should be
(inner_I * inner_J) * _config.precision * 2

Additionally, when defining max_acc_rows as
(_config.core_config[key.target_core].accum_spad_size KB) / (dim * 4 * 2)
the denominator considers core_height, precision, and double_buffer, which indicates that
the stored data is 32-bit

[HamHyungkyu]

Thanks for pointing this out!

You’re right — in most cases (like TPUs or Tensor Cores), accumulation is done in 32-bit for 16-bit MAC operations.
That said, some edge devices also use 16-bit accumulation to save power.

I’ll update the code so it can handle both cases.
Thanks again for bringing this up!