Update blog

Author: cmdr2

content/blog/2025-10-27-1761560082.md

@@ -11,13 +11,15 @@ tags:
 A possible intuition for understanding GPU memory hierarchy (and the performance penalty for data transfer between various layers) is to think of it like a manufacturing logistics problem:
 1. CPU (host) to GPU (device) is like travelling overnight between two cities. The CPU city is like the "headquarters", and contains a mega-sized warehouse of parts (think football field sizes), also known as 'Host memory'.
 2. Each GPU is like a different city, containing its own warehouse outside the city, also known as 'Global Memory'. This warehouse stockpiles whatever it needs from the headquarters city (CPU).
-3. Each SM/Core/Tile is a factory located in different areas of the city. Each factory contains a small warehouse (shed) for stockpiling whatever inventory it needs, also known as 'Shared Memory'.
+3. Each SM/Core/Tile is a factory located in different areas of the city. Each factory contains a small warehouse for stockpiling whatever inventory it needs, also known as 'Shared Memory'.
 4. Each warp is a bulk stamping machine inside the factory, producing 32 items in one shot. There's a tray next to each machine, also known as 'Registers'. This tray is used for keeping stuff temporarily for each stamping process.
 
 This analogy can help understand the scale and performance penalty for data transfers.
 
 For e.g. reading constantly from the Global Memory is like driving between the factory and the warehouse outside the city each time (with the traffic of city roads). This is much slower than going to the shed inside the factory (i.e. Shared Memory), and much much slower than just sticking your hand into the tray next to your stamping machine (i.e. Registers). And reading from the Host Memory (CPU) is like taking an overnight trip to another city.
 
+A bit of detail: the warehouse of each factory (i.e. Shared Memory) isn't a single building, but is actually a collection of individual sheds called 'banks' (e.g. 32 sheds/banks). Each shed has its own entry, and an employee stands at the entrance of the shed to service your request.
+
 Therefore the job of running a computation graph (like ONNX) efficiently on GPU(s) is like planning the logistics of a manufacturing company. You've got raw materials in the main warehouse that you need to transfer between cities, and store/process/transfer artifacts across different factories and machines. You need to make sure that:
 - the production process follows the chart laid out in the computation graph
 - every machine in each factory is being utilized optimally