-
Notifications
You must be signed in to change notification settings - Fork 64
Description
As reported at https://lore.kernel.org/lkml/000000000000c9af510621b0a773@google.com/T/, KMSAN sometimes smashes the kernel stack.
Previously we used to increase stack sizes in KMSAN builds: 060de96, but later it was deemed unnecessary, because the stack usage was not that big as compared to KASAN.
Trying to assess the impact of KMSAN on stack frame sizes, I took two kernel images built with and without CONFIG_KMSAN=y containing roughly 300K functions, and grepped them for sub.*rsp, assuming that the value subtracted from $rsp at function prologue is the stack frame size.
I filtered out some corner cases (functions with duplicate names, functions that only had sub.*rsp in one of the two builds, functions with more than one sub.*rsp instruction) and ended up with a list of 45415 functions with their stack frame sizes in KMSAN and non-KMSAN builds: bloat-kmsan-nokmsan.txt
aa_af_perm 0x100 0xd0
aa_alloc_profile 0x78 0x10
aa_audit_file 0xd0 0x98
aa_audit_perm_mask 0x90 0x70
From this list it was possible to calculate the relative bloat: bloat-percentage.txt
(first column is the line number in the sorted file)
1 2650.00% dsp_bf_encrypt 0x1b8 0x10
2 2550.00% camellia_setup_tail 0x1a8 0x10
3 2000.00% writecache_status 0x150 0x10
4 2000.00% __blkcg_rstat_flush 0x150 0x10
5 1750.00% trace_raw_output_btrfs__writepage 0x128 0x10
6 1600.00% kcm_seq_show 0x110 0x10
7 1600.00% ceph_read_dir 0x110 0x10
...
32745 102.38% btrfs_log_inode 0x2a8 0x150
32746 102.13% btrfs_fiemap 0x2f8 0x178
32747 100.00% zswap_decompress 0xc0 0x60
32748 100.00% ZSTD_freeDDict 0x30 0x18
32749 100.00% ZSTD_estimateCDictSize 0x40 0x20
...
34579 100.00% aa_profile_label_perm 0xa0 0x50
34580 100.00% aa_perm_mask_to_str 0x30 0x18
34581 98.70% bch2_fs_ioctl 0x4c8 0x268
34582 97.37% bch2_gc_gens 0x258 0x130
...
44672 1.92% nldev_set_sys_set_doit 0x350 0x340
44673 1.64% ext4_normal_submit_inode_data_buffers 0x1f0 0x1e8
44674 0.00% ZSTD_flushStream 0x18 0x18
44675 0.00% ZSTD_estimateDStreamSize_fromFrame 0x28 0x28
44676 0.00% zstd_cstream_workspace_bound 0x20 0x20
...
45387 0.00% acpi_dev_for_each_child 0x10 0x10
45388 0.00% aa_info_message 0x88 0x88
45389 -6.67% mptcp_crypto_hmac_sha 0xe0 0xf0
45390 -11.11% set_track_prepare 0x80 0x90
45391 -12.50% ethtool_get_ringparam 0x38 0x40
...
45410 -50.00% __x64_sys_utime32 0x10 0x20
45411 -50.00% __x64_sys_utime 0x10 0x20
45412 -50.00% intel_irq_remapping_deactivate 0x10 0x20
45413 -50.00% __ia32_sys_utime32 0x10 0x20
45414 -50.00% __ia32_sys_utime 0x10 0x20
45415 -62.50% __populate_section_memmap 0x18 0x40
So 44673 out of 45415 interesting functions (98.4%) had their stack frames increased, and for 34580 (76.1%) their stack usage increased by >= 100%.
This suggests we should probably double the stacks in KMSAN builds and see if that helps.