Skip to content

修复 RoPE 多频率实验中角度变化的描述 (#8) #17

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
joyehuang merged 6 commits into from
Feb 16, 2026
Merged

修复 RoPE 多频率实验中角度变化的描述 (#8) #17

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
633eb38
修复 RoPE 多频率实验中角度变化的描述
joyehuang Feb 16, 2026
7b99c9d
根据审核意见改进横向观察的描述
joyehuang Feb 16, 2026
42191e9
修正数值错误并提高描述精确度
joyehuang Feb 16, 2026
25a6830
修正数值错误并增加取模说明（最终版）
joyehuang Feb 16, 2026
f0e72d9
修复视觉对齐问题并增加结尾空行（最终版）
joyehuang Feb 16, 2026
70a08e4
修正频率比描述的不准确性
joyehuang Feb 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 19 additions & 4 deletions modules/01-foundation/02-position-encoding/experiments/exp2_multi_frequency.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,10 +64,25 @@ def demonstrate_multi_frequency():
print(f"{degree:>12.1f}°".ljust(15), end="")
print()

print("\n💡 观察:")
print(" - 维度0（高频）：位置10就转了100度，变化快")
print(" - 维度3（低频）：位置1000才转了0.1度，变化慢")
print(" - 不同维度组合 → 唯一标识每个位置！")
print("\n💡 观察（理解这个表格的两个维度）:")
print(" 【注意】表格中的角度已对360°取模，所以都在0-360°范围内")
print()
print(" 【纵向看 - 沿 token 维度】：")
print(" - 维度0（高频）：随位置增加，角度变化快")
print(" 例如：位置10时累计旋转约573度（表中显示213°是取模后的结果）")
print(" - 维度3（低频）：随位置增加，角度变化相对较慢")
print(" 例如：转一圈需要约23个token（维度0只需6.3个，约4倍差距）")
print()
print(" 【横向看 - 沿嵌入维度】：")
print(" - 从维度0到维度3：频率递减，转动速度逐渐变慢")
print(" 例如：观察同一位置（表格同一行），从维度0到维度3，")
print(" 频率递减导致累计旋转圈数也递减")
print(" 参考上面的频率表：")
print(" - 维度0每6.3个token转一圈")
print(" - 维度3约每23个token转一圈（约4倍慢）")
print()
print(" 【结论】：不同维度组合 → 唯一标识每个位置！")
print()


def demonstrate_absolute_vs_relative():
Expand Down