RuntimeError: mat1 and mat2 shapes cannot be multiplied (128x5376 and 5248x6000)

[traddo]

(timedart) steven@ubuntu2404:~/work/TimeDART$ bash scripts/pretrain/eeg.sh 
Args in experiment:
Namespace(task_name='pretrain', downstream_task='classification', is_training=1, model_id='EEG', model='TimeDART', llm_path='Qwen/Qwen2.5-0.5B', backbone='Qwen2.5-0.5B', data='EEG', root_path='datasets/eeg_no_big/', data_path='ETTh1.csv', features='M', target='OT', freq='h', checkpoints='./outputs/checkpoints/', pretrain_checkpoints='./outputs/pretrain_checkpoints/', transfer_checkpoints='ckpt_best.pth', load_checkpoints=None, select_channels=1, use_norm=1, accumulation_steps=4, seq_len=336, input_len=3000, label_len=0, pred_len=96, test_pred_len=96, seasonal_patterns='Monthly', top_k=5, num_kernels=3, enc_in=2, dec_in=2, c_out=2, d_model=128, n_heads=16, e_layers=2, d_layers=1, d_ff=256, moving_avg=25, factor=1, distil=True, dropout=0.2, fc_dropout=0, head_dropout=0.2, embed='timeF', activation='gelu', output_attention=False, individual=0, pct_start=0.3, patch_len=75, stride=75, num_workers=5, itr=1, train_epochs=50, batch_size=128, patience=3, learning_rate=0.001, des='test', loss='MSE', lradj='decay', use_amp=False, use_gpu=True, gpu=0, use_multi_gpu=False, devices='0', time_steps=1000, scheduler='cosine', lr_decay=0.95, mask_ratio=1.0, num_classes=8, lm=3, positive_nums=3, rbtp=1, temperature=0.2, masked_rule='geometric', mask_rate=0.5)
Use GPU: cuda:0
number of model params 32109552
>>>>>>>start pre_training : pretrain_TimeDART_EEG_M_il3000_ll0_pl96_dm128_df256_nh16_el2_dl1_fc1_dp0.2_hdp0.2_ep50_bs128_lr0.001_ts1000_sccosine>>>>>>>>>>>>>>>>>>>>>>>>>>
8
torch.Size([3000, 2]) torch.Size([])
train 12787
8
torch.Size([3000, 2]) torch.Size([])
val 1421
Current learning rate: 0.0010000
Using MSELoss
Traceback (most recent call last):
  File "/home/steven/work/TimeDART/run.py", line 267, in <module>
    exp.pretrain()
  File "/home/steven/work/TimeDART/exp/exp_timedart.py", line 103, in pretrain
    train_loss = self.pretrain_one_epoch(
                 ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/steven/work/TimeDART/exp/exp_timedart.py", line 179, in pretrain_one_epoch
    pred_x = self.model(batch_x)
             ^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/timedart/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/timedart/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/steven/work/TimeDART/models/TimeDART.py", line 420, in forward
    return self.pretrain(batch_x)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/steven/work/TimeDART/models/TimeDART.py", line 385, in pretrain
    predict_x = self.projection(predict_x)  # [batch_size, input_len, num_features]
                ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/timedart/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/timedart/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/steven/work/TimeDART/layers/TimeDART_EncDec.py", line 394, in forward
    x = self.forecast_head(x)  # [batch_size, pred_len * num_features]
        ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/timedart/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/timedart/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/envs/timedart/lib/python3.11/site-packages/torch/nn/modules/linear.py", line 116, in forward
    return F.linear(input, self.weight, self.bias)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: mat1 and mat2 shapes cannot be multiplied (128x5376 and 5248x6000)

[Melmaphother]

Dear traddo:
Thank you for your interest in our work~

Have you modified the script and what is your running script?
128x5376 seems [batch_size, seq_len * d_model] = [128, 41 * 128]
5248x6000 seems [seq_len*d_model, input_len * n_channels] = [42 * 128, 3000 * 2]
and seq_len = input_len/patch_len = 3000/75 = 40, but it seems that you get 41 and 42.

[traddo]

python -u run.py \
    --task_name pretrain \
    --downstream_task classification \
    --root_path datasets/eeg_no_big/ \
    --model_id EEG \
    --model TimeDART \
    --data EEG \
    --e_layers 2 \
    --d_layers 1 \
    --input_len 3000 \
    --enc_in 2 \
    --dec_in 2 \
    --c_out 2 \
    --num_classes 8 \
    --n_heads 16 \
    --d_model 128 \
    --d_ff 256 \
    --patch_len 75 \
    --stride 75 \
    --head_dropout 0.2 \
    --dropout 0.2 \
    --time_steps 1000 \
    --scheduler cosine \
    --lr_decay 0.95 \
    --learning_rate 0.001 \
    --batch_size 128 \
    --train_epochs 50 \
    --gpu 0

Datasets download from Google Drive.

[Korol943]

Dear traddo: Thank you for your interest in our work~

Have you modified the script and what is your running script? 128x5376 seems [batch_size, seq_len * d_model] = [128, 41 * 128] 5248x6000 seems [seq_len*d_model, input_len * n_channels] = [42 * 128, 3000 * 2] and seq_len = input_len/patch_len = 3000/75 = 40, but it seems that you get 41 and 42.

Thank you for your detailed analysis! Based on your explanation, it seems there is an inconsistency in how seq_len is calculated within the model's code (one part results in 41, another in 42). Could you please advise on how to fix the code, or is there a specific parameter in the eeg.sh script that I should change to resolve this mismatch?I also encountered the same error, and I was using the 'egg_no_big' data from Google Drive.