Help required to use new dataset

[anshumanpatil]

In below PR -

#25

I have tried to use new dataset from hf got somewhat success but not completely successful

Completed : -

New dataset added related to fitness

• Changed dataset formatting and created pre training data.

• Used dataset - fitness-question-answers

• Created pre training data with - ibm/granite-4-h-tiny

Tried but unsuccessful : -

Working on

Adjusting hyperparameters.

• Changed MAX_SEQ_LEN and found that 180 works.

• Still working on learning rate and batch size

Any Suggestion/Help is welcome
Thanks in advance

[tekaratzas]

What isn't working?

[anshumanpatil]

What isn't working?

it hallucinates

basically gives weird answers

loss function do not reach below 0 at the end of training

I also tried learning rate and batch size

still working on it