I am getting in average 106 sec per page in mac M1 max 64gb.
mps mode result [transformers] Setting pad_token_id to eos_token_id:248044 for open-end generation.
[worker] inference: 106.0s for 1 page(s) (tokens: [1120])
[worker] 1 page(s) in 119015ms
Is it the best I can get ?
I am getting in average 106 sec per page in mac M1 max 64gb.
mps mode result [transformers] Setting pad_token_id to eos_token_id:248044 for open-end generation.
[worker] inference: 106.0s for 1 page(s) (tokens: [1120])
[worker] 1 page(s) in 119015ms
Is it the best I can get ?