It appears that low bitrates such as 24 Kbps doesn't work. If you bump the bitrate up slightly higher to say like 24.1 Kbps it will work but then https://github.com/theori-io/nrsc5 will complain about unexpected block type 0.
I was using the mode MP1 and not sure if this requires use of MP3?
Not sure if I'm doing something wrong but figured I'd mention it.