Ever watched a twitch stream and wished you could get more of the chat spam, copypasta, and memes?
This code goes along with my post about language modeling using recurrent neural networks. The post puts a focus on data collection, sanitization, and formatting to move data from the Twitch IRC into a tensorflow-compatible format.
Live demo available on the post if you want to experiment with what the network thinks a typical Twitch user would say.
###How to run it
-
Edit
scrape.jsto supply your own Twitch IRC Token and username. You can get the oauth token from http://twitchapps.com/tmi/. Runnode scrape.jsto scrape messages from Twitch's IRC channels. Alternatively, you can also use the provideddatabase.txtwhich I collected by running the scraper overnight for ~8 hours. Fill in the channels array with a list of streamers you want to scrape from. -
Run
node topwords.jsto format a vocabulary dictionary (words.json) that maps IDs to each word that appeared at least five times. -
Run
python arrays.pyto joindatabase.txtandwords.jsoninto a numpy array representing each sentence as a vector of word IDs. -
Run
python main.pyto train the network using Tensorflow. I left it running for about ~3 hours on a GPU to get the model in the live demo. -
Use
sample.pyto sample messages from the model.server.pycreates a simple web API to retrieve samples.