Streamin Data Sources

[tarikaltuncu]

Hi, is D-ETM good for streamin data sources? How do you suggest training in mini-batches of data as it accumulates?

I also have another related question. I understand that DETM can adjust the model for concept drifts by shifting the mean and varience of topics by gaussian distribution. What if there is a particular topic that exists only (and heavily) in only one (or a few) particular time interval. Would DETM be able to detect it? Or is there an assumption that all topics exist at all time intervals?

[Emekaborisama]

No, from my experience, all the topics don't exist at all time.

Yes, there will surely be a topic that occurs only one in a particular time interval.

[Emekaborisama]

When you say streaming data source, do you mean realtime data from google sheet or SQL?

[mona-timmermann]

You might want to have a look at the paper on Dynamic Mixture Models by Wei et al. (2007) and similar papers on topic modeling for streaming data.