Error in cpp_linear(), std:bad_alloc

[amacanovic]

Hello Matthew,

First, thank you for your work on this package and the related paper! It has been very helpful in guiding my thinking about text pre-processing.

I am running into the following issue when running preText(), at the step of creating dfm 1 of 64 :
Generating document distances... Currently working on dfm 1 of 64 Error in cpp_linear(mt1 = x, mt2 = y, method = match(method, c("cosine", : std::bad_alloc

Googling for this issue hasn't brought me far. Can you help me understand what is going wrong?

Thank you!

[amacanovic]

Hello, an update:
I thought this might've been due to a lack of memory, so I tried with a smaller set, now running into the following at the same step:

Error in stats::cmdscale(distances, k = dimensions) : NA values not allowed in 'd'

[busbyethan]

I encountered the same error, and have also been unable to resolve the error. Were you ever able to figure it out?

[amacanovic]

Hi,

No, I haven't manage to get around it.
Will update here if I do eventually.

[laurebokobza]

Hi,

I encounter the same error:
Error in stats::cmdscale(distances, k = dimensions) : NA values not allowed in 'd' when working on a subset corpus of 1,000 documents, and did not manage to find a way around it. The error message occurs when working on dfm 5/64 (I am not using n-grams). Any help appreciated!

[yjpimentel]

Hi, I had the same error with a similar problem while running textstat_dist. It turns out the corpus was far too large for the command and I solved it by trimming by DFM using dfm_trim(dfm, min_termfreq = x). Increasing the minimum term frequency to 1000 from 100 sorted out the error.

Hope that helps!