I'm getting some duplicate posts when analyzing the results from a query like g.search('maga', gabs=1000) (where every single field is duplicated so it looks like the same post). I was hoping to filter out reblogged posts so I thought this could be the right direction, but since every field is the same, maybe not. Is this a mistake?
For example, the output of g.search('maga', gabs=1000) includes 45 duplicate rows with id: 6215292716481084 , interestingly at indices 120, 140, 160, ..., 980, 1000. Not sure why this is happening.
I'm getting some duplicate posts when analyzing the results from a query like
g.search('maga', gabs=1000)(where every single field is duplicated so it looks like the same post). I was hoping to filter out reblogged posts so I thought this could be the right direction, but since every field is the same, maybe not. Is this a mistake?For example, the output of
g.search('maga', gabs=1000)includes 45 duplicate rows withid: 6215292716481084, interestingly at indices 120, 140, 160, ..., 980, 1000. Not sure why this is happening.