Full vs. Incremental Loading

[amigotune]

Hi!

It's not clear to me if Square does incremental or full loads. Is it able to do incremental ones? Where can I find more information about this?

Cheers!

[jeffreyiacono]

hey @amigoricardo happy to help answer your question, but wanted to make sure I understand it a bit more: are you asking if you can use this framework to do data processing that continues to build from a given starting point or if you need to do a complete rebuild each time? We've used this to do both and the amount of data and destination structure has really been the deciding factor in picking one set up over the other. Let me know!

[amigotune]

Hi! I want to sync two tables according to a specific query. Thus, I would like to Square to identify, at each run, the minimum amount of UPDATEs, INSERTs and DELETEs in order to sync them.