Skip to content

Conversation

@kevin-induro
Copy link

Tell us what you do here

  • improving, documenting, or customizing an existing source (please link an issue or describe below)

Short description

The primary motivation is to switch the basic resources (i.e. those created from ALL_OBJECTS with the crm_objects function) from fetching all the Hubspot objects to fetching only those needed based on the incremental load date.

Additional Context

There are still a few different things that can be improved.

  • I noticed that the *_property_history tables have columns for all the properties of the object and null values for all those columns. It should really just have the columns _dlt_id, _dlt_load_id, object_id, property_name, source_id, source_type, timestamp, updated_by_user_id, value
  • I think there's room for improvement on the property history side as well. It'll likely be faster to use the search API to get the recent changes, then pass the IDs from that into the batch API rather than trying to get all of the objects histories every time.
  • I'm not happy with the settings properties. I think things can be generally put together better, and I think I made it worse while trying to follow the previous convention. I also think that there are more settings that can be configurable when declaring the source and refactoring the settings could allow that better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant