Rework attachment handling during database migrations

[labkey-adam]

Rationale

Repackage API classes into org.labkey.api.migration

The Documents table lives in core but is effectively owned and managed by many different modules. The previous approach to migrating this table attempted to copy all attachments at once, which accommodated container filtering and (barely) the global attachment approach of Labbook, but not domain row and other complex filtering. As a result, we copied far too many rows that shouldn't have ended up in the cloned database and wasted hours of copying time.

This new approach makes each schema handler responsible for copying the documents it manages, immediately after its tables have been populated. The schema handlers can usually delegate all the work to the existing AttachmentType implementations, which already know how to retrieve parent EntityIds for each type. I've adjusted AttachmentType slightly to make it more flexible (supporting the original use case where we selected EntityIds from the documents table and the new use case where we select those EntityIds from the populated tables instead).

Also, TempTableInClauseGenerator was hard-coded to create temp tables in the LabKey data source temp schema, which of course fails when using a large IN clause against an external data source. Teach TempTableInClauseGenerator how to use a different temp schema and introduce a SqlDialect.appendInClauseSql() variant that takes a customized InClauseGenerator for this purpose. The attachment copying code collects potentially large sets of EntityIds and RowIds from the target tables, then uses this mechanism to select the appropriate rows from the source database's core.Documents table.

[labkey-jeckels]

There's a fair amount of edits in codepaths beyond migration. Hopefully automated tests will give good coverage without needing separate manual testing?

[labkey-adam]

There's a fair amount of edits in codepaths beyond migration. Hopefully automated tests will give good coverage without needing separate manual testing?

The changes (outside migration) are primarily adjusting AttachementTypes, right? These aren't really used, other than one code path (that I've tested multiple times during this development). I doubt there's any automated testing of the admin page that uses them. My leveraging AttachmentType here means they've gotten more use than ever and I've fixed several issues that no one noticed.