Skip to content

Batch migration

mihir jha edited this page Mar 15, 2024 · 7 revisions
  • The CML Utility is primarily designed to facilitate the migration of individual projects. However, there is a wrapper script available that enables batch migration of multiple projects. Two python scripts are available, one for export and another for import.
  • Batch migration script reads the list of project names from export-config.ini/import-config.ini files. Each section defined here corresponds to a specific project, with the section name corresponding to the project name. You can include project-specific configurations within each respective section, while configurations shared across multiple projects can be placed inside the "default" section.
  • BatchSize variable provided inside the script controls the number of projects that can be exported/imported simultaneously. To prevent system errors like running out of memory, it is essential to select an appropriate batch size. Each export/import operation of a project generates a distinct session on the workspace, utilizing 1 CPU and 0.5 GB of memory. Therefore, the batch size should be determined considering the available resources on both the source and target workspaces.
  • Before initiating the batch migration, ensure that enough disk space is available on the host machine for downloading all or a batch of projects.
  • In case of failure during batch migration, although the script can be rerun, however to speed up the execution of the batch it is recommended to delete all the project-names already exported or imported from the configuration file.
  • Logs for each project are collected inside the individual project directory.

Note: Batch migration scripts are provided for reference, it is not a feature of the utility. Feel free to modify as per your requirement. Batch migration was certified with 50 projects with a batch size of 10 due to resource constraint

Clone this wiki locally