renamed paths after renaming VS Code in sidenav and cleaned up unused…

docs/best-practices/datacoves/folder-structure.md

@@ -70,4 +70,4 @@ The `visualization` folder is used to place configs related to superset or other
 The `visualization/streamlit` folder is used for Streamlit apps. This folder is only needed if using Streamlit.
 
 ### .vscode/settings.json
-The `.vscode/settings.json` folder is used for customized settings in order to override the default workspace settings. This file can contain secrets so be sure to add it to the `.gitignore` to avoid version control. See our [How to Override default VS sCode settings](how-tos/vscode/override.md) for more info
+The `.vscode/settings.json` folder is used for customized settings in order to override the default workspace settings. This file can contain secrets so be sure to add it to the `.gitignore` to avoid version control. See our [How to Override default VS sCode settings](how-tos/vs-code/override.md) for more info

docs/getting-started/developer/transform-tab.mdx

@@ -6,7 +6,7 @@ sidebar_position: 4
 
 ## This video guide covers:
 
-- [User Settings Configurations](how-tos/vscode/initial.md)
+- [User Settings Configurations](how-tos/vs-code/initial.mdx)
 - Terminal setup
 - A high level overview of VS Code editor
 - How to reset your environment

docs/getting-started/developer/working-with-dbt-datacoves/lineage-view.md

@@ -39,6 +39,6 @@ You can also look up other dbt models and change the parent and child nodes disp
 
 ## Configuration
 
-Lineage Panel has 3 configurable settings. As with any VSCode setting, these can be [overridden](/docs/how-tos/vscode/override) in the `settings.json` file located in `workspace/.vscode/settings.json`:
+Lineage Panel has 3 configurable settings. As with any VSCode setting, these can be [overridden](/docs/how-tos/vs-code/override) in the `settings.json` file located in `workspace/.vscode/settings.json`:
 
 - Default Expansion (`dbt.lineage.defaultExpansion`: number): How many nodes should expand by default for the currently-opened model.

docs/how-tos/vscode/_category_.yaml → docs/how-tos/vs-code/_category_.yaml

@@ -1,4 +1,4 @@
-label: vscode
+label: VS Code
 position: 50
 link:
   type: generated-index

docs/how-tos/vscode/override.md → docs/how-tos/vs-code/override.md

@@ -4,7 +4,7 @@ sidebar_position: 94
 ---
 # How to override default VS Code settings
 
-Once you have [set up your Visual Studio Code](./initial.md) environment, you have the possibility to override certain settings we provide by default. This is done in your Workspace settings.
+Once you have [set up your Visual Studio Code](./initial.mdx) environment, you have the possibility to override certain settings we provide by default. This is done in your Workspace settings.
 
 ## Create your workspace settings file
 

migrated-files/_sidebar.md

@@ -0,0 +1,149 @@
+- [Home](/)
+- **Getting Started**
+- [Administrator](/getting-started/Admin/)
+  - [Account Pre-reqs](/getting-started/Admin/create-account.md)
+  - [Configure Airflow](/getting-started/Admin/configure-airflow.md)
+  - [Configure Git Repository](/getting-started/Admin/configure-repository.md)
+  - [Configure Git Repository Using dbt-coves](/getting-started/Admin/configure-repository-using-dbt-coves.md)
+  - [Creating Airflow Dags](/getting-started/Admin/creating-airflow-dags.md)
+  - [User Management](/getting-started/Admin/user-management.md)
+- [Developer](/getting-started/developer/)
+  - [Snowflake Extension](/getting-started/developer/snowflake-extension.md)
+  - [Transform Tab](/getting-started/developer/transform-tab.md)
+  - [Working with dbt in Datacoves](/getting-started/developer/working-with-dbt-datacoves.md)
+  - [Using Git](/getting-started/developer/using-git.md)
+- **How to**
+- [How to](/how-tos/)
+  - [Airflow](/how-tos/airflow/)
+    - [Airflow - Initial setup](/how-tos/airflow/initial-setup.md)
+    - [Airflow - Accessing the Airflow API](/how-tos/airflow/use-airflow-api.md)
+    - [Airflow - Sync Internal Airflow database](/how-tos/airflow/sync-database.md)
+    - [Airflow - Trigger a DAG using Datasets](how-tos/airflow/api-triggered-dag.md)
+    - [DAGs - Add Dag Documentation](/how-tos/airflow/create-dag-level-docs.md)
+    - [DAGs - Calling External Python Scripts](/how-tos/airflow/external-python-dag.md)
+    - [DAGs - Dynamically Set Schedule](/how-tos/airflow/dynamically-set-schedule.md)
+    - [DAGs - Generate DAGs from yml](/how-tos/airflow/generate-dags-from-yml.md)
+    - [DAGs - Get Current Git Branch Name from a DAG Task](/how-tos/airflow/get-current-branch-name.md)
+    - [DAGS - Load from S3 to Snowflake](/how-tos/airflow/s3-to-snowflake.md)
+    - [DAGs - Run ADF Pipelines](/how-tos/airflow/run-adf-pipeline.md)
+    - [DAGs - Run Airbyte sync jobs](/how-tos/airflow/run-airbyte-sync-jobs.md)
+    - [DAGs - Run dbt](/how-tos/airflow/run-dbt.md)
+    - [DAGs - Retry dbt jobs](/how-tos/airflow/retry-dbt-tasks.md)
+    - [DAGs - Run Databricks Notebooks](/how-tos/airflow/run-databricks-notebook.md)
+    - [DAGs - Run Fivetran sync jobs](/how-tos/airflow/run-fivetran-sync-jobs.md)
+    - [DAGs - Test DAGs](/how-tos/airflow/test-dags.md)
+    - [DAGs - Using Variables and Connections](/how-tos/airflow/use-variables-and-connections.md)
+    - [Notifications - Send Emails](/how-tos/airflow/send-emails.md)
+    - [Notifications - Send Microsoft Teams notifications](/how-tos/airflow/send-ms-teams-notifications.md)
+    - [Notifications - Send Slack notifications](/how-tos/airflow/send-slack-notifications.md)
+    - [Secrets - AWS Secrets Manager](/how-tos/airflow/use-aws-secrets-manager.md)
+    - [Secrets - Datacoves Secrets Manager](/how-tos/airflow/use-datacoves-secrets-manager.md)
+    - [Worker - Custom Worker Environment](/how-tos/airflow/customize-worker-environment.md)
+    - [Worker - Request Memory and CPU](/how-tos/airflow/request-resources-on-workers.md)
+  - [Datacoves](/how-tos/datacoves/)
+    - [Configure Connection Templates](/how-tos/datacoves/how_to_connection_template.md)
+    - [Configure Datacoves Secret](/how-tos/datacoves/how_to_secrets.md)
+    - [Configure Environments](/how-tos/datacoves/how_to_environments.md)
+    - [Configure Groups](/how-tos/datacoves/how_to_groups.md)
+    - [Configure Integrations](/how-tos/datacoves/how_to_integrations.md)
+    - [Configure Invitations](/how-tos/datacoves/how_to_invitations.md)
+    - [Configure Projects](/how-tos/datacoves/how_to_projects.md)
+      - [Configure AWS Secrets Manager](/how-tos/datacoves/how_to_configure_aws_secrets_manager.md)
+      - [Configure Azure DevOps](/how-tos/datacoves/how_to_configure_azure_DevOps.md)
+        - [Create your EntraID App](/how-tos/datacoves/create_your_entraid_application.md)
+        - [Add EntraID App to DevOps Portal](/how-tos/datacoves/add_entraid_app_as_user.md)
+        - [Gather DevOps Auth Details](/how-tos/datacoves/gather_azure_devops_auth_details.md)
+        - [Authenticate Azure DevOps](/how-tos/datacoves/authenticate_azure_devops.md)
+    - [Configure Service Connections](/how-tos/datacoves/how_to_service_connections.md)
+    - [Manage Users](/how-tos/datacoves/how_to_manage_users.md)
+    - [Update Repository](/getting-started/Admin/configure-repository.md)
+    - [Configure VSCode Environment Variables](/how-tos/datacoves/how_to_environment_variables.md)
+  - [Datahub](/how-tos/datahub/)
+    - [Configure dbt metadata ingestion](/how-tos/datahub/how_to_datahub_dbt.md)
+    - [Configure Snowflake metadata ingestion](/how-tos/datahub/how_to_datahub_snowflake.md)
+    - [Manage datahub using CLI](/how-tos/datahub/how_to_datahub_cli.md)
+  - [DataOps](/how-tos/dataops/)
+    - [Releasing a new feature](/how-tos/dataops/releasing-new-feature)
+  - [dbt](/how-tos/dbt/)
+    - [Advanced Debugging](/how-tos/dbt/advenced-dbt-debug)
+    - [Compilation Errors](/how-tos/dbt/compilation-errors)
+    - [Database Errors](/how-tos/dbt/database-errors)
+    - [Dependency Errors](/how-tos/dbt/dependency-errors)
+    - [Runtime Errors](/how-tos/dbt/runtime-errors)
+  - [Git](/how-tos/git/)
+    - [SSH Keys configuration](/how-tos/git/ssh-keys)
+  - [Metrics & Logs](/how-tos/metrics-and-logs/)
+    - [Git Sync/S3 Failures](/how-tos/metrics-and-logs/view-failed-git-sync.md)
+  - [My Airflow](/how-tos/my_airflow/)
+    - [Migrating from Environment Service Connections](/how-tos/my_airflow/migrating-service-connections.md)
+    - [My Import](/how-tos/my_airflow/my-import.md)
+    - [Use My Airflow](/how-tos/my_airflow/start-my-airflow.md)
+  - [Snowflake](/how-tos/snowflake/)
+    - [Setting up Snowflake Key-Based Auth](/how-tos/snowflake/snowflake-key-based-auth)
+    - [Warehouses, Schemas and Roles](/how-tos/snowflake/warehouses-schemas-roles)
+  - [Superset](/how-tos/superset/)
+    - [Add a Database](/how-tos/superset/how_to_database.md)
+    - [Add a Dataset](/how-tos/superset/how_to_data_set.md)
+  - [VS Code](/how-tos/vscode/)
+    - [Datacoves Copilot](/how-tos/vscode/datacoves-copilot/)
+      - [Config](/how-tos/vscode/datacoves-copilot/llm-config.md)
+      - [Usage](/how-tos/vscode/datacoves-copilot/llm.md)
+    - [Initial Configuration](/how-tos/vscode/initial.md)
+      - [BigQuery](/how-tos/vscode/bigquery_setup.md)
+      - [Databricks](/how-tos/vscode/databricks_setup.md)
+      - [Redshift](/how-tos/vscode/redshift_setup.md)
+      - [Snowflake](/how-tos/vscode/snowflake_setup.md)
+    - [Custom Environment Variables](/how-tos/vscode/environment-variables.md)
+    - [Override VS Code settings](/how-tos/vscode/override.md)
+    - [Reset User Env](/how-tos/vscode/reset-user-env.md)
+    - [Reset Git](how-tos/vscode/reset-git.md)
+- **Best Practices**
+- [Best Practices](/explanation/best-practices/)
+  - [Datacoves](/explanation/best-practices/datacoves/)
+    - [Folder Structure](/explanation/best-practices/datacoves/folder-structure)
+  - [dbt](/explanation/best-practices/dbt/)
+    - [dbt Guidelines](/explanation/best-practices/dbt/dbt-guidelines)
+    - [Object Naming Standards](/explanation/best-practices/dbt/object-naming)
+    - [What are Inlets, Bays, and Coves](/explanation/best-practices/dbt/inlets-bays-coves)
+  - [Git](/explanation/best-practices/git/)
+  - [Snowflake](/explanation/best-practices/snowflake/)
+    - [Security Model](/explanation/best-practices/snowflake/security-model)
+    - [GDPR and Time-Travel](/explanation/best-practices/snowflake/time-travel)
+- **Reference**
+- [Reference](/reference/)
+  - [Administration Menu](reference/admin-menu/)
+    - [Account Settings & Billing](/reference/admin-menu/settings_billing.md)
+    - [Connection Templates](/reference/admin-menu/connection_templates.md)
+    - [Environments](/reference/admin-menu/environments.md)
+    - [Groups](/reference/admin-menu/groups.md)
+    - [Integrations](/reference/admin-menu/integrations.md)
+    - [Invitations](/reference/admin-menu/invitations.md)
+    - [Projects](/reference/admin-menu/projects.md)
+    - [Secrets](/reference/admin-menu/secrets.md)
+    - [Service Connections](/reference/admin-menu/service_connections.md)
+    - [Users](/reference/admin-menu/users.md)
+  - [Airflow](/reference/airflow/)
+    - [Airflow Best Practices](/reference/airflow/airflow-best-practices.md)
+    - [Airflow Config Defaults](/reference/airflow/airflow-config-defaults.md)
+    - [Airflow Variables](/reference/airflow/airflow-variables.md)
+    - [DAG Generators](/reference/airflow/dag-generators.md)
+    - [Datacoves Airflow Decorators](/reference/airflow/datacoves-decorators.md)
+    - [Datacoves CLI Commands](/reference/airflow/datacoves-commands.md)
+    - [Datacoves Environment Service Connection Variables](/reference/airflow/environment-service-connection-vars.md)
+    - [Datacoves Operators](/reference/airflow/datacoves-operator.md)
+  - [Datacoves](/reference/datacoves/)
+    - [Versioning](/reference/datacoves/versioning.md)
+    - [VPC Deployment](/reference/datacoves/vpc-deployment.md)
+  - [Metrics & Logs](/reference/metrics-and-logs/)
+    - [Grafana](/reference/metrics-and-logs/grafana.md)
+  - [Security](/reference/security/)
+  - [VS Code](/reference/vscode/)
+    - [CSVs](/reference/vscode/csv-in-datacoves.md)
+    - [Datacoves Environment Variables](/reference/vscode/datacoves-env-vars.md)
+    - [VS Code Tips](/reference/vscode/tips.md)
+- **Tutorials**
+- [Tutorials](/tutorials/)
+  - [Learning Resources](https://datacoves.com/learning-resources?_gl=1*18cmhau*_ga*MjYwMzYwODE1LjE3NTIyNTAwNDk.*_ga_WFBP8GG4YV*czE3NTYyNDYzMzAkbzUwJGcxJHQxNzU2MjQ2NDg4JGo2MCRsMCRoMA..)
+  - [Educational Data Resources](https://datacoves.com/data-resources#Educational)
+
+<footer id="mb-footer"></footer>

migrated-files/add-frontmatter-titles.js

@@ -0,0 +1,72 @@
+// add-frontmatter-titles.js
+import fs from 'fs';
+import path from 'path';
+
+const sidebarFile = '_sidebar.md';
+const docsFolder = 'docs';
+
+function parseSidebar(filePath) {
+  const content = fs.readFileSync(filePath, 'utf8');
+  const linkRegex = /^\s*-\s*\[(.*?)\]\((.*?)\)/gm;
+  const matches = [...content.matchAll(linkRegex)];
+
+  const docs = matches.map((m, index) => {
+    let [_, title, link] = m;
+    title = title.trim();
+
+    if (/^https?:\/\//.test(link)) return null; // skip external links
+
+    if (link.startsWith('/')) link = link.slice(1);
+
+    const folderReadme = path.join(docsFolder, link, 'README.md');
+    const fileMd = path.join(docsFolder, link + '.md');
+
+    if (fs.existsSync(fileMd)) {
+      return { title, filePath: fileMd, position: index + 1 };
+    } else if (fs.existsSync(folderReadme)) {
+      return { title, filePath: folderReadme, position: index + 1 };
+    } else {
+      console.warn(`⚠️ Skipping link (no file found): ${link}`);
+      return null;
+    }
+  });
+
+  return docs.filter(Boolean);
+}
+
+function addTitleAndPosition(filePath, title, position) {
+  let content = fs.readFileSync(filePath, 'utf8');
+
+  if (content.startsWith('---')) {
+    const endIndex = content.indexOf('---', 3);
+    if (endIndex !== -1) {
+      let fm = content.slice(3, endIndex).trim();
+
+      fm = /^title:/m.test(fm)
+        ? fm.replace(/^title:.*$/m, `title: ${title}`)
+        : `title: ${title}\n${fm}`;
+
+      fm = /sidebar_position:/m.test(fm)
+        ? fm.replace(/sidebar_position:.*$/m, `sidebar_position: ${position}`)
+        : `${fm}\nsidebar_position: ${position}`;
+
+      const newContent = `---\n${fm}\n---\n` + content.slice(endIndex + 3).trimStart();
+      fs.writeFileSync(filePath, newContent, 'utf8');
+      console.log(`✅ Updated ${filePath}`);
+      return;
+    }
+  }
+
+  const newContent = `---\ntitle: ${title}\nsidebar_position: ${position}\n---\n\n${content}`;
+  fs.writeFileSync(filePath, newContent, 'utf8');
+  console.log(`✅ Added front matter to ${filePath}`);
+}
+
+function main() {
+  const docs = parseSidebar(sidebarFile);
+  docs.forEach(({ title, filePath, position }) => {
+    addTitleAndPosition(filePath, title, position);
+  });
+}
+
+main();