feat(vscode): add VS Code extension for remote delta uploads

[mikahoy045]

• Add context-engine-uploader extension with auto-sync and watch mode capabilities
• Include standalone upload client bundled within extension for portability
• Add nginx reverse proxy configuration for RMCP services and upload endpoints
• Provide extension documentation covering build, packaging, and configuration
• Support auto-detection of workspace folder as default target path
• Add status bar integration with indexing spinner an

[voarsh2]

I will review and provide some feedback where appropriate when I have time @mikahoy045 - I have only quickly glance at the large vscode folder diff - this extension, assumes you're always running a project, with the Context Engine repo in workdir? Or is it self contained? (Since, the standalone remote upload client is 100% self contained, so that script can run anywhere to upload to the upload service (which would plop files into the watcher service's watch location for processing....

[mikahoy045]

we can say yes, it is self contained, so you can install it basically on any pc (I tested with windsurf). You dont need to clone the Context Engine repository, it would run as the command you explained to me. The only drawback is, you need to have python installed in the pc where you installed the vscode extension, just the python because the dependency already 100% self contained. You can choose to auto init + watcher at start up, or manual. it would read the folder path automatically (tested on mac).

[voarsh2]

@mikahoy045 immediate things that standout:

1. Remove python_libs/ entirely - Extension should assume user has Python installed - pip install
2. Delete duplicated standalone_upload_client.py - Reference the upstream one instead
3. Simplify extension architecture - Make it a thin wrapper that calls existing scripts

Proper Architecture:

// Instead of bundling everything, just:
const { spawn } = require('child_process');
const path = require('path');

// Call the actual upstream script:
spawn('python3', [path.join(upstreamRepo, 'scripts/standalone_upload_client.py'), ...args]);

---

CI/CD can build/bundle the required scripts, instead of duplicating upstream.

Plan of actions:

Current Behavior:

• Extension expects standalone_upload_client.py in its own directory (extensionRoot)
• Falls back to user-configurable scriptWorkingDirectory
• Includes the script + all Python deps (1.6MB) in the extension package
• Assumes user has Python installed but bundles all dependencies anyway

Recommended Refactoring Plan

Option 1: External Dependency Model (Cleanest)

  // package.json
  {
    "dependencies": {},
    "contributes": {
      "configuration": {
        "properties": {
          "contextEngineUploader.upstreamRepoPath": {
            "type": "string",
            "description": "Path to context-engine repository containing scripts/"
          },
          "contextEngineUploader.pythonPath": {
            "type": "string",
            "default": "python3"
          }
        }
      }
    }
  }

Extension Logic:
// Remove bundled script and python_libs entirely
// Reference upstream scripts directly:
const scriptPath = path.join(upstreamRepoPath, 'scripts', 'standalone_upload_client.py');

Option 2: CI/CD Bundle Model (Robust)

  Build Pipeline (GitHub Actions):
  # .github/workflows/build-extension.yml
  - name: Bundle Upstream Scripts
    run: |
      # Clean clone of upstream scripts only
      git clone --depth 1 --filter=blob:none --sparse ${{ upstream_repo }}
      cd context-engine
      git sparse-checkout set scripts/

      # Copy only the needed script (no python_libs!)
      cp scripts/standalone_upload_client.py extension/

      # Package extension
      vsce package

Recommended Architecture

Cleanup (Immediate)

1. Delete entire python_libs/ directory (-1.6MB)
2. Remove bundled standalone_upload_client.py
3. Add configuration for upstream repo path

Phase 2: Dependency Resolution

  // Smart dependency resolution
  function resolveScriptPath() {
    const config = vscode.workspace.getConfiguration('contextEngineUploader');

    // 1. Check for user-configured upstream repo
    const upstreamPath = config.get('upstreamRepoPath');
    if (upstreamPath) {
      return path.join(upstreamPath, 'scripts', 'standalone_upload_client.py');
    }

    // 2. Check if we're in a context-engine workspace
    const workspaceFolders = vscode.workspace.workspaceFolders;
    if (workspaceFolders) {
      for (const folder of workspaceFolders) {
        const candidatePath = path.join(folder.uri.fsPath, 'scripts', 'standalone_upload_client.py');
        if (fs.existsSync(candidatePath)) {
          return candidatePath;
        }
      }
    }

    // 3. Fall back to bundled version (CI/CD built)
    return path.join(extensionRoot, 'standalone_upload_client.py');
  }

Phase 3: CI/CD Pipeline

  # Build and release automation
  name: Build Extension
  on:
    push:
      tags: ['v*']
  jobs:
    build:
      steps:
        - name: Bundle latest upstream scripts
          run: |
            # Fetch only the script, no dependencies
            curl -o standalone_upload_client.py \
              https://raw.githubusercontent.com/context-engine/main/scripts/standalone_upload_client.py

        - name: Package extension
          run: vsce package --no-dependencies

TL;DR

vscode-extension/
├── context-engine-uploader/
│ ├── src/
│ │ ├── extension.js # Clean extension logic
│ │ ├── package.json # Extension manifest
│ │ └── README.md # Updated docs
│ ├── .github/
│ │ └── workflows/
│ │ └── build.yml # CI/CD pipeline
│ └── dist/ # Generated .vsix files

CI/CD Pipeline (.github/workflows/build.yml):

  name: Build Extension
  on:
    push:
      tags: ['v*']
    pull_request:
      paths: ['vscode-extension/**']

  jobs:
    build:
      runs-on: ubuntu-latest
      steps:
        - uses: actions/checkout@v4

        - name: Setup Node.js
          uses: actions/setup-node@v4
          with:
            node-version: '18'

        - name: Bundle upstream scripts
          run: |
            mkdir -p context-engine-uploader/src

            # Fetch latest standalone_upload_client.py
            curl -o context-engine-uploader/src/standalone_upload_client.py \
              https://raw.githubusercontent.com/context-engine/main/scripts/standalone_upload_client.py

        - name: Install vsce
          run: npm install -g @vscode/vsce

        - name: Package extension
          run: |
            cd context-engine-uploader
            vsce package

        - name: Upload artifact
          uses: actions/upload-artifact@v4
          with:
            name: extension-vsix
            path: '*.vsix'

Keep source repo focused, uncluttered, instruct user to - pip install requests urllib3 certifi charset_normalizer - CI/CD can make the visx package to import into vscode forks

Do you agree with the ideas to make this cleaner/maintainable? And if so, can you do it? Or should I help?

[m1rl0k]

See if you can I am at capacity the next few weeks, but your changes are merged! Thanks for this awesome feature!!

[mikahoy045]

Hi @voarsh2 , Sorry I was just check on this. That's a great cleaner approach, but I have some question for this snippets.

// Instead of bundling everything, just:
const { spawn } = require('child_process');
const path = require('path');

// Call the actual upstream script:
spawn('python3', [path.join(upstreamRepo, 'scripts/standalone_upload_client.py'), ...args]);

Wouldn’t that code cause the file to be excluded when I build the extension? Please correct me if I’m wrong, but as far as I know, the VS Code extension build process only packages the files inside the extension folder. So on another PC, one that hasn’t cloned this repository, the extension would fail because it won’t be able to find standalone_upload_client.py in the script folder.

It’s the same idea as bundling Python libraries. You don’t want users to install Python or run any pip install commands (sorry, in previous post I said that a person must install python; which it's not necessary). I even have a friend who’s a full-stack JavaScript developer and refuses to deal with anything related to Python. So the plugin should be completely self-contained and work for a broader range of users without requiring them to install anything.

what do you think ? @voarsh2 @m1rl0k

[voarsh2]

Wouldn’t that code cause the file to be excluded when I build the extension? Please correct me if I’m wrong, but as far as I know, the VS Code extension build process only packages the files inside the extension folder. So on another PC, one that hasn’t cloned this repository, the extension would fail because it won’t be able to find standalone_upload_client.py in the script folder.

It’s the same idea as bundling Python libraries. You don’t want users to install Python or run any pip install commands (sorry, in previous post I said that a person must install python; which it's not necessary). I even have a friend who’s a full-stack JavaScript developer and refuses to deal with anything related to Python. So the plugin should be completely self-contained and work for a broader range of users without requiring them to install anything.

Hi @mikahoy045,

You're absolutely correct about the VSCode packaging constraints - I should have clarified the CI/CD approach more clearly. The code snippet I showed would indeed fail in a distributed extension since external files aren't accessible.

To clarify the architecture: the standalone script remains in the upstream repository, and the CI/CD build process fetches and bundles it during package creation. The VSCode extension source folder remains clean.

Here's the corrected approach:

Build Process (.github/workflows/build.yml):

  - name: Bundle upstream scripts
    run: |
      # Copy from same repository, no external fetch needed
      cp scripts/standalone_upload_client.py vscode-extension/context-engine-uploader/

      # Package extension
      cd vscode-extension/context-engine-uploader
      vsce package

Extension Logic:
// Works because script is bundled during CI/CD
const scriptPath = path.join(extensionRoot, 'standalone_upload_client.py');
spawn('python3', [scriptPath, ...args]);

Key Benefits:

• Clean source repository (no duplicated files)
• Self-contained distribution (no external dependencies at runtime)
• Always bundles latest upstream script version
• Eliminates the 1.6MB python_libs bloat

The only change needed in your current approach is removing the bundled Python libraries, since users requiring Python can install basic HTTP dependencies via pip if needed.

This maintains the self-contained nature while fixing the architectural bloat. Does this refined approach address your concerns about distribution and packaging?

[mikahoy045]

Yes, I agree that this should be built by CI/CD (GitHub Actions). Your idea is: during the build pipeline, we copy the required Python files into the extension directory and produce the final packaged extension there. That way we always ship the latest dependencies. Thanks for pointing this out, @voarsh2

Key Benefits:

• Clean source repository (no duplicated files) (Ok)
• Self-contained distribution (no external dependencies at runtime) (Ok)
• Always bundles latest upstream script version (Ok)

I can implement this.

Regarding this part :

• Eliminates the 1.6MB python_libs bloat

The only change needed in your current approach is removing the bundled Python libraries, since users requiring Python can install basic HTTP dependencies via pip if needed.

My concern is: wouldn’t this require users to have Python installed on their OS? For many general users, this isn’t acceptable. Some hardcore JS developers don’t want Python at all, and users with Miniconda often don’t want anything altering their base environment.

My vision for the extension is that users should be able to download it from the VS Code Marketplace and use it immediately, without needing to install or configure Python manually.

If the goals only to make the maintained code clean, I can do some work around by publishing the python_libs in another public repository so that it can be pull by the ci/cd before build.

  - name: Bundle upstream scripts
    run: |
      # Copy from same repository, no external fetch needed
      cp scripts/standalone_upload_client.py vscode-extension/context-engine-uploader/
     
      cd vscode-extension/context-engine-uploader

      # copy python libs before build
      git clone https://github.com/myrepo/vscex-python-lib.git

      # Build
      vsce package

What do you think about this approach?
If you have any workaround or alternative method that still avoids requiring users to install Python while keeping the extension lightweight, I’d really appreciate your input.

[voarsh2]

I'm doing a cleanup now and will submit a PR. 90% done - just tweaking some issues where I found it wasn't working on Windows (I normally use Linux)

[voarsh2]

Working good on Windows now too:

[voarsh2]

See #24 for the proposed cleanup - I opted to keep build to .sh/.bat/Dockerfile without CI/CD at this time - allows build and publish - can do GitHub workflows or other CI/CD (I use Bamboo personally) in another follow up.