Secure Trusted Upload Facility (STUF)

Purpose and Intent

The Secure Trusted Upload Facility (STUF) is designed to solve a critical challenge faced by organizations: How to securely receive confidential files from project participants while maintaining the highest standards of security, privacy, and trust.

Core Problem

Organizations frequently need to collect sensitive information from external collaborators, including:

• Intellectual property
• Confidential documents
• Research data
• Compliance materials
• Proprietary code

Traditional file sharing methods often fall short in security, auditability, or ease of use, forcing organizations to choose between security and usability.

The STUF Solution

STUF provides a trusted, secure mechanism for receiving confidential files with:

1. Strong Security by Design

   • Two-factor authentication (email + SMS verification)
   • Encrypted storage with strict access controls
   • Comprehensive audit logging
   • Secure file validation and scanning

2. Trust Through Transparency

   • "Code under glass" approach provides complete visibility
   • All deployed code and configuration is preserved in a private git repository
   • Trust Architects have read access to verify security implementations
   • No hidden processes or black-box operations

3. Flexible Deployment Options

   • Fully managed (Pyx-hosted Zulip + Pyx-hosted STUF)
   • Hybrid (Pyx-hosted Zulip + Self-hosted STUF)
   • Fully self-hosted (Self-hosted Zulip + Self-hosted STUF)
   • Clear migration paths between deployment models

4. Seamless Integration

   • Deep integration with Zulip for notifications and management
   • Pluggable Django app for Trust Architect administration
   • API-based architecture for future integrations

5. Governance-Focused Metadata

   • Configurable metadata collection for uploads
   • IP ownership declarations
   • License condition tracking
   • Customizable fields based on project requirements

Key Differentiators

What makes STUF different from other file upload solutions:

• Earned Trust: Complete transparency into the system's operation
• Governance First: Built for sensitive IP and compliance scenarios
• User-Centric: Simple, intuitive interfaces for all technical levels
• Deployment Freedom: No vendor lock-in with multiple hosting options
• Purpose-Built: Designed specifically for secure, trusted file collection

Architecture

STUF consists of five main components:

1. Single Page Application (SPA) - User-facing interface for file uploads
2. API Service - Backend service handling authentication and file uploads
3. Storage Bucket - Cloud-based storage for files and configuration
4. Pluggable Django App - Management interface for Trust Architects
5. Database - Stores system configuration and user information

Getting Started

For detailed information on deploying and using STUF, please refer to:

• Technical Architecture
• User Guides
• Deployment Documentation
• Implementation Guide
• Operations Guide

Vision and Roadmap

STUF is continuously evolving to meet the needs of organizations requiring secure file uploads. For information on upcoming features and enhancements, see our Vision and Roadmap.

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Local Development with Docker Compose

STUF can be run locally using Docker Compose for development and testing purposes.

Prerequisites

• Docker and Docker Compose installed on your system
• Git for cloning the repository

Setup Instructions

1. Clone the repository:

   git clone https://github.com/pyx-industries/stuf.git
   cd stuf

2. Run the test suite to verify setup:

   make test

   This will automatically create a Python virtual environment and install dependencies. If you want to run tests individually, say for the api service, you can activate the virtualenv in your console manually with source .venv/bin/activate and run tests with pytest api/, for example.

3. Create a .env file from the example and edit as needed:

   cp .env.example .env

   Note: The .env file contains configuration for Keycloak realms, roles, and client IDs. Edit this file to customize your local development environment.

4. Start the Docker Compose environment:

   docker compose up -d

5. Access the components:

   • SPA (Frontend): http://localhost:3000
     • Login with admin/password or other test users from realm-export.json
   • API: http://localhost:8000
     • Health check: http://localhost:8000/api/health
     • API info: http://localhost:8000/api/info
   • Keycloak Admin Console: http://localhost:8080/admin (admin/admin)
   • MinIO:
     • API endpoint: http://localhost:9000
     • Web console: http://localhost:9001 (minioadmin/minioadmin)

Environment Components

• Keycloak: Authentication and authorization server
  • Development-only admin credentials: admin/admin
  • Configured with realm: stuf
  • Clients:
    • stuf-spa: Public client for the SPA
    • stuf-api: Confidential client with service account
  • Roles:
    • admin: Administrator role with full access
    • collection: Collection-specific access roles
  • Test users:
    • admin@example.com / password (admin role)
• MinIO: S3-compatible object storage
• API: FastAPI backend service
• SPA: React frontend application

Development Tools

Pre-commit Hooks

This project uses pre-commit hooks with ruff for code formatting and linting:

1. Install development dependencies:

   pip install -r requirements-dev.txt

2. Install pre-commit hooks:

   pre-commit install

The hooks will automatically run ruff formatting and linting on changed files during commits. You can also run them manually:

# Run on all files
pre-commit run --all-files

# Run on staged files only
pre-commit run

Container Publishing

STUF provides built-in container publishing for CI/CD workflows and local development.

CI Publishing (Automatic)

Container images are automatically published to GitHub Container Registry when:

• Tests pass on the master branch
• Images are tagged with both latest and commit SHA

Published images:

• ghcr.io/owner/repo-api:latest / ghcr.io/owner/repo-api:abc1234
• ghcr.io/owner/repo-spa:latest / ghcr.io/owner/repo-spa:abc1234

Local Publishing

For testing or manual publishing:

# Build containers locally
make build-containers

# Publish development images (requires GITHUB_TOKEN)
export GITHUB_TOKEN=your_personal_access_token
make publish-dev

# Publish release images (requires git tag)
git tag v1.0.0
make publish-release

To create a personal access token for local publishing:

1. GitHub → Settings → Developer settings → Personal access tokens
2. Generate token with write:packages and read:packages scopes
3. Copy token and export as GITHUB_TOKEN environment variable

Stopping the Environment

To stop the Docker Compose environment:

docker-compose down

To stop and remove volumes (will delete all data):

docker-compose down -v