Add comprehensive GitHub Copilot instructions for MDS development workflow #3679
+236
−0
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,236 @@ | ||
| # Mines Digital Services (MDS) - GitHub Copilot Instructions | ||
|
|
||
| **CRITICAL: Always follow these instructions first and only search for additional context if information here is incomplete or found to be in error.** | ||
|
|
||
| ## Overview | ||
|
|
||
| MDS is a full-stack mining regulatory application for British Columbia with multiple services: | ||
| - **Frontend**: Core Web (ministry interface), Minespace Web (proponent interface), Common (shared components) | ||
| - **Backend**: Core API (Python Flask), Document Manager, NRIS API, Permits service | ||
| - **Database**: PostgreSQL with Flyway migrations | ||
| - **Infrastructure**: Docker containers orchestrated with Docker Compose and Make | ||
|
|
||
| ## Working Effectively | ||
|
|
||
| ### Prerequisites Validation | ||
| Always run `make valid` first to check your environment. This validates: | ||
| - Node.js v20.16.0 (exact version required) | ||
| - Yarn 3.2.4 | ||
| - Docker and Docker Compose v2 | ||
| - OpenShift CLI access (for production environments) | ||
|
|
||
| ### Initial Setup Commands (Run Once) | ||
| **CRITICAL TIMING**: Never cancel these commands. Set timeouts appropriately: | ||
|
|
||
| 1. **Environment Setup**: `make env` - Creates .env files from templates | ||
| - **Timeout: 5+ minutes** - May prompt for external service access | ||
| - **NEVER CANCEL**: Wait for completion even if appears slow | ||
|
|
||
| 2. **Dependencies**: `yarn install` | ||
|
Collaborator
There was a problem hiding this comment. Just checked here- there's a couple more dependencies in
Not sure if copilot needs to be told that yarn & yarn install are the same command. |
||
| - **Timeout: 10+ minutes** - Downloads all workspace dependencies | ||
| - **NEVER CANCEL**: Large monorepo with multiple services | ||
| - **CRITICAL LIMITATION**: Will fail in sandboxed environments due to artifacts.developer.gov.bc.ca network restrictions | ||
| - **Expected Failure**: "getaddrinfo EAI_AGAIN artifacts.developer.gov.bc.ca" errors are normal in restricted environments | ||
|
|
||
| 3. **Backend Services**: `make be` / `make lite` | ||
| - **Timeout: 15+ minutes** - Builds and starts all Docker containers | ||
| - **NEVER CANCEL**: Database migrations, image building, service startup | ||
| - **CRITICAL LIMITATION**: Will fail in sandboxed environments due to yarn install failures in Docker builds | ||
| - **Expected**: Multiple container builds including PostgreSQL, Redis, Python services (when network access available) | ||
|
|
||
| 4. **Database Seeding**: `make seeddb` | ||
| - **Timeout: 5+ minutes** - Creates test data | ||
| - **NEVER CANCEL**: Generates factory data for development | ||
|
|
||
| ### Development Workflow | ||
|
|
||
| #### Starting Services | ||
| Run these commands in separate terminals: | ||
|
|
||
| 1. **Backend**: `make be` (if not already running) | ||
| 2. **Common Watcher**: `cd services/common && yarn watch` | ||
| 3. **Core Web**: `cd services/core-web && yarn serve` (runs on localhost:3000) | ||
| 4. **Minespace Web**: `cd services/minespace-web && yarn serve` (runs on localhost:3020) | ||
|
|
||
| **Alternative for Codespaces**: `make restart` runs all services as background processes | ||
|
|
||
| #### Stopping Services | ||
| - **Frontend**: Ctrl+C in terminal running yarn commands | ||
| - **Backend**: `make stop` | ||
|
|
||
| ### Build Commands | ||
|
|
||
| #### Full Builds | ||
| - **All Services**: `make all` | ||
| - **Timeout: 60+ minutes** - Builds all Docker containers | ||
| - **NEVER CANCEL**: Complete infrastructure build | ||
|
|
||
| - **Frontend Only**: `cd services/core-web && yarn build` | ||
| - **Timeout: 10+ minutes** - Webpack production build | ||
| - **NEVER CANCEL**: TypeScript compilation, bundling, optimization | ||
|
|
||
| - **Backend Rebuild**: `make be-rebuild` | ||
| - **Timeout: 30+ minutes** - Forces backend container rebuild | ||
| - **NEVER CANCEL**: Python dependencies, application setup | ||
|
|
||
| ### Testing Commands | ||
|
|
||
| #### Frontend Tests | ||
| - **Core Web Unit Tests**: `yarn workspace @mds/core-web run ci-test` | ||
| - **Timeout: 15+ minutes** - Jest, coverage reports | ||
| - **NEVER CANCEL**: Complete test suite with coverage | ||
|
|
||
| - **Minespace Unit Tests**: `yarn workspace @mds/minespace-web run test` | ||
| - **Timeout: 10+ minutes** - Jest test suite | ||
| - **NEVER CANCEL**: Full component and utility testing | ||
|
|
||
| #### Backend Tests | ||
| - **Core API Tests**: `make testbe` | ||
| - **Timeout: 20+ minutes** - Pytest with database integration | ||
| - **NEVER CANCEL**: Database setup, migration tests, API integration | ||
|
|
||
| - **Specific Test Folder**: `make testbe_folder f=folder_name` | ||
| - **Timeout: 5+ minutes** - Focused test execution | ||
|
|
||
| #### End-to-End Tests | ||
| - **Cypress Core**: `make run-cypress-core` or `yarn workspace @mds/core-web cypress run` | ||
| - **Timeout: 30+ minutes** - Full user scenarios | ||
| - **NEVER CANCEL**: Browser automation, database interactions | ||
|
|
||
| - **Functional Tests**: `cd tests/functional-tests && ./run_tests.sh` | ||
| - **Timeout: 45+ minutes** - Gradle-based Geb/Spock tests | ||
| - **NEVER CANCEL**: Complete end-to-end validation | ||
|
|
||
| ### Validation Requirements | ||
|
|
||
| #### Pre-commit Validation | ||
| **ALWAYS run these before committing** (when dependencies are available): | ||
| 1. **Linting**: | ||
| - Frontend: `yarn workspace @mds/core-web lint` | ||
| - Root: `yarn lint` (runs across all workspaces) | ||
| 2. **Formatting**: `prettier --write --config .prettierrc.json <files>` | ||
| 3. **Type Checking**: Automatic via TypeScript compilation | ||
|
|
||
| **Note**: In environments without full dependencies, focus on manual code review and structural validation. | ||
|
|
||
| #### Manual Testing Scenarios | ||
| **CRITICAL**: After making changes, ALWAYS test these user flows: | ||
|
|
||
| **Core Web (Ministry Interface)**: | ||
| 1. Login with IDIR credentials | ||
| 2. Navigate to Mine Dashboard | ||
| 3. Create/edit a mine record | ||
| 4. Upload a document | ||
| 5. Generate a report | ||
|
|
||
| **Minespace Web (Proponent Interface)**: | ||
| 1. Login with BCeID credentials | ||
| 2. Access "My Mines" dashboard | ||
| 3. Submit a permit application | ||
| 4. Upload required documents | ||
| 5. Track application status | ||
|
|
||
| **API Integration**: | ||
| 1. Test API endpoints with valid/invalid data | ||
| 2. Verify authentication token handling | ||
| 3. Check database transaction rollback on errors | ||
|
|
||
| ### Common Development Tasks | ||
|
|
||
| #### Environment Issues | ||
| - **Docker Problems**: `make clean` then `make be` (removes all containers/volumes) | ||
| - **Node Version**: `nvm use` (uses .nvmrc file) | ||
| - **Dependency Issues**: Delete `node_modules`, run `yarn install` | ||
| - **Database Reset**: `make cleandb` (destroys and recreates database) | ||
|
|
||
| #### Architecture Navigation | ||
| - **Frontend Routes**: `services/core-web/src/routes/` and `services/minespace-web/src/routes/` | ||
| - **API Endpoints**: `services/core-api/app/api/` | ||
| - **Database Migrations**: `migrations/sql/` | ||
| - **Shared Components**: `services/common/src/` | ||
|
|
||
| #### Key File Locations | ||
| - **Configuration**: Look for `.env`, `webpack.config.js`, `package.json` in service directories | ||
| - **Tests**: `tests/` folders in each service, `cypress/` for E2E | ||
|
Collaborator
There was a problem hiding this comment. not all test files are in |
||
| - **Documentation**: `README.md` files in each service directory | ||
|
|
||
| ### Known Issues and Workarounds | ||
|
|
||
| #### Network Dependencies (CRITICAL LIMITATIONS) | ||
| - **FontAwesome Pro**: Requires ARTIFACTORY_TOKEN for private BC Gov registry (artifacts.developer.gov.bc.ca) | ||
|
Collaborator
There was a problem hiding this comment. This is probably an issue for copilot, but locally we can grab these from openshift. Not sure how much of this is supposed to be for humans as well. No access to openshift is definitely a limitation |
||
| - **BC Gov Artifacts**: External registry unreachable in sandboxed environments (EAI_AGAIN errors) | ||
| - **Yarn Install Failures**: External dependencies cause `yarn install` and Docker builds to fail | ||
| - **Docker Build Failures**: `make lite`, `make be`, and `make all` will fail due to network restrictions | ||
|
|
||
| #### Workarounds for Limited Environments | ||
| 1. **Manual .env Setup**: Copy .env-example files to .env instead of using `make env` | ||
| 2. **Skip Yarn Install**: Document the failure, note that full dependency installation is not possible | ||
| 3. **Document Build Failures**: Note that Docker-based development requires network access to BC Gov services | ||
| 4. **Use Local Development**: Focus on code analysis, documentation, and structure understanding | ||
|
|
||
| #### Alternative Validation in Restricted Environments | ||
| When full builds are not possible: | ||
| 1. **Code Structure Analysis**: Use file system exploration and search to understand architecture | ||
| 2. **Manual Code Review**: Check TypeScript/JavaScript syntax, Python imports, and API structure | ||
| 3. **Documentation Review**: Examine README files, inline comments, and test files | ||
| 4. **Static Analysis**: Look for linting configurations (.eslintrc, .flake8) and follow their rules | ||
| 5. **Configuration Validation**: Verify .env-example files have required variables | ||
|
|
||
| #### Platform-Specific | ||
| - **M1 Mac**: Use `docker-compose -f docker-compose.M1.yaml` or let Makefile auto-detect | ||
| - **Windows**: Use WSL2 or Linux containers, path compatibility issues with native Windows | ||
|
|
||
| #### CodeSpaces | ||
| - **Special Commands**: `make init` and `make restart` designed for CodeSpaces | ||
| - **Desktop Access**: Use remote desktop on port 6080 for Cypress GUI testing | ||
|
|
||
| ### Critical Warnings | ||
|
|
||
| 1. **NEVER CANCEL BUILDS**: All build commands can take 30+ minutes | ||
| 2. **SET PROPER TIMEOUTS**: Minimum 60 minutes for full builds, 30 minutes for tests | ||
| 3. **VALIDATE MANUALLY**: Always test user scenarios after code changes | ||
| 4. **CHECK CI REQUIREMENTS**: Run linting and formatting before commits | ||
| 5. **MONITOR DOCKER RESOURCES**: Large build processes require adequate disk space | ||
|
|
||
| ### Quick Reference Commands | ||
|
|
||
| ```bash | ||
| # Essential workflow | ||
| make valid # Validate environment (1 min) | ||
| make env # Setup environment (5+ min, NEVER CANCEL) | ||
| yarn install # Install dependencies (10+ min, NEVER CANCEL) | ||
| make be # Start backend (15+ min, NEVER CANCEL) | ||
| make seeddb # Seed database (5+ min, NEVER CANCEL) | ||
|
|
||
| # Development | ||
| make stop # Stop all services | ||
| make clean # Full cleanup | ||
| make rebuild # Rebuild containers (30+ min, NEVER CANCEL) | ||
|
|
||
| # Testing | ||
| make testbe # Backend tests (20+ min, NEVER CANCEL) | ||
| yarn workspace @mds/core-web run ci-test # Frontend tests (15+ min, NEVER CANCEL) | ||
|
Collaborator
There was a problem hiding this comment. is it worth mentioning the |
||
|
|
||
| # Manual validation required after changes | ||
| yarn lint # Code linting | ||
| prettier --write # Code formatting | ||
| ``` | ||
|
|
||
| ### Repository Structure Summary | ||
| ``` | ||
| / | ||
| ├── services/ | ||
| │ ├── core-web/ # Ministry frontend (React/TypeScript) | ||
| │ ├── minespace-web/ # Proponent frontend (React/TypeScript) | ||
| │ ├── common/ # Shared frontend components | ||
| │ ├── core-api/ # Main backend API (Python Flask) | ||
| │ ├── document-manager/ # Document storage service | ||
| │ └── ... # Additional microservices | ||
| ├── migrations/ # Database schema migrations | ||
| ├── tests/ # Integration and functional tests | ||
| ├── bin/ # Build and utility scripts | ||
| ├── Makefile # Main automation commands | ||
| └── docker-compose.yaml # Container orchestration | ||
| ``` | ||
|
|
||
| **Remember**: This codebase requires patience with build times and careful manual validation of changes. The CI/CD pipeline expects all formatting, linting, and testing standards to be met. | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Something is wonky here! I ran
make validafter a long time of not running it....did we secretly update to v20.19.0 at some point? That's what I've got locally but it doesn't match either file that references the version. Anyway, nothing seems to explode with 20.19.0. 🤷♀️