Skip to content

Development

Jump to bottom
Jeffrie Budde edited this page Apr 10, 2026 · 1 revision

Development guide

This guide covers local setup, testing, code quality, and contributing to the OneLogin Migration Wizard.

Environment setup

Prerequisites

  • Python 3.10 or higher (< 3.14)
  • pip and virtualenv
  • Git for version control

Initial setup

  1. Clone the repository:
git clone <repository-url>
cd Okta_to_Onelogin
  1. Create and activate virtual environment:
python3 -m venv .venv
source .venv/bin/activate  # macOS/Linux
# or
.venv\Scripts\activate  # Windows
  1. Install packages:

This project uses a monorepo structure with three packages. Use the provided script to install all packages:

# Install all packages in development mode (recommended)
./scripts/dev-install.sh

Or install packages individually:

# Core package (required)
pip install -e packages/core

# CLI package
pip install -e packages/cli

# GUI package
pip install -e packages/gui

# With test dependencies
pip install -e "packages/core[test]"
pip install -e "packages/cli[test]"
pip install -e "packages/gui[test]"

Note: Editable mode (-e) allows code changes to take effect immediately without reinstalling.

Verifying installation

# Check command is available
onelogin-migration --help

# Or via GUI
onelogin-migration-gui

# Verify module imports
python -c "from onelogin_migration_core import manager; print('Core OK')"
python -c "from onelogin_migration_cli import app; print('CLI OK')"
python -c "from onelogin_migration_gui import main; print('GUI OK')"

# Run tests
./scripts/test-all.sh

Project structure

Okta_to_Onelogin/                      # Monorepo root
├── packages/                          # Independent packages
│   ├── core/                          # Core migration library
│   │   ├── src/onelogin_migration_core/
│   │   │   ├── __init__.py
│   │   │   ├── clients.py             # API clients
│   │   │   ├── config.py              # Configuration
│   │   │   ├── config_parser.py       # YAML parser
│   │   │   ├── credentials.py         # Credential management
│   │   │   ├── secure_settings.py     # Settings manager
│   │   │   ├── manager.py             # Migration orchestration
│   │   │   ├── exporters.py           # Source data export
│   │   │   ├── importers.py           # OneLogin import
│   │   │   ├── transformers.py        # Field mapping
│   │   │   ├── custom_attributes.py   # Attribute provisioning
│   │   │   ├── csv_generator.py       # CSV generation
│   │   │   ├── state_manager.py       # State persistence
│   │   │   ├── progress.py            # Progress tracking
│   │   │   ├── constants.py           # Constants
│   │   │   └── db/                    # Database management
│   │   │       ├── database_manager.py
│   │   │       ├── connector_db.py    # Connector catalog
│   │   │       ├── connector_refresh.py
│   │   │       ├── telemetry.py       # Analytics
│   │   │       └── encryption.py
│   │   ├── tests/                     # Core tests
│   │   ├── pyproject.toml
│   │   └── README.md
│   ├── cli/                           # Command-line interface
│   │   ├── src/onelogin_migration_cli/
│   │   │   ├── __init__.py
│   │   │   ├── app.py                 # Main CLI
│   │   │   ├── credentials.py         # Credentials commands
│   │   │   ├── database.py            # Database commands
│   │   │   └── telemetry.py           # Telemetry commands
│   │   ├── tests/                     # CLI tests
│   │   ├── pyproject.toml
│   │   └── README.md
│   └── gui/                           # Graphical interface
│       ├── src/onelogin_migration_gui/
│       │   ├── __init__.py
│       │   ├── main.py                # GUI entry point
│       │   ├── components.py          # UI components
│       │   ├── theme_manager.py       # Theme support
│       │   ├── helpers.py
│       │   ├── steps/                 # Wizard steps
│       │   │   ├── base.py
│       │   │   ├── welcome.py
│       │   │   ├── source.py
│       │   │   ├── target.py
│       │   │   ├── options.py
│       │   │   ├── objects.py
│       │   │   ├── analysis.py        # Analysis step (NEW)
│       │   │   ├── summary.py
│       │   │   └── progress.py
│       │   ├── dialogs/
│       │   │   └── analysis_detail/   # Analysis dialogs (NEW)
│       │   │       ├── dialog.py
│       │   │       ├── export/        # CSV/XLSX exporters
│       │   │       ├── tables/        # Table managers
│       │   │       └── utils/         # Formatters, validators
│       │   └── styles/                # UI theming
│       ├── tests/                     # GUI tests
│       ├── pyproject.toml
│       └── README.md
├── scripts/                           # Development scripts
│   ├── dev-install.sh                 # Install all packages
│   ├── test-all.sh                    # Run all tests
│   ├── lint-all.sh                    # Lint all packages
│   ├── format-all.sh                  # Format all packages
│   └── build_catalog_db.py            # Database utilities
├── config/
│   ├── migration.template.yaml        # YAML template
│   └── migration.yaml                 # User config (gitignored)
├── templates/
│   └── user-upload-template.csv
├── tools/                             # Build tools
│   ├── build_app.sh                   # macOS app build
│   └── build_app.bat                  # Windows app build
├── dist/                              # Built applications
├── artifacts/                         # Generated exports (gitignored)
└── README.md                          # Main documentation

Running tests

Full test suite

# Run tests for all packages
./scripts/test-all.sh

# Run all tests with pytest directly
pytest

# With verbose output
pytest -v

# With coverage report for all packages
pytest --cov=onelogin_migration_core --cov=onelogin_migration_cli --cov=onelogin_migration_gui

# Generate HTML coverage report
pytest --cov=onelogin_migration_core --cov-report=html
# Open htmlcov/index.html in browser

Package-specific tests

# Core package tests
cd packages/core && pytest tests/ -v

# CLI package tests
cd packages/cli && pytest tests/ -v

# GUI package tests
cd packages/gui && pytest tests/ -v

Specific tests

# Run single test file
pytest packages/core/tests/test_config.py

# Run specific test function
pytest packages/core/tests/test_config.py::test_load_settings

# Run tests matching pattern
pytest -k "test_cli"

# Run tests for specific package
pytest packages/core/

Test organization

Core Package Tests (packages/core/tests/):

Test File Coverage
test_credentials.py Secure credential storage, keyring, vault
test_secure_settings.py Non-sensitive settings manager
test_config_parser.py YAML parsing, credential extraction
test_clients.py API client behavior, rate limiting, pagination
test_config.py Configuration dataclasses and validation
test_migration.py Migration orchestration, state management
test_provider_abstraction.py Multi-provider architecture, SourceClient protocol, deprecation warnings
test_field_mapper.py FieldMapper protocol, OktaFieldMapper, manager delegation
test_progress.py Progress calculation and snapshot updates
test_database_manager.py Database operations and schema
test_load_connectors.py Connector database loading

CLI Package Tests (packages/cli/tests/):

Test File Coverage
test_cli_entrypoint.py CLI command registration
test_credentials_cli.py Credential management commands
test_database_cli.py Database management commands
test_telemetry_cli.py Telemetry commands

GUI Package Tests (packages/gui/tests/):

Test File Coverage
test_gui_main.py GUI initialization and wizard
test_gui_credentials.py Credential save/prefill, multi-provider ProviderSettingsPage
test_analysis_step.py Analysis step functionality
test_theme_manager.py Theme switching

Writing tests

Use pytest fixtures and mocking:

import pytest
from unittest.mock import Mock, patch

def test_example():
    # Arrange
    settings = MigrationSettings(...)

    # Act
    result = function_under_test(settings)

    # Assert
    assert result.expected_value == "expected"

Code quality

Linting and formatting all packages

# Lint all packages
./scripts/lint-all.sh

# Format all packages
./scripts/format-all.sh

Linting with Ruff (individual packages)

# Check all packages
ruff check packages/

# Auto-fix issues
ruff check packages/ --fix

# Check specific package
ruff check packages/core/src/

Formatting with Black (individual packages)

# Format all code
black packages/

# Check formatting without changes
black packages/ --check

# Format specific package
black packages/core/src/

Type checking (optional)

# Install mypy
pip install mypy

# Run type checking on all packages
mypy packages/core/src/
mypy packages/cli/src/
mypy packages/gui/src/

Making changes

Development workflow

  1. Create feature branch:
git checkout -b feature/my-new-feature

  1. Make changes to code in appropriate package:

    • Core logic: packages/core/src/onelogin_migration_core/
    • CLI commands: packages/cli/src/onelogin_migration_cli/
    • GUI features: packages/gui/src/onelogin_migration_gui/

  2. Add tests in corresponding tests/ directory

  3. Run tests:

pytest
  1. Format and lint:
./scripts/format-all.sh
./scripts/lint-all.sh

  1. Update documentation in wiki if needed

  2. Commit changes:

git add .
git commit -m "Add feature: description"
  1. Push and create PR:
git push origin feature/my-new-feature

Common development tasks

Add new source provider:

  1. Implement FieldMapper protocol in packages/core/src/onelogin_migration_core/field_mapper.py with provider-specific field transforms
  2. Implement SourceClient protocol in packages/core/src/onelogin_migration_core/clients.py, attach the mapper via field_mapper property, and register in _PROVIDER_REGISTRY
  3. Add provider entry to SOURCE_PROVIDERS in packages/gui/src/onelogin_migration_gui/steps/source.py
  4. Add tests covering the new FieldMapper, SourceClient, and build_source_client() dispatch
  5. Update Architecture wiki page with new provider details

Add new CLI command:

  1. Add command function in packages/cli/src/onelogin_migration_cli/app.py
  2. Or create new subcommand module (e.g., for db, telemetry)
  3. Add tests in packages/cli/tests/
  4. Update Command-Line wiki page

Add credential management feature:

  1. Add command to packages/cli/src/onelogin_migration_cli/credentials.py
  2. Implement logic using credential manager from core package
  3. Add tests in packages/cli/tests/test_credentials_cli.py
  4. Update Command-Line wiki page

Add new configuration option:

  1. For non-sensitive settings: Add field to NonSensitiveSettings in packages/core/src/onelogin_migration_core/secure_settings.py
  2. For credentials: Use CredentialManager in packages/core/src/onelogin_migration_core/credentials.py
  3. Update migration from YAML in packages/core/src/onelogin_migration_core/config_parser.py
  4. Add tests in packages/core/tests/
  5. Update Configuration wiki page

Extend field mapping:

  1. Modify mapping logic in packages/core/src/onelogin_migration_core/transformers.py
  2. Update constants in packages/core/src/onelogin_migration_core/constants.py
  3. Add tests with example data
  4. Update README field mapping section

Add GUI wizard step:

  1. Create new step file in packages/gui/src/onelogin_migration_gui/steps/
  2. Inherit from BaseStep
  3. Implement initUI() and isComplete() methods
  4. Register in wizard navigation in main.py
  5. Add tests in packages/gui/tests/
  6. Update GUI wiki page

Add GUI analysis feature:

  1. Add table manager in packages/gui/src/onelogin_migration_gui/dialogs/analysis_detail/tables/
  2. Add export logic in packages/gui/src/onelogin_migration_gui/dialogs/analysis_detail/export/
  3. Update analysis dialog to include new feature
  4. Add tests for data processing
  5. Update GUI wiki page

Modify migration workflow:

  1. Update packages/core/src/onelogin_migration_core/manager.py for orchestration changes
  2. Modify exporters, importers, or transformers in core package
  3. Update state manager if state tracking changes
  4. Add comprehensive tests in packages/core/tests/
  5. Update Architecture wiki page

Add database feature:

  1. Add database logic in packages/core/src/onelogin_migration_core/db/
  2. Add CLI commands in packages/cli/src/onelogin_migration_cli/database.py
  3. Update schema if needed
  4. Add tests in both core and CLI packages
  5. Update Command-Line and Architecture wiki pages

Building and distribution

Build packages

Each package can be built independently:

# Install build tools
pip install build

# Build core package
cd packages/core && python -m build

# Build CLI package
cd packages/cli && python -m build

# Build GUI package
cd packages/gui && python -m build

Build standalone applications

# macOS application
./tools/build_app.sh

# Windows application
./tools/build_app.bat

# Output in dist/

Install from wheels

# Install from built wheels
pip install packages/core/dist/onelogin_migration_core-0.2.0-py3-none-any.whl
pip install packages/cli/dist/onelogin_migration_cli-0.2.0-py3-none-any.whl
pip install packages/gui/dist/onelogin_migration_gui-0.2.0-py3-none-any.whl

Version bumping

Update version in each package's pyproject.toml:

  1. Core package (packages/core/pyproject.toml):
[project]
version = "0.2.1"  # Increment version
  1. CLI package (packages/cli/pyproject.toml):
[project]
version = "0.2.1"
dependencies = [
    "onelogin-migration-core>=0.2.1,<0.3.0",  # Update dependency
    # ...
]
  1. GUI package (packages/gui/pyproject.toml):
[project]
version = "0.2.1"
dependencies = [
    "onelogin-migration-core>=0.2.1,<0.3.0",  # Update dependency
    # ...
]

  1. Update version-specific documentation:

    • Add release notes to CHANGELOG
    • Update version references in README
    • Update wiki pages with version indicators

  2. Rebuild all packages:

cd packages/core && python -m build && cd ../..
cd packages/cli && python -m build && cd ../..
cd packages/gui && python -m build && cd ../..

Version history:

  • v0.2.0 - Monorepo structure, analysis features, database management
  • v0.1.x - Legacy single-package structure

Documentation

Wiki location

The wiki is maintained in this repository at:

<repository-root>/docs/wiki/

Current wiki pages should be mirrored to GitHub Wiki.

Updating wiki

When adding features, update relevant pages:

  • Home.md - Overview and quick start
  • Architecture.md - System design changes
  • Configuration.md - New config options
  • Command-Line.md - CLI command changes
  • GUI.md - GUI workflow updates
  • Development.md - Dev process changes

README vs Wiki

  • README.md - Comprehensive reference for all features
  • Wiki - Organized guides by topic
  • Keep both in sync for major features

Debugging

CLI debugging

# Enable verbose logging
onelogin-migration-tool migrate --config config/migration.yaml -v

# Run with Python debugger
python -m pdb -m onelogin_migration_tool migrate --config config/migration.yaml

GUI debugging

# Run GUI with verbose terminal logging
onelogin-migration-tool gui -v

# Check Qt environment
python -c "from PySide6 import QtCore; print(QtCore.__version__)"

Common issues

Import errors after code changes:

# Reinstall in editable mode
pip install -e .

Tests fail after dependency update:

# Recreate virtual environment
deactivate
rm -rf .venv
python3 -m venv .venv
source .venv/bin/activate
pip install -e .[gui,test]

GUI doesn't show changes:

  • Check if PySide6 cached compiled UI files
  • Restart GUI application
  • Verify changes saved to correct file

Contributing guidelines

Code style

  • Follow PEP 8 (enforced by Black)
  • Use type hints for function signatures
  • Write docstrings for public functions
  • Keep functions focused and testable

Commit messages

  • Use imperative mood: "Add feature" not "Added feature"
  • First line: brief summary (50 chars)
  • Blank line, then detailed explanation if needed

Pull requests

  • Include tests for new functionality
  • Update documentation
  • Ensure CI passes (tests, linting)
  • Reference related issues

Review process

  1. Automated tests run on PR
  2. Code review by maintainers
  3. Address feedback
  4. Merge when approved

Resources

Implementing a new source provider

The full pipeline — export, attribute discovery, user/group/app transforms — is entirely delegated through two protocols: FieldMapper and SourceClient. No wizard logic, manager code, or GUI screens need to change.

Step 1 — Implement FieldMapper

Create a new class in packages/core/src/onelogin_migration_core/field_mapper.py that satisfies the FieldMapper protocol. All nine methods must be implemented.

# field_mapper.py
from .transformers import FieldTransformer
from .constants import KNOWN_STANDARD_FIELDS

class AzureADFieldMapper:
    """FieldMapper for the Microsoft Azure AD source provider.

    Azure AD Graph API returns flat user objects with displayName, userPrincipalName, etc.
    """

    def transform_user(self, raw_user: dict) -> dict | None:
        upn = raw_user.get("userPrincipalName") or raw_user.get("mail")
        if not upn:
            return None
        return {
            "firstname": raw_user.get("givenName"),
            "lastname": raw_user.get("surname"),
            "email": raw_user.get("mail") or upn,
            "username": upn,
            "state": 0 if raw_user.get("accountEnabled") is False else 1,
            "external_id": raw_user.get("id", ""),
            "phone": raw_user.get("businessPhones", [None])[0],
            "department": raw_user.get("department"),
            "title": raw_user.get("jobTitle"),
            "company": raw_user.get("companyName"),
        }

    def transform_group(self, raw_group: dict) -> dict | None:
        name = raw_group.get("displayName")
        return {"name": name} if name else None

    def transform_application(self, raw_app: dict, connector_lookup: dict) -> dict | None:
        labels = self.extract_app_labels(raw_app)
        sign_on = self.extract_app_signon_mode(raw_app)
        for label in labels:
            connectors = connector_lookup.get(label, {})
            connector_id = connectors.get(sign_on) or connectors.get(None)
            if connector_id:
                return self.build_app_payload(raw_app, connector_id)
        return None

    def discover_custom_attributes(self, raw_users: list[dict]) -> set[str]:
        STANDARD = {"id", "userPrincipalName", "mail", "givenName", "surname",
                    "displayName", "accountEnabled", "businessPhones", "department",
                    "jobTitle", "companyName", "mobilePhone", "officeLocation"}
        attrs: set[str] = set()
        for user in raw_users:
            for key, value in user.items():
                if key in STANDARD or value is None:
                    continue
                if isinstance(value, (dict, list)):
                    continue
                if isinstance(value, str) and not value.strip():
                    continue
                normalized = self.normalize_custom_attribute_name(key)
                if normalized:
                    attrs.add(normalized)
        return attrs

    def normalize_custom_attribute_name(self, source_key: str) -> str:
        return FieldTransformer.normalize_custom_attribute_name(source_key)

    def extract_app_labels(self, raw_app: dict) -> list[str]:
        labels = []
        for key in ("displayName", "appDisplayName", "name"):
            candidate = FieldTransformer.normalize_app_label(raw_app.get(key))
            if candidate:
                labels.append(candidate)
        return labels

    def extract_app_signon_mode(self, raw_app: dict) -> str | None:
        # Azure AD uses "signInAudience" and "web"/"spa"/"publicClient" to indicate type
        # Map to a normalized OneLogin sign-on mode string
        sign_in = raw_app.get("signInAudience", "")
        if "AzureADMyOrg" in sign_in or "AzureADMultipleOrgs" in sign_in:
            return "saml_2_0"
        return None

    def build_app_payload(self, raw_app: dict, connector_id: int) -> dict | None:
        name = raw_app.get("displayName") or raw_app.get("appDisplayName")
        if not name:
            return None
        return FieldTransformer.clean_payload({
            "name": name,
            "connector_id": connector_id,
            "description": raw_app.get("description"),
            "visible": True,
            "configuration": {},
        })

    def build_app_parameters(self, raw_app: dict, sign_on_mode: str | None,
                             allows_new_parameters: bool) -> dict:
        if not allows_new_parameters:
            return {}
        params = {}
        web = raw_app.get("web") or {}
        redirect_uris = web.get("redirectUris")
        if isinstance(redirect_uris, list) and redirect_uris:
            params["redirect_uris"] = redirect_uris
        return params

See OktaFieldMapper in field_mapper.py for a complete production reference — it covers SAML/OIDC parameter extraction and Okta-specific field filtering in detail.

Step 2 — Implement SourceClient

Add a new client class in packages/core/src/onelogin_migration_core/clients.py:

# clients.py
class AzureADSourceClient:
    """SourceClient for the Microsoft Azure AD Graph API."""

    def __init__(self, settings: SourceApiSettings) -> None:
        self._settings = settings
        self._mapper = AzureADFieldMapper()

    @property
    def field_mapper(self) -> AzureADFieldMapper:
        return self._mapper

    @property
    def settings(self) -> SourceApiSettings:
        return self._settings

    def test_connection(self) -> tuple[bool, str]:
        # TODO: implement actual auth check
        return True, "ok"

    def list_users(self) -> list[dict]:
        # Call Microsoft Graph API: GET /v1.0/users
        ...

    def list_groups(self) -> list[dict]:
        # Call Microsoft Graph API: GET /v1.0/groups
        ...

    def list_group_memberships(self, groups=None) -> list[dict]:
        ...

    def list_applications(self) -> list[dict]:
        # Call Microsoft Graph API: GET /v1.0/applications
        ...

    def list_policies(self) -> list[dict]:
        return []

    def export_all(self, categories=None) -> dict:
        return {
            "users": self.list_users(),
            "groups": self.list_groups(),
            "memberships": self.list_group_memberships(),
            "applications": self.list_applications(),
        }

# Register so build_source_client() can find it
_PROVIDER_REGISTRY["azure_ad"] = AzureADSourceClient

The SourceApiSettings model accepts arbitrary extra fields — tenant_id, client_id, etc. — through its extra dict, so no model changes are needed.

Step 3 — Add GUI form fields

In packages/gui/src/onelogin_migration_gui/steps/source.py, add your provider to SOURCE_PROVIDERS:

SOURCE_PROVIDERS = {
    "Okta": {
        "domain": {"label": "Okta Domain", "placeholder": "company.okta.com"},
        "token":  {"label": "API Token", "secret": True},
    },
    "Azure AD": {
        "tenant_id":     {"label": "Tenant ID",     "placeholder": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"},
        "client_id":     {"label": "Client ID",     "placeholder": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"},
        "client_secret": {"label": "Client Secret", "secret": True},
    },
}

The ProviderSettingsPage widget reads this dict and rebuilds its form fields automatically. No other GUI code changes.

Step 4 — Config YAML

source:
  provider: azure_ad          # must match the key in _PROVIDER_REGISTRY
  tenant_id: your-tenant-id
  client_id: your-client-id
  # client_secret comes from keyring — never put it in the YAML

Writing tests

Add a test file packages/core/tests/test_azure_ad_field_mapper.py following the same pattern as test_field_mapper.py:

from onelogin_migration_core.field_mapper import AzureADFieldMapper

class TestAzureADFieldMapperUsers:
    def test_transforms_active_user(self):
        mapper = AzureADFieldMapper()
        raw = {
            "id": "aad-001",
            "userPrincipalName": "alice@corp.com",
            "givenName": "Alice",
            "surname": "Smith",
            "mail": "alice@corp.com",
            "accountEnabled": True,
        }
        result = mapper.transform_user(raw)
        assert result["firstname"] == "Alice"
        assert result["email"] == "alice@corp.com"
        assert result["state"] == 1
        assert result["external_id"] == "aad-001"

    def test_disabled_user_maps_state_zero(self):
        mapper = AzureADFieldMapper()
        raw = {"userPrincipalName": "bob@corp.com", "accountEnabled": False, "id": "aad-002"}
        assert mapper.transform_user(raw)["state"] == 0

    def test_returns_none_when_no_upn(self):
        mapper = AzureADFieldMapper()
        assert mapper.transform_user({"givenName": "Ghost"}) is None

Run your new tests the same way as the core suite:

PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 PYTHONPATH=packages/core/src \
  .venv/bin/python -m pytest packages/core/tests/test_azure_ad_field_mapper.py -v

Also add a protocol conformance check to confirm your mapper satisfies isinstance(AzureADFieldMapper(), FieldMapper).

Troubleshooting

build_source_client() raises ValueError: unknown provider The provider string in your YAML must exactly match the key passed to _PROVIDER_REGISTRY. Check spelling and case.

GUI combo shows provider but form is empty The provider key in SOURCE_PROVIDERS (GUI dict) must match the display name you select — they can differ from the registry key but the dict must have an entry.

isinstance(mapper, FieldMapper) returns False A required method is missing. Run python -c "from onelogin_migration_core.field_mapper import FieldMapper, AzureADFieldMapper; import inspect; print([m for m in dir(FieldMapper) if not m.startswith('_') and m not in dir(AzureADFieldMapper)])" to see which ones are absent.

Clone this wiki locally