Skip to content

Architecture Overview

Jump to bottom
nbmaiti edited this page Nov 6, 2025 · 2 revisions

Introduction

Module Organization

The following block diagram illustrates the major components and their relationships in the DMT Console.

flowchart TD
    A[cmd/app main.go] --> B[config]
    A --> C[internal/app]
    A --> internal_controller[internal/app]

        subgraph internal_controller[internal]
            direction TB
            H[internal/controller/http]
            I[internal/controller/openapi]
            J[internal/controller/ws]
        end
    internal_controller --> E[internal/usecase]

    F --> L[internal/entity/dto]
    E --> F[internal/entity]
    E --> G[pkg/db]
    F --> M[internal/entity/github]
    G --> N[pkg/consoleerrors]
    G --> O[pkg/logger]
    G --> P[pkg/httpserver]
    C --> K[internal/app/migrations]

    classDef main fill:#fffbe6,stroke:#333,stroke-width:2px
    classDef config fill:#e6f7ff,stroke:#333,stroke-width:1px
    classDef controller fill:#f0e6ff,stroke:#333,stroke-width:1px
    classDef usecase fill:#ffe6f7,stroke:#333,stroke-width:1px
    classDef entity fill:#f7ffe6,stroke:#333,stroke-width:1px
    classDef pkg fill:#e6e6ff,stroke:#333,stroke-width:1px
    classDef test fill:#fff0e6,stroke:#333,stroke-width:1px
    classDef mocks fill:#e6f7e6,stroke:#333,stroke-width:1px
    class A main
    class B config
    class C,K app
    class D,H,I,J controller
    class E usecase
    class F,L,M entity
    class G,N,O,P pkg
Loading

Components

  • cmd/app main.go: Application entry point - initializes and starts the server
  • config: Configuration management - handles environment variables and settings
  • internal/app: Application layer - orchestrates startup and shutdown processes
  • internal/app/migrations: Database migration scripts and management
  • internal/controller: Presentation layer - handles incoming requests and responses
  • internal/controller/http: HTTP request handlers and routing
  • internal/controller/openapi: OpenAPI specification and documentation endpoints
  • internal/controller/ws: WebSocket connection handlers for real-time communication
  • internal/usecase: Business logic layer - implements application use cases
  • internal/entity: Domain entities and business objects
  • internal/entity/dto: Data Transfer Objects for API communication
  • internal/entity/github: GitHub-specific domain models and types
  • pkg/db: Database connection and query utilities
  • pkg/consoleerrors: Custom error types and error handling utilities
  • pkg/logger: Logging configuration and utilities
  • pkg/httpserver: HTTP server setup and middleware
  • internal/mocks: Mock implementations for interfaces and components, used in unit and integration tests to simulate dependencies and isolate logic under test
  • integration-test: End-to-end testing suite
  • tmp: Temporary files and build artifacts

Database, Secrets, Authentication Flow

The diagram below details how configuration, secrets, authentication, and network endpoints interact in the Console application.

flowchart TD
    classDef netio stroke:#0074D9,stroke-width:3px
    classDef noteedge stroke:#666,stroke-width:2px,stroke-dasharray:5 5
    
    subgraph ConfigSources[Configuration Sources]
        ENV[.env / Environment Variables]
        YAML[config.yml]
    end
    
    subgraph SecretsSources[Secrets Sources]
        ENVKEY[APP_ENCRYPTION_KEY env]
        KEYRING[System Keyring D-Bus]
    end
    
    DECIDE[Choose Key Source]
    
    subgraph TLSAuth[HTTPS Authentication]
        TLSCFG[TLS Config]
        CERTS[Certificates / Keys]
        JWT[JWT/Auth authentication]
    end
    
    subgraph HTTPRoutes[HTTP/HTTPS Routing]
        HTTP[HTTP Route]
        HTTPS[HTTPS Route]
        CTRL[Controller internal/controller/http]
        WSMAN[WSMAN internal/controller/ws]
    end
    
    EXT_HTTP[External HTTP/HTTPS Endpoint curl/web UI/frontend]
    EXT_WSMAN[External WSMAN Endpoint device only]
    APP[cmd/app/main.go]
    CFG[config.NewConfig]
    DBMGR[pkg/db]
    SEC[security.Storage]
    SQLITE[SQLite DB]
    
    note1[APP reads DB path from config/env, passes to DB manager]
    note2[APP reads encryption key from env or keyring via security.Storage mutually exclusive]
    note3[DB manager handles migrations, errors, logging]
    note4[security.Storage abstracts secrets backend env or keyring]
    note5[TLS config and certs loaded from config/env, used for HTTPS authentication]
    note6[External clients interact via HTTP/HTTPS/WSMAN endpoints; HTTP/HTTPS routes handled by controller, HTTPS uses TLS config/certs; WSMAN handles device management via WS protocols]
    
    ENV --> CFG
    YAML --> CFG
    CFG --> APP
    APP --> DBMGR
    DBMGR --> SQLITE
    DECIDE --> ENVKEY
    DECIDE --> KEYRING
    ENVKEY --> SEC
    KEYRING --> SEC
    SEC --> APP
    CFG --> TLSCFG
    TLSCFG --> CERTS
    TLSCFG --> JWT
    JWT --> CTRL
    TLSCFG --> APP
    APP --> HTTP
    APP --> HTTPS
    EXT_HTTP <--> CTRL
    EXT_WSMAN <--> WSMAN
    HTTP --> CTRL
    HTTPS --> CTRL
    CTRL --> WSMAN
    TLSCFG --> HTTPS
        APP -..- note1
        SEC -..- note2
        DBMGR -..- note3
        SEC -..- note4
        TLSCFG -..- note5
        CTRL -..- note6
    
    class EXT_HTTP,EXT_WSMAN,HTTP,HTTPS,WSMAN netio
    class note1,note2,note3,note4,note5,note6 noteedge
Loading

Components explanations

cmd/app/main.go

Configuration and logger initialization. Then the main function "continues" in internal/app/app.go.

config

Configuration. First, config.yml is read, then environment variables overwrite the yaml config if they match. The config structure is in the config.go. The env-required: true tag obliges you to specify a value (either in yaml, or in environment variables).

For configuration, we chose the cleanenv library. It does not have many stars on GitHub, but is simple and meets all the requirements.

Reading the config from yaml contradicts the ideology of 12 factors, but in practice, it is more convenient than reading the entire config from ENV. It is assumed that default values are in yaml, and security-sensitive variables are defined in ENV.

docs

Swagger documentation. Auto-generated by swag library. You don't need to correct anything by yourself.

integration-test

Integration tests. They are launched as a separate container, next to the application container. It is convenient to test the Rest API using go-hit.

internal/app

There is always one Run function in the app.go file, which "continues" the main function.

This is where all the main objects are created. Dependency injection occurs through the "New ..." constructors (see Dependency Injection). This technique allows us to layer the application using the Dependency Injection principle. This makes the business logic independent from other layers.

Next, we start the server and wait for signals in select for graceful completion. If app.go starts to grow, you can split it into multiple files.

For a large number of injections, wire can be used.

The migrate.go file is used for database auto migrations. It is included if an argument with the migrate tag is specified. For example:

$ go run -tags migrate ./cmd/app

internal/controller

Server handler layer (MVC controllers). The template shows 2 servers:

  • REST http (Gin framework)

Server routers are written in the same style:

  • Handlers are grouped by area of application (by a common basis)
  • For each group, its own router structure is created, the methods of which process paths
  • The structure of the business logic is injected into the router structure, which will be called by the handlers

internal/controller/http

Simple REST versioning. For v2, we will need to add the http/v2 folder with the same content. And in the file internal/app add the line:

handler := gin.New()
v1.NewRouter(handler, t)
v2.NewRouter(handler, t)

In v1/router.go and above the handler methods, there are comments for generating swagger documentation using swag.

internal/entity

Entities of business logic (models) can be used in any layer. There can also be methods, for example, for validation.

internal/usecase

Business logic.

  • Methods are grouped by area of application (on a common basis)
  • Each group has its own structure
  • One file - one structure

Repositories, webapi, rpc, and other business logic structures are injected into business logic structures (see Dependency Injection).

internal/usecase/repo

A repository is an abstract storage (database) that business logic works with.

internal/usecase/webapi

It is an abstract web API that business logic works with. For example, it could be another microservice that business logic accesses via the REST API. The package name changes depending on the purpose.

Dependency Injection

In order to remove the dependence of business logic on external packages, dependency injection is used.

For example, through the New constructor, we inject the dependency into the structure of the business logic. This makes the business logic independent (and portable). We can override the implementation of the interface without making changes to the usecase package.

package usecase

import (
    // Nothing!
)

type Repository interface {
    Get()
}

type UseCase struct {
    repo Repository
}

func New(r Repository) *UseCase{
    return &UseCase{
        repo: r,
    }
}

func (uc *UseCase) Do()  {
    uc.repo.Get()
}

It will also allow us to do auto-generation of mocks (for example with mockery) and easily write unit tests.

We are not tied to specific implementations in order to always be able to change one component to another. If the new component implements the interface, nothing needs to be changed in the business logic.

Clean Architecture

Key idea

Programmers realize the optimal architecture for an application after most of the code has been written.

A good architecture allows decisions to be delayed to as late as possible.

The main principle

Dependency Inversion (the same one from SOLID) is the principle of dependency inversion. The direction of dependencies goes from the outer layer to the inner layer. Due to this, business logic and entities remain independent from other parts of the system.

So, the application is divided into 2 layers, internal and external:

  1. Business logic (Go standard library).
  2. Tools (databases, servers, message brokers, any other packages and frameworks).

Clean Architecture

The inner layer with business logic should be clean. It should:

  • Not have package imports from the outer layer.
  • Use only the capabilities of the standard library.
  • Make calls to the outer layer through the interface (!).

The business logic doesn't know anything about Postgres or a specific web API. Business logic has an interface for working with an abstract database or abstract web API.

The outer layer has other limitations:

  • All components of this layer are unaware of each other's existence. How to call another from one tool? Not directly, only through the inner layer of business logic.
  • All calls to the inner layer are made through the interface (!).
  • Data is transferred in a format that is convenient for business logic (internal/entity).

For example, you need to access the database from HTTP (controller). Both HTTP and database are in the outer layer, which means they know nothing about each other. The communication between them is carried out through usecase (business logic):

sequenceDiagram
    participant HTTP
    participant UseCase
    participant Repository as Repository (Postgres)
    
    HTTP->>UseCase: request
    UseCase->>Repository: query
    Repository-->>UseCase: data
    UseCase-->>HTTP: response
Loading

The symbols > and < show the intersection of layer boundaries through Interfaces. The same is shown in the picture:

Example

Or more complex business logic:

sequenceDiagram
    participant HTTP
    participant UseCase
    participant Repository
    participant WebAPI
    participant RPC
    
    HTTP->>UseCase: request
    UseCase->>Repository: query
    Repository-->>UseCase: data
    UseCase->>WebAPI: call
    WebAPI-->>UseCase: response
    UseCase->>RPC: call
    RPC-->>UseCase: response
    UseCase->>Repository: store
    Repository-->>UseCase: confirmation
    UseCase-->>HTTP: final response
Loading

Layers

Example

Clean Architecture Terminology

  • Entities are structures that business logic operates on. They are located in the internal/entity folder. In MVC terms, entities are models.
  • Use Cases is business logic located in internal/usecase.

The layer with which business logic directly interacts is usually called the infrastructure layer. These can be repositories internal/usecase/repo, external webapi internal/usecase/webapi, any pkg, and other microservices. In the template, the infrastructure packages are located inside internal/usecase.

You can choose how to call the entry points as you wish. The options are:

  • controller (in our case)
  • delivery
  • transport
  • gateways
  • entrypoints
  • primary
  • input

Additional layers

The classic version of Clean Architecture was designed for building large monolithic applications and has 4 layers.

In the original version, the outer layer is divided into two more, which also have an inversion of dependencies to each other (directed inward) and communicate through interfaces.

The inner layer is also divided into two (with separation of interfaces), in the case of complex logic.

Complex tools can be divided into additional layers. However, you should add layers only if really necessary.

Alternative approaches

In addition to Clean architecture, Onion architecture and Hexagonal (Ports and adapters) are similar to it. Both are based on the principle of Dependency Inversion. Ports and adapters are very close to Clean Architecture, the differences are mainly in terminology.

Architecture Flow

  • Solid arrows show the main flow of control and dependencies
  • Dotted arrows indicate supporting/testing modules
  • Colors group related components by architectural layer
  • Network I/O and authentication flows are shown in the detailed diagram above

Useful links

Console Wiki

ADRs

Clone this wiki locally