Trading System - Soft Real-Time Scheduler

CPSC 346 Extra Credit Project - C Implementation

A multi-threaded soft real-time trading system demonstrating advanced Operating Systems concepts including explicit state machines, deadline-based scheduling, lock-free data structures, CPU affinity, RT scheduling, and fault injection.

Architecture

┌─────────────────────────────────────────────────────────────────────────┐
│                           Main Process                                   │
├─────────────────────────────────────────────────────────────────────────┤
│                                                                          │
│   ┌─────────────┐      ┌─────────────┐      ┌─────────────────────────┐ │
│   │    Feed     │      │   Ring      │      │         Engine          │ │
│   │   Thread    │─────>│   Buffer    │─────>│                         │ │
│   │             │      │   (SPSC)    │      │  ┌───────────────────┐  │ │
│   │ - Tick Gen  │      │             │      │  │    Dispatcher     │  │ │
│   │ - Workloads │      │ Lock-free   │      │  │      Thread       │  │ │
│   │ - Deadline  │      │ IPC Queue   │      │  └─────────┬─────────┘  │ │
│   │   Timing    │      │             │      │            │            │ │
│   └─────────────┘      └─────────────┘      │            v            │ │
│                                              │      ┌─────────┐       │ │
│   State Machine:                             │      │  MPMC   │       │ │
│   INIT -> RUNNING -> DRAINING -> SHUTDOWN    │      │  Queue  │       │ │
│                                              │      └────┬────┘       │ │
│                                              │           │            │ │
│                                              │      ┌────┴────┐       │ │
│                                              │      v         v       │ │
│                                              │  ┌──────┐  ┌──────┐    │ │
│                                              │  │Worker│  │Worker│    │ │
│                                              │  │  0   │  │  N   │    │ │
│                                              │  └──────┘  └──────┘    │ │
│                                              └─────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────┘

Features Checklist

Feature	Status	Description
POSIX threads (pthread) worker pool	✅	Configurable number of worker threads
Mutex + condition variables	✅	MPMC queue with pthread_mutex/cond
Bounded ring buffer queue	✅	Fixed-size SPSC and MPMC queues
Backpressure policies	✅	BLOCK, DROP, SHED_OLD
Graceful shutdown SIGINT/SIGTERM	✅	Proper draining + pthread_join
CPU affinity (pthread_setaffinity_np)	✅	Pin threads to specific CPUs
RT scheduling (SCHED_FIFO/RR)	✅	pthread_setschedparam with error handling
Latency histograms (p50/p95/p99)	✅	High-resolution timing instrumentation
Structured logging	✅	Levels, timestamps, thread ID
Config file (INI format)	✅	Full configuration via file
CLI flags	✅	Comprehensive command-line options
Unit tests	✅	Test harness for queue/state machine
Makefile (Debug/Release)	✅	Separate builds with sanitizers
Static analysis warnings	✅	-Wall -Wextra -Wpedantic
Metrics export (CSV/JSON)	✅	Export to files for analysis
Fault injection	✅	Worker hang, crash, tick drop

Building

# Release build
make

# Debug build (with AddressSanitizer + UBSan)
make debug

# Run unit tests
make test

# Clean build artifacts
make clean

Quick Start

# Build
make

# Basic run (200 ticks/sec, 5 seconds, 2 workers)
./trading_system -r 200 -d 5 -n 2

# With metrics export
./trading_system -r 500 -d 10 -n 4 -C metrics.csv -J metrics.json

# With config file
./trading_system -c config.ini

Command-Line Options

Configuration

Flag	Description	Default
-c <file>	Load config from INI file	-
-r <rate>	Tick rate per second	100
-w <type>	Workload: steady, burst, random	steady
-d <sec>	Duration in seconds	30
-n <num>	Number of worker threads	1
-s <strat>	Strategy: ma, mom	ma
-S	Use shared strategy (with locks)	false
-y <syms>	Comma-separated symbols	AAPL

Threading

Flag	Description	Default
-a <cpu>	Enable CPU affinity from CPU N	disabled
-R <prio>	Enable RT scheduling (priority 1-99)	disabled
-b <pol>	Backpressure: block, drop, shed	block

Output

Flag	Description	Default
-l <level>	Log level: debug, info, warn, error	info
-L <file>	Log to file	stderr
-C <file>	Export CSV metrics	-
-J <file>	Export JSON metrics	-

Fault Injection

Flag	Description
-f worker_hang:<ms>	Workers randomly hang for N ms
-f worker_crash	Workers randomly crash
-f tick_drop:<prob>	Drop ticks with probability 0.0-1.0
-f early_exit:<ticks>	Feed exits after N ticks
-f queue_overflow	Disable backpressure handling

Configuration File (config.ini)

[feed]
tick_rate = 500
duration = 10
workload = steady
symbols = AAPL, MSFT, GOOG, AMZN

[engine]
workers = 4
strategy = ma
shared_strategy = false
backpressure = block

[threading]
enable_affinity = false
cpu_start = 0
enable_rt = false
rt_priority = 50
rt_policy = fifo

[logging]
level = info

[metrics]
export_csv = true
export_json = true

[faults]
enabled = false
tick_drop_prob = 0.0
worker_hang_prob = 0.0
worker_hang_ms = 100

Output Example

================================================================================
         TRADING SYSTEM - CPSC 346 EXTRA CREDIT (C Implementation)
================================================================================

Features:
  [x] Explicit State Machine (INIT -> RUNNING -> DRAINING -> SHUTDOWN)
  [x] Deadline-based Scheduling (monotonic clock, no sleep())
  [x] Lock-free Ring Buffer for IPC
  [x] POSIX Threads with Mutex + Condition Variables
  [x] CPU Affinity: Disabled
  [x] RT Scheduling: Disabled
  [x] Structured Logging with Levels
  [x] Latency Histograms (p50/p95/p99)
  [x] Config File Support (INI format)
  [x] Metrics Export (CSV/JSON)
  [x] Fault Injection System

Configuration:
  Tick rate:       500/sec
  Workload:        steady
  Duration:        10 seconds
  Workers:         4
  Strategy:        Moving Average
  Shared Strategy: No
  Backpressure:    BLOCK
  Symbols:         AAPL
  Fault Injection: Disabled

================================================================================

12:34:56.789 [INFO ] [T12345] [MAIN    ] Starting system (PID: 54321)
12:34:56.790 [INFO ] [T12346] [FEED    ] Initializing (PID: 54321)
12:34:56.790 [INFO ] [T12346] [FEED    ] State: INIT -> RUNNING
...

================================================================================
                           ENGINE STATISTICS
================================================================================
Runtime:           10.00 seconds
Ticks received:    5000
Ticks processed:   5000
Ticks dropped:     0
Throughput:        500.0 ticks/sec

Signals generated: BUY=234, SELL=256, Total=490
Queue overflows:   0

Latency (tick-to-signal):
  Average:         45.234 us (0.045 ms)
  Minimum:         12.000 us (0.012 ms)
  Maximum:         1234.000 us (1.234 ms)
  p50:             40.000 us
  p95:             120.000 us
  p99:             350.000 us

Worker Statistics:
  Worker-0:        1250 ticks processed
  Worker-1:        1248 ticks processed
  Worker-2:        1251 ticks processed
  Worker-3:        1251 ticks processed
================================================================================

Metrics Export

CSV Format

metric,value
runtime_sec,10.000
ticks_received,5000
ticks_processed,5000
latency_avg_us,45.234
latency_p50_us,40.000
latency_p95_us,120.000
latency_p99_us,350.000
throughput_tps,500.0

JSON Format

{
  "runtime_sec": 10.000,
  "ticks": {
    "received": 5000,
    "processed": 5000,
    "dropped": 0
  },
  "latency_us": {
    "avg": 45.234,
    "min": 12.000,
    "max": 1234.000,
    "p50": 40.000,
    "p95": 120.000,
    "p99": 350.000
  },
  "throughput_tps": 500.0
}

Testing

# Unit tests
make test

# Demo scenarios
make demo

# Performance test (10000 ticks/sec)
make perf

# Fault injection tests
make test-faults

# Memory check (Linux only)
make valgrind

Design Decisions

Lock-free Ring Buffer for IPC

• Single Producer / Single Consumer (SPSC) pattern
• No mutex overhead on hot path
• Cache-line aligned head/tail to prevent false sharing

MPMC Queue for Worker Distribution

• Multiple workers need fair access to work items
• Condition variables enable efficient blocking when queue is empty
• Timeout support prevents deadlocks

Deadline-based Scheduling (No sleep())

• sleep() is imprecise and wastes CPU time slice
• Spin-waiting with pause instruction is more responsive
• Monotonic clock ensures consistent timing

Explicit State Machine

• Clear lifecycle: INIT → RUNNING → DRAINING → SHUTDOWN
• Proper resource cleanup during shutdown
• Easy to add new states or transitions

Files

File	Description
trading_system.h	Common header (types, utilities, logging, config)
main.c	Entry point, CLI parsing, setup
feed.c	Market data feed (tick generation)
engine.c	Trading engine (workers, strategies)
test_system.c	Unit tests
config.ini	Example configuration
Makefile	Build system

Requirements

• C11 compiler (GCC ≥ 4.9 or Clang ≥ 3.3)
• POSIX-compliant OS (Linux or macOS)
• pthread library
• math library (-lm)

License

Educational project for CPSC 346 at Gonzaga University.