HFT Spoofing Detection

This repository contains tools and notebooks for detecting spoofing behaviour in high-frequency trading markets. The pipeline trains an XGBoost classifier on simulated orderbook data and applies it to real Euronext data replayed through the Atom execution engine.

Overview

spoofing lib  ──►  simulated order flow  ──►  XGBoost training  ──┐
                                                                    ▼
Euronext data  ──►  data_prep  ──►  Atom replay  ──►  model inference

1. Simulation — the spoofing library generates synthetic orderbook sessions with labelled spoofer agents (is_spoofer target)
2. Training — HFT Training Algo.ipynb trains an XGBoost model on the simulated data using orderbook features (spread, price levels, volume imbalance, order rank, …)
3. Preparation — data_prep formats raw Euronext market data into the Atom order file format
4. Replay — Atom replays the formatted Euronext data through its orderbook engine
5. Inference — the trained model is applied to the replayed Euronext data to detect spoofing

Stack

• JupyterLab with R kernel and language server
• Python (pandas, numpy, scikit-learn, xgboost)
• Java 17 + Maven
• R (tidyverse, ggplot2, dplyr, rJava)

Repositories cloned automatically at startup:

• relmaazouz/atom — Atom orderbook execution engine
• relmaazouz/data_prep — Euronext → Atom format conversion
• relmaazouz/spoofing — Spoofing simulation library
• relmaazouz/ml — ML notebooks (XGBoost training and inference)

Getting Started

Prerequisites

• Docker
• Docker Compose

Start the environment

docker compose up --build

The first build takes several minutes. Subsequent starts are fast.

On startup, the atom and data_prep repositories are automatically cloned into ./notebooks/.

Access JupyterLab

Open your browser at:

http://localhost:8888

Enter the token: changeme

To change the token, edit the JUPYTER_TOKEN variable in docker-compose.yml.

Build the Atom JAR

Open a terminal in JupyterLab (File > New > Terminal) and run:

cd ~/work/atom
mvn -f pom.xml package

Stop the environment

docker compose down

Using Atom

All commands below are run from a JupyterLab terminal (File > New > Terminal) inside ~/work/atom/dev/.

cd ~/work/atom/dev

Generate and Analyse Data

Generate a simulation with ZIT agents and redirect the output to a file:

java -cp target/atom-1.15.jar fr.cristal.smac.atom.Generate <nbAgents> <nbOrderbooks> <nbTurns> <nbDays> > myfile

Example — 10 agents, 1 orderbook, 1000 ticks, 1 day:

java -cp target/atom-1.15.jar fr.cristal.smac.atom.Generate 10 1 1000 1 > myfile

Extract price and agent data for analysis:

# Price series
grep '^Price' myfile > prices.csv

# Specific agent
grep '^Agent' myfile | grep ZIT1 > agent.csv

Plot prices in R (JupyterLab R notebook):

prices <- read.csv(file='prices.csv', sep=";", header=TRUE)
plot(prices$price, type='l', col='red')

Replay Files

Replay an existing order file through the Atom engine:

java -cp target/atom-1.15.jar fr.cristal.smac.atom.Replay <dataFile>

Example with a bundled sample file:

java -cp target/atom-1.15.jar fr.cristal.smac.atom.Replay ../dist/data/orderFileExample1

To verify consistency between a generated file and its replay (virtuous loop):

java -cp target/atom-1.15.jar fr.cristal.smac.atom.Generate 10 1 1000 1 > myfile1
java -cp target/atom-1.15.jar fr.cristal.smac.atom.Replay myfile1 > myfile2
diff myfile1 myfile2

Order File Format

Order lines follow the syntax (;-separated):

Order;<orderbook>;<agent>;<id>;<type>;<direction>;<price>;<qty>

Type	Description
L	Limit order
M	Market order
I	Iceberg order
C	Cancel order
U	Update order (quantity)
Direction: B (Bid) / A (Ask)

Market commands (lines starting with !):

Command	Effect
!P	Print orderbook state
!C	Switch to continuous fixing
!F	Switch to fix fixing
!K	Close market and reinit