Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# .env.example
PASSWORD_SECRET_KEY=your-secret-key-here-generate-with-openssl-rand-hex-32
MEDIA_HOST=http://localhost:8000
DATABASE_URL=sqlite:///data/labelu.sqlite
8 changes: 4 additions & 4 deletions .github/workflows/claude-fix-issue.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ jobs:
with:
python-version: '3.11'

- name: Install Poetry
run: pip install poetry==1.2.2
- name: Install uv
uses: astral-sh/setup-uv@v4

- name: Install Claude Code
run: npm install -g @anthropic-ai/claude-code
Expand All @@ -50,13 +50,13 @@ jobs:
1. 仔细阅读 CLAUDE.md 了解项目架构
2. 分析问题根因
3. 按照项目的分层架构(adapter/application/domain/common)修复代码
4. 运行测试确保通过: poetry run pytest
4. 运行测试确保通过: uv run --group test pytest
5. 如果修复涉及新的行为,请添加相应的测试用例
6. 用 git add 和 git commit 提交修改,commit message 遵循 conventional commits 格式
PROMPT
cat /tmp/claude-prompt.txt | claude --print \
--model claude-opus-4-6 \
--allowedTools "Bash(poetry*),Bash(git*),Read,Write,Edit,Glob,Grep"
--allowedTools "Bash(uv*),Bash(git*),Read,Write,Edit,Glob,Grep"

- name: Push and create PR
env:
Expand Down
20 changes: 9 additions & 11 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@ jobs:
fail-fast: false
matrix:
python-version: [3.11]
poetry-version: ['1.2.2']
os: [ubuntu-24.04]
node-version: [20.8.1]
runs-on: ${{ matrix.os }}
Expand All @@ -59,6 +58,9 @@ jobs:
with:
python-version: ${{ matrix.python-version }}

- name: Install uv
uses: astral-sh/setup-uv@v4

- name: Download frontend
env:
CURRENT_BRANCH: ${{ github.ref_name }}
Expand Down Expand Up @@ -99,7 +101,7 @@ jobs:
);
const backendInfo = {
version: process.env.NEXT_VERSION,
name: projectInfo.tool.poetry.name || 'LabelU',
name: (projectInfo.project && projectInfo.project.name) || (projectInfo.tool && projectInfo.tool.poetry && projectInfo.tool.poetry.name) || 'LabelU',
build_time: new Date().toISOString(),
commit: process.env.GITHUB_SHA,
};
Expand All @@ -122,15 +124,11 @@ jobs:
process.exit(1);
}

- uses: abatilo/actions-poetry@v2
with:
poetry-version: ${{ matrix.poetry-version }}

- name: Install dependencies
run: poetry install --without dev
run: uv sync --frozen --group test

- name: Run tests
run: poetry run pytest --cov=./ --cov-report=xml
run: uv run --group test pytest --cov=./ --cov-report=xml

- name: Upload coverage to Codecov
uses: codecov/codecov-action@v3
Expand All @@ -148,10 +146,10 @@ jobs:
# main or alpha
if: ${{ github.ref_name == 'main' || github.ref_name == 'alpha' }}
env:
PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }}
UV_PUBLISH_TOKEN: ${{ secrets.PYPI_TOKEN }}
run: |
poetry config pypi-token.pypi $PYPI_TOKEN
poetry publish --build --skip-existing
uv build
uv publish --skip-existing

- name: Semantic Release
run: |
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ __pycache__/
# Distribution / packaging
.Python
build/
data
develop-eggs/
dist/
downloads/
Expand Down
85 changes: 85 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# LabelU

## Project Overview

LabelU is a multimodal data annotation platform (image/video/audio) built with FastAPI + SQLAlchemy. It provides task management, file attachment handling, sample annotation, and real-time collaboration via WebSocket.

## Architecture

Layered architecture following Domain-Driven Design principles:

- **Domain Layer**: `labelu/internal/domain/models/` - SQLAlchemy ORM models (User, Task, TaskSample, TaskAttachment, TaskPreAnnotation, TaskCollaborator, TaskSampleUpdater)
- **Application Layer**: `labelu/internal/application/` - Business logic split into:
- `command/` - Request/input Pydantic models
- `response/` - Response Pydantic models
- `service/` - Business logic orchestration (uses `db.begin()` for transactions)
- **Adapter Layer**: `labelu/internal/adapter/` - External interfaces:
- `routers/` - FastAPI HTTP route handlers
- `persistence/` - CRUD database operations
- `ws/` - WebSocket handlers
- **Common**: `labelu/internal/common/` - Shared utilities:
- `config.py` - Pydantic BaseSettings configuration
- `db.py` - SQLAlchemy engine, session, Base
- `security.py` - JWT token creation, password hashing
- `error_code.py` - Error codes enum + exception handlers
- `converter.py`, `xml_converter.py`, `tf_record_converter.py` - Export format converters
- `websocket.py` - WebSocket connection manager
- **Dependencies**: `labelu/internal/dependencies/user.py` - FastAPI dependency injection (auth)
- **Middleware**: `labelu/internal/middleware/` - Content-type and tracing middleware

## Development Commands

- **Run server**: `labelu` or `python -m labelu.main`
- **Run server with options**: `labelu --host 0.0.0.0 --port 8000 --media-host http://localhost:8000`
- **Run tests**: `python -m pytest labelu/tests/ -v`
- **Run single test**: `python -m pytest labelu/tests/path/test_file.py::test_name -v`
- **Format code**: `black labelu/`
- **Lint**: `flake8 labelu/`
- **DB migration**: `alembic -c labelu/alembic_labelu/alembic.ini upgrade head`
- **Install**: `poetry install`

## Key Files

- Entry point: `labelu/main.py`
- Config: `labelu/internal/common/config.py`
- Database: `labelu/internal/common/db.py`
- Auth: `labelu/internal/common/security.py` + `labelu/internal/dependencies/user.py`
- Error handling: `labelu/internal/common/error_code.py`
- Alembic migrations: `labelu/alembic_labelu/versions/`
- Tests: `labelu/tests/`
- Version: `labelu/version.py`

## Environment Variables

- `PASSWORD_SECRET_KEY`: JWT signing key (required in production)
- `DATABASE_URL`: DB connection string (default: `sqlite:///<data_dir>/labelu.sqlite`)
- `MEDIA_HOST`: Media file serving host URL (default: `http://localhost:8000`)

Data directory is determined by `appdirs.user_data_dir("labelu")`.

## Tech Stack

- Python 3.11, FastAPI ^0.90, SQLAlchemy ^1.4 (1.x style), Pydantic v1
- Auth: python-jose (JWT), passlib + bcrypt
- DB: SQLite (default) or MySQL (optional)
- Migrations: Alembic
- Package manager: Poetry
- WebSocket: websockets ^10

## Testing

- Tests in `labelu/tests/` using pytest
- Test DB: SQLite file `./test.db` (not in-memory)
- Test fixtures in `labelu/tests/conftest.py`
- Test user: `test@example.com` / `test@123`
- Tables (except `user`) are cleaned between tests via `autouse` fixture
- `scope="module"` for `client` and `testuser_token_headers` fixtures

## Conventions

- Routers delegate to services; services use CRUD persistence layer
- All responses wrapped in `OkResp[T]` or `OkRespWithMeta[T]` (GenericModel-based)
- Custom exceptions: `LabelUException` with `ErrorCode` enum
- Soft delete pattern: `deleted_at` timestamp on models
- Transaction management: `with db.begin():` blocks in service layer
- Session uses `autocommit=True` mode (legacy SQLAlchemy pattern)
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ FROM python:3.11-slim
WORKDIR /labelu

RUN pip3 install -i https://test.pypi.org/pypi/ --extra-index-url https://pypi.org/simple -U labelu
ENV MEDIA_HOST http://labelu.shlab.tech
ENV MEDIA_HOST=http://localhost:8000

EXPOSE 8000

Expand Down
26 changes: 14 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,26 +104,28 @@ DATABASE_URL=mysql://<username>:<password>@<host>/<your dbname> labelu migrate_t
### Local development

```bash
# Download and Install miniconda
# https://docs.conda.io/en/latest/miniconda.html
# Install uv
# https://docs.astral.sh/uv/getting-started/installation/

# Create virtual environment(python = 3.11)
conda create -n labelu python=3.11

# Activate virtual environment
conda activate labelu
# Clone the repository
git clone https://github.com/opendatalab/labelU.git
cd labelU

# Install peotry
# https://python-poetry.org/docs/#installing-with-the-official-installer
# Create virtual environment and install all dependencies (Python >= 3.11)
uv sync

# Install all package dependencies
poetry install
# Copy the example environment file and configure it
cp .env.example .env
# Edit .env and set your values:
# PASSWORD_SECRET_KEY - JWT secret key, generate with: openssl rand -hex 32
# MEDIA_HOST - Media server URL (default: http://localhost:8000)
# DATABASE_URL - Database connection URL (default: sqlite:///data/labelu.sqlite)

# Download the frontend statics from labelu-kit repo
sh ./scripts/resolve_frontend.sh true

# Start labelu, server: http://localhost:8000
uvicorn labelu.main:app --reload
uv run uvicorn labelu.main:app --reload
```


Expand Down
26 changes: 14 additions & 12 deletions README_zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,26 +101,28 @@ DATABASE_URL=mysql://<username>:<password>@<host>/<your dbname> labelu migrate_t
### 本地开发

```bash
# 安装miniconda
# https://docs.conda.io/en/latest/miniconda.html
# 安装 uv
# https://docs.astral.sh/uv/getting-started/installation/

# 创建虚拟环境(python = 3.11)
conda create -n labelu python=3.11

# 激活虚拟环境
conda activate labelu
# 克隆项目
git clone https://github.com/opendatalab/labelU.git
cd labelU

# 安装 peotry
# https://python-poetry.org/docs/#installing-with-the-official-installer
# 创建虚拟环境并安装所有依赖包 (Python >= 3.11)
uv sync

# 安装所有依赖包
poetry install
# 复制环境变量示例文件并根据实际情况修改
cp .env.example .env
# 编辑 .env 并设置以下变量:
# PASSWORD_SECRET_KEY - JWT 密钥,可通过以下命令生成:openssl rand -hex 32
# MEDIA_HOST - 媒体服务器地址(默认:http://localhost:8000)
# DATABASE_URL - 数据库连接地址(默认:sqlite:///data/labelu.sqlite)

# 从 LabelU-kit 下载前端资源
sh ./scripts/resolve_frontend.sh true

# 启动labelu, 默认访问地址: http://localhost:8000
uvicorn labelu.main:app --reload
uv run uvicorn labelu.main:app --reload
```

## 快速上手
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
"""add export_job table

Revision ID: a1b2c3d4e5f6
Revises: 2eb983c9a254
Create Date: 2026-03-19 10:00:00.000000

"""
from alembic import op
import sqlalchemy as sa

# revision identifiers, used by Alembic.
revision = 'a1b2c3d4e5f6'
down_revision = '2eb983c9a254'
branch_labels = None
depends_on = None


def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
tables = inspector.get_table_names()

if 'export_job' not in tables:
op.create_table(
'export_job',
sa.Column('id', sa.Integer(), autoincrement=True, nullable=False),
sa.Column('task_id', sa.Integer(), nullable=True),
sa.Column('created_by', sa.Integer(), nullable=True),
sa.Column('export_type', sa.String(length=32), nullable=True),
sa.Column('status', sa.String(length=32), nullable=True),
sa.Column('sample_count', sa.Integer(), nullable=True),
sa.Column('processed_count', sa.Integer(), nullable=True),
sa.Column('file_path', sa.Text(), nullable=True),
sa.Column('error_message', sa.Text(), nullable=True),
sa.Column('sample_ids', sa.Text(), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=True),
sa.ForeignKeyConstraint(['task_id'], ['task.id']),
sa.ForeignKeyConstraint(['created_by'], ['user.id']),
sa.PrimaryKeyConstraint('id'),
)
op.create_index('ix_export_job_id', 'export_job', ['id'])
op.create_index('ix_export_job_task_id', 'export_job', ['task_id'])


def downgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
tables = inspector.get_table_names()

if 'export_job' in tables:
op.drop_index('ix_export_job_task_id', table_name='export_job')
op.drop_index('ix_export_job_id', table_name='export_job')
op.drop_table('export_job')
31 changes: 31 additions & 0 deletions labelu/internal/adapter/persistence/crud_export_job.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
import json
from typing import List, Optional
from sqlalchemy.orm import Session
from labelu.internal.domain.models.export_job import ExportJob, ExportStatus


def create(db: Session, task_id: int, user_id: int, export_type: str, sample_ids: List[int]) -> ExportJob:
job = ExportJob(
task_id=task_id,
created_by=user_id,
export_type=export_type,
sample_count=len(sample_ids),
sample_ids=json.dumps(sample_ids),
)
db.add(job)
db.flush()
db.refresh(job)
return job


def get(db: Session, job_id: int) -> Optional[ExportJob]:
return db.query(ExportJob).filter(ExportJob.id == job_id).first()


def update_status(db: Session, job: ExportJob, status: str, **kwargs) -> ExportJob:
job.status = status
for k, v in kwargs.items():
setattr(job, k, v)
db.flush()
db.refresh(job)
return job
15 changes: 7 additions & 8 deletions labelu/internal/adapter/persistence/crud_sample.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,8 @@ def list_by(
query = db.query(TaskSample).filter(*query_filter)

# case when for state enum
whens = {state: index for index, state in enumerate(SampleState)}
sort_logic = case(value=TaskSample.state, whens=whens).label(TaskSample.state.key)
whens = {state.value: index for index, state in enumerate(SampleState)}
sort_logic = case(whens, value=TaskSample.state).label(TaskSample.state.key)

if sorting:
sort_strings = sorting.split(",")
Expand Down Expand Up @@ -74,12 +74,11 @@ def get(db: Session, sample_id: int) -> TaskSample:
)


def get_by_ids(db: Session, sample_ids: List[int]) -> List[TaskSample]:
return (
db.query(TaskSample)
.filter(TaskSample.id.in_(sample_ids), TaskSample.deleted_at == None)
.all()
)
def get_by_ids(db: Session, sample_ids: List[int], task_id: Union[int, None] = None) -> List[TaskSample]:
query_filter = [TaskSample.id.in_(sample_ids), TaskSample.deleted_at == None]
if task_id is not None:
query_filter.append(TaskSample.task_id == task_id)
return db.query(TaskSample).filter(*query_filter).all()


def update(db: Session, db_obj: TaskSample, obj_in: Dict[str, Any]) -> TaskSample:
Expand Down
Loading