Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,14 @@ We actively welcome your pull requests.
* Dynabench uses pre-commit hooks to ensure style consistency and prevent common mistakes. Enable it by:

```sh
pip install pre-commit && pre-commit install
# Install pre-commit tool
uv tool install pre-commit --with pre-commit-uv

# Set up git hooks for this repository
pre-commit install
```

After this pre-commit hooks will be run before every commit.
After this, pre-commit hooks will be run before every commit.

* Read the [editorconfig](https://github.com/mlcommons/dynabench/blob/main/.editorconfig) file to understand the exact coding style preferences.

Expand Down
16 changes: 0 additions & 16 deletions Dockerfile

This file was deleted.

37 changes: 30 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,36 @@

Dynabench is a research platform for dynamic data collection and benchmarking.

Documentation:
1. [Platform overview](https://github.com/mlcommons/dynabench/blob/main/docs/overview.md)
2. [Developer guide](docs/setup_guide.md)
3. [Contributing guidelines](https://github.com/mlcommons/dynabench/blob/main/CONTRIBUTING.md)
4. [Task owners manual](https://github.com/mlcommons/dynabench/blob/main/docs/owners.md)
5. [Evaluation Server Developer Guide](https://github.com/mlcommons/dynabench/blob/main/docs/evaluation.md) [Possibly Outdated]
6. [Frontend Developer Guide](https://github.com/mlcommons/dynabench/blob/main/docs/frontend.md) [Possibly Outdated]
Dynabench enables researchers to:
- Create dynamic benchmarks that evolve as models improve
- Collect adversarial examples through human-and-model-in-the-loop interaction
- Evaluate AI models on challenging, naturally occurring data
- Track model performance over time as benchmarks adapt

## Getting Started

### For Users

Learn about the platform and create tasks:
- [Platform Overview](docs/overview.md) - Platform architecture and features
- [Task Owners Manual](docs/owners.md) - Creating and managing tasks
- [Adding Tasks](docs/adding_task.md) - Guide for adding new tasks

### For Developers

Set up your local development environment:
- [Developer Setup Guide](docs/setup_guide.md) - Complete development environment setup
- [Database Setup](docs/database.md) - MySQL installation alternatives
- [Email Configuration](docs/email.md) - Email system setup
- [Evaluation Server Guide](docs/evaluation.md) - Evaluation server setup (possibly outdated)
- [Frontend Developer Guide](docs/frontend.md) - Frontend development (possibly outdated)

### For Contributors

Contribute to the Dynabench project:

1. Fork and clone the repository
2. Read the documentation [Contributing Guidelines](CONTRIBUTING.md) to set up the development workflow and pre-commit hooks

## License

Expand Down
3 changes: 2 additions & 1 deletion api/.env.example
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
#Database
DB_HOST=127.0.0.1
# Use host.docker.internal for Docker setup, or 127.0.0.1 for local setup
DB_HOST=host.docker.internal
DB_PORT=3306
DB_NAME=dynabench
DB_USER=dynabench
Expand Down
1 change: 0 additions & 1 deletion api/.python-version

This file was deleted.

29 changes: 29 additions & 0 deletions api/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Dynabench Legacy API (Bottle)
FROM python:3.13-slim

# Install system dependencies
RUN apt-get update && apt-get install -y \
git \
build-essential

# Install uv
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv

# Set working directory
WORKDIR /app

# Copy project files
COPY pyproject.toml uv.lock ./

# Install dependencies
RUN uv sync --frozen --no-dev

# Copy application code
COPY . .

# Expose port
ENV PORT=8081
EXPOSE 8081

# Run API server
CMD ["uv", "run", "python", "server.py", "dev"]
2 changes: 1 addition & 1 deletion api/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name = "api"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.13"
requires-python = ">=3.13,<3.14"
dependencies = [
"aiohttp>=3.13.2",
"augly>=1.0.0",
Expand Down
25 changes: 0 additions & 25 deletions api/requirements.txt

This file was deleted.

2,871 changes: 1,068 additions & 1,803 deletions api/uv.lock

Large diffs are not rendered by default.

3 changes: 2 additions & 1 deletion backend/.env.example
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
#Database
DB_HOST=127.0.0.1
# Use host.docker.internal for Docker setup, or 127.0.0.1 for local setup
DB_HOST=host.docker.internal
DB_PORT=3306
DB_NAME=dynabench
DB_USER=dynabench
Expand Down
1 change: 0 additions & 1 deletion backend/.python-version

This file was deleted.

29 changes: 29 additions & 0 deletions backend/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Dynabench Modern Backend (FastAPI)
FROM python:3.13-slim

# Install system dependencies
RUN apt-get update && apt-get install -y \
git \
build-essential

# Install uv
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv

# Set working directory
WORKDIR /app

# Copy project files
COPY pyproject.toml uv.lock ./

# Install dependencies
RUN uv sync --frozen --no-dev

# Copy application code
COPY . .

# Expose port
ENV PORT=8000
EXPOSE 8000

# Run FastAPI server
CMD ["uv", "run", "uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
2 changes: 1 addition & 1 deletion backend/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name = "backend"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.13"
requires-python = ">=3.13,<3.14"
dependencies = [
"aleph-alpha-client>=11.4.0",
"anthropic>=0.72.1",
Expand Down
42 changes: 0 additions & 42 deletions backend/requirements.txt

This file was deleted.

Loading
Loading