Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 8 additions & 15 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,32 +12,25 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install uv
uses: astral-sh/setup-uv@v4

- name: Install Poetry
run: |
curl -sSL https://install.python-poetry.org | python3 -
poetry self update 1.8.5

- name: Verify Poetry installation
run: poetry --version
- name: Set up Python ${{ matrix.python-version }}
run: uv python install ${{ matrix.python-version }}

- name: Install dependencies
run: poetry install --with dev
run: uv sync --group dev

- name: Run tests
run: poetry run pytest -s -vv --cov-report=xml --cov-branch --cov=weni_cli
run: uv run pytest -s -vv --cov-report=xml --cov-branch --cov=weni_cli

- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v5
with:
token: ${{ secrets.CODECOV_TOKEN }}

- name: Run linters
run: poetry run flake8 weni_cli
run: uv run flake8 weni_cli

- name: Run mypy
run: poetry run mypy .
run: uv run mypy .
26 changes: 9 additions & 17 deletions .github/workflows/publish.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,27 +13,19 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Python 3.12
uses: actions/setup-python@v5
with:
python-version: 3.12
- name: Install uv
uses: astral-sh/setup-uv@v4

- name: Install Poetry
run: |
curl -sSL https://install.python-poetry.org | python3 -
poetry self update 1.8.5

- name: Verify Poetry installation
run: poetry --version
- name: Set up Python 3.12
run: uv python install 3.12

- name: Build package
run: poetry build
run: uv build

- name: Publish to pypi
shell: bash
run: |
poetry config pypi-token.pypi ${{ secrets.PYPI_TOKEN }}
poetry publish
run: uv publish
env:
UV_PUBLISH_TOKEN: ${{ secrets.PYPI_TOKEN }}

- name: Create GitHub release
uses: softprops/action-gh-release@v1
Expand All @@ -43,4 +35,4 @@ jobs:
body: |
Release ${{ github.ref }}
draft: false
prerelease: false
prerelease: false
12 changes: 4 additions & 8 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -96,12 +96,10 @@ ipython_config.py
# install all needed dependencies.
#Pipfile.lock

# poetry
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
#poetry.lock
# uv
# uv.lock should be committed to version control for reproducible builds.
# https://docs.astral.sh/uv/concepts/projects/layout/#the-lockfile
#uv.lock

# pdm
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
Expand Down Expand Up @@ -162,8 +160,6 @@ cython_debug/
#.idea/

### Python Patch ###
# Poetry local configuration file - https://python-poetry.org/docs/configuration/#local-configuration
poetry.toml

# ruff
.ruff_cache/
Expand Down
40 changes: 37 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ A command-line interface (CLI) tool to manage and interact with projects on the
## Requirements

- Python >= 3.10
- Poetry >= 1.8.5
- [uv](https://docs.astral.sh/uv/) (recommended) or pip

## Installation

Expand All @@ -22,6 +22,11 @@ You can install the CLI directly using pip:
pip install weni-cli
```

### Install via uv
```bash
uv tool install weni-cli
```

### Manual Installation

1. Clone the repository:
Expand All @@ -32,8 +37,7 @@ pip install weni-cli

2. Install dependencies and make the CLI executable:
```bash
poetry shell
poetry install
uv sync
```

## Quick Start [Step by Step]
Expand Down Expand Up @@ -147,6 +151,7 @@ The agent will now be able to:
- **Select project**: Set the project to be used by the CLI.
- **Current project**: Display information about the currently selected project.
- **Push project definition**: Upload the agents definition file to the selected project.
- **Agent evaluation**: Initialize and run agent evaluations with `weni eval`.

## Usage

Expand Down Expand Up @@ -183,6 +188,35 @@ weni project push <definition_file.yaml>
```
Replace `<definition_file.yaml>` with the path to your YAML definition file.

### 6. Initialize an evaluation plan
Create an `agenteval.yml` test plan in the current directory:
```bash
weni eval init
```

You can also choose a directory:
```bash
weni eval init --plan-dir <path_to_directory>
```

### 7. Run agent evaluation
Run all tests from `agenteval.yml`:
```bash
weni eval run
```

Common options:
```bash
# Run only specific tests
weni eval run --filter "greeting,checkout_flow"

# Verbose output with detailed reasoning
weni eval run --verbose

# Custom plan directory
weni eval run --plan-dir <plan_directory>
```

## Agent Definition File Structure

Below is an example of a valid agent definition file structure:
Expand Down
16 changes: 8 additions & 8 deletions docs/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Thank you for your interest in contributing to Weni CLI! This guide will help yo
### Prerequisites

- Python >= 3.12
- Poetry >= 1.8.5
- [uv](https://docs.astral.sh/uv/)
- Git

### Setting Up Development Environment
Expand All @@ -22,7 +22,7 @@ Thank you for your interest in contributing to Weni CLI! This guide will help yo

3. Install dependencies:
```bash
poetry install
uv sync --group dev
```

4. Create a new branch:
Expand Down Expand Up @@ -50,12 +50,12 @@ Thank you for your interest in contributing to Weni CLI! This guide will help yo

2. Run tests:
```bash
poetry run pytest
uv run pytest
```

3. Check coverage:
```bash
poetry run pytest --cov
uv run pytest --cov
```

### Documentation
Expand Down Expand Up @@ -149,16 +149,16 @@ List documentation updates

```bash
# Format code
poetry run black .
uv run black .

# Run linter
poetry run flake8
uv run flake8

# Run tests
poetry run pytest
uv run pytest

# Build documentation
poetry run mkdocs serve
uv run --group docs mkdocs serve
```

## Release Process
Expand Down
Loading
Loading