Skip to content

fix: ensure CQL learner transitions from behavior cloning correctly #355

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
natinew77-creator wants to merge 2 commits into
base: master
Choose a base branch
from
Open

fix: ensure CQL learner transitions from behavior cloning correctly #355

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
85c009d
docs: clarify installation instructions for running examples
natinew77-creator Dec 10, 2025
b744727
Fix CQL learner always executing behavior cloning (#305)
natinew77-creator Dec 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 28 additions & 13 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,11 @@ in our [technical report][Paper].

## Installation

> ⚠️ **Important Note**: The PyPI package (`dm-acme`) may be significantly out of
> date compared to this repository. If you want to run the examples or use the
> latest agents, we strongly recommend **installing from source** (see option 2
> below).

To get up and running quickly just follow the steps below:

1. While you can install Acme in your standard python environment, we
Expand All @@ -63,31 +68,41 @@ To get up and running quickly just follow the steps below:
pip install --upgrade pip setuptools wheel
```

1. While the core `dm-acme` library can be pip installed directly, the set of
2. **Recommended: Installing from source** (required for running examples):

The examples in this repository require the latest code which may not be
available in the PyPI package. Install from source by cloning this repository:

```bash
git clone https://github.com/google-deepmind/acme.git
cd acme
pip install -e .[jax,tf,envs]
```

This installs Acme in "editable" mode, allowing you to run all examples and
use the latest agents.

3. **Alternative: Installing from PyPI** (may be outdated):

While the core `dm-acme` library can be pip installed directly, the set of
dependencies included for installation is minimal. In particular, to run any
of the included agents you will also need either [JAX] or [TensorFlow]
depending on the agent. As a result we recommend installing these components
as well, i.e.
depending on the agent:

```bash
pip install dm-acme[jax,tf]
```

1. Finally, to install a few example environments (including [gym],
[dm_control], and [bsuite]):
To install a few example environments (including [gym], [dm_control], and
[bsuite]):

```bash
pip install dm-acme[envs]
```

1. **Installing from github**: if you're interested in running the
bleeding-edge version of Acme, you can do so by cloning the Acme GitHub
repository and then executing following command from the main directory
(where `setup.py` is located):

```bash
pip install .[jax,tf,testing,envs]
```
> Note: If you encounter import errors when running examples after pip
> install (e.g., `cannot import name 'sac' from 'acme.agents.jax'`), please
> install from source using option 2 above.

## Citing Acme

Expand Down
6 changes: 2 additions & 4 deletions acme/agents/jax/cql/learning.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -446,10 +446,8 @@ def step(self):
transitions = next(self._demonstrations)

counts = self._counter.get_counts()
if 'learner_steps' not in counts:
cur_step = 0
else:
cur_step = counts['learner_steps']
steps_key = self._counter.get_steps_key()
cur_step = counts.get(steps_key, 0)
in_initial_bc_iters = cur_step < self._num_bc_iters

if in_initial_bc_iters:
Expand Down