Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .bumpversion.cfg
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 4.7.0rc1
current_version = 4.7.0rc2
commit = False
tag = False
parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)(rc(?P<build>\d+))?
Expand Down
54 changes: 54 additions & 0 deletions docs/css/markers.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
/**
* Styles for adding markers to headings that do not change the URL anchor.
* This works thanks to the mkdocs-material plugin.
*
* Example usage:
*
* ## Feature X {: .beta }
*
* ## Feature Y {: .deprecated }
*
*/

.experimental::after {
content: "EXPERIMENTAL";
background-color: #e91e63;
color: white;
padding: 2px 6px;
border-radius: 4px;
font-size: 0.7em;
margin-left: 10px;
font-weight: bold;
}

.beta::after {
content: "BETA";
background-color: #ffc107;
color: black;
padding: 2px 6px;
border-radius: 4px;
font-size: 0.8em;
margin-left: 10px;
}

.new::after {
content: "NEW";
background-color: #2196f3;
color: white;
padding: 2px 6px;
border-radius: 4px;
font-size: 0.7em;
margin-left: 10px;
font-weight: bold;
}

.deprecated::after {
content: "DEPRECATED";
background-color: #757575;
color: white;
padding: 2px 6px;
border-radius: 4px;
font-size: 0.7em;
margin-left: 10px;
font-weight: bold;
}
34 changes: 23 additions & 11 deletions docs/guides/tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,11 +112,11 @@ Even if the task does not have any form input, an entry will still need to be ma
}
```

## The schedule file (DEPRECATED)
> This section is deprecated and will be removed in version 5.0.0, please refer to the [new scheduling system](#the-schedule-api)
> below.
## The schedule file {: .deprecated }

> from `4.3.0` we switched from [schedule] package to [apscheduler] to allow schedules to be stored in the DB and schedule tasks from the API.
!!! Warning
As of [v4.7.0] this is deprecated, and it will be removed in v5.0.0.
Please use the [new scheduling system](#the-schedule-api) below.

The schedule file is essentially the crontab associated with the task.
Continuing with our previous example:
Expand Down Expand Up @@ -157,11 +157,17 @@ To keep things organized and consistent (similar to how workflows are handled),

## The schedule API

> from `4.3.0` we switched from [schedule] package to [apscheduler] to allow schedules to be stored in the DB and schedule tasks from the API.
!!! Info
In [v4.4.0] we switched from [schedule] package to [apscheduler] to allow schedules to be stored in the DB and retrieve schedule tasks from the API.
The apscheduler library has its own decorator to schedule tasks: `@scheduler.scheduled_job()` (from `orchestrator.schedules.scheduler`).
We therefore deprecated the old `@schedule` decorator (from `orchestrator.schedules.scheduling`) and made it forwards compatible.

In [v4.7.0] we deprecated `@scheduler.scheduled_job()` provided by [apscheduler] in favor of a more dynamic API based system described below.
Although we no longer support the `@scheduler.scheduled_job()` decorator, it is still available because it is part of [apscheduler].
Therefore, we do NOT recommend using it for new schedules. Because you will miss a Linker Table join between schedules and workflows/tasks.

Consult the [v4.7 upgrade guide] for more details.

> from `4.7.0` we deprecated `@scheduler.scheduled_job()` provided by [apscheduler] in favor of a more dynamic API based system.
> Although we do no longer support the `@scheduler.scheduled_job()` decorator, it is still available because it is part of [apscheduler].
> Therefore, we do NOT recommend using it for new schedules. Because you will miss a Linker Table join between schedules and workflows/tasks.


Schedules can be created, updated, and deleted via the REST API, and retrieved via the already existing GraphQL API. It
Expand Down Expand Up @@ -226,16 +232,18 @@ For detailed configuration options, see the [APScheduler scheduling docs].

The scheduler automatically loads any schedules that are imported before the scheduler starts.

> In previous versions, schedules needed to be explicitly listed in an ALL_SCHEDULERS variable.
> This is no longer required; ALL_SCHEDULERS is deprecated as of orchestrator-core 4.7.0 and will be removed in 5.0.0.
> Follow-up ticket to remove deprecated code: [#1276](https://github.com/workfloworchestrator/orchestrator-core/issues/1276)
!!! Info
In previous versions, schedules needed to be explicitly added in the `ALL_SCHEDULERS` variable.
This is no longer required; `ALL_SCHEDULERS` is deprecated as of orchestrator-core v4.7.0 and will be removed in v5.0.0.
Follow-up ticket to remove deprecated code: [#1276](https://github.com/workfloworchestrator/orchestrator-core/issues/1276)

## The scheduler

The scheduler is invoked via `python main.py scheduler`.
Try `--help` or review the [CLI docs][cli-docs] to learn more.

### Initial schedules

From version orchestrator-core >= `4.7.0`, the scheduler uses the database to store schedules instead of hard-coded schedule files.
Previous versions (orchestrator-core < `4.7.0` had hard-coded schedules. These can be ported to the new system by creating them via the API or CLI.
Run the following CLI command to import previously existing orchestrator-core schedules and change them if needed via the API.
Expand Down Expand Up @@ -300,6 +308,7 @@ if [ $status -ne 0 ]; then
fi
```

<!-- link definitions -->

[schedule]: https://pypi.org/project/schedule/
[apscheduler]: https://pypi.org/project/APScheduler/
Expand All @@ -313,3 +322,6 @@ fi
[trigger docs]: https://apscheduler.readthedocs.io/en/master/api.html#triggers
[registering-workflows]: ../../../getting-started/workflows#register-workflows
[cli-docs]: ../../../reference-docs/cli/#orchestrator.cli.scheduler.show_schedule
[v4.4.0]: https://github.com/workfloworchestrator/orchestrator-core/releases/tag/4.4.0
[v4.7.0]: https://github.com/workfloworchestrator/orchestrator-core/releases/tag/4.7.0
[v4.7 upgrade guide]: ../guides/upgrading/4.7.md
23 changes: 0 additions & 23 deletions docs/guides/upgrading/4.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,26 +71,3 @@ This will update the `target` for all workflows that are `SYSTEM` or `VALIDATE`

This is a breaking change, so you will need to test your workflows after making this change to ensure that they are
working as expected.

## Scheduling flows
In 4.4.0 we have introduced a new way to schedule workflows via the decorator [@scheduler.scheduled_job(...)](../tasks.md#the-schedule-file-deprecated).
In 4.7.0 we introduce a new way to schedule workflows via the [Scheduler API](../tasks.md#the-schedule-api).

The 4.7.0 migration will create the `apscheduler_jobs` table if missing, so upgrades from older versions still succeed, but the table will initially be empty.

### Who needs to take action?

**Users upgrading from 4.4–4.6:**
- Nothing special required; your scheduler data already exists.

**Users upgrading from <4.4 who used the old decorator:**
- Your schedules will not automatically reappear. Restore them by running:
-
```
python main.py scheduler load-initial-schedule
```
(More information: [Scheduler API](../tasks.md#the-schedule-api).)
Make sure your schedule definitions are imported so the scheduler can discover them.

**Users who never used the decorator:**
- No action required; the migration creates the table and you can add schedules normally.
163 changes: 163 additions & 0 deletions docs/guides/upgrading/4.7.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
from orchestrator import begin

# 4.7 Upgrade Guide

In this document we describe important changes to be aware of when upgrading from `orchestrator-core` v4.x to v4.7.

If you are not yet using v4.x, please consult the 4.0 upgrade guide.

## About 4.7

This release introduces a deprecation and a breaking change to a beta feature:

1. Scheduling tasks via decorator is **deprecated** in favor of using the API
2. [Workflow authorization callbacks] (Beta) **change** from `def` to `async def`

### Scheduling tasks via decorator is deprecated in favor of using the API

In [v4.4.0] we introduced the [apscheduler] library with a persistent jobstore.
The decorator [@scheduler.scheduled_job(...)](../tasks.md#the-schedule-file) allowed to schedule tasks from code.

In [v4.7.0] this has evolved to scheduling tasks via the [Scheduler API].
Since tasks scheduled from code cannot be related to their task definition in the DB, they cannot be managed through the API.
Therefore, using the decorator to schedule tasks is deprecated.
This release contains a DB migration which ensures you can update from any v4.x version to v4.7.0.

**Please check which of the 3 scenarios applies for you and which action is required.**

#### 1. I use the scheduler with only the default schedules

**Short answer**

If you want to keep using the default schedules, run `python main.py scheduler load-initial-schedule` from the CLI where you normally run your scheduler.

Otherwise, no action is required.

**Long answer**

As of v4.6.5, the default core schedules that are scheduled with the decorator were:

- [clean-tasks]: Runs the task `task_clean_up_tasks` every 6 hours
- [resume-workflows]: Runs the task `task_resume_workflows` task every hour
- [validate-products]: Runs the task `task_validate_products` every night (only if no previous run is incomplete)
- [subscriptions-validator]: Runs a validate workflow for each active subscription

In v4.7.0 the only remaining schedule is `validate-products`. (this will be removed in [#1250])

The other 3 schedules have been removed.
When you upgrade from anywhere between v4.4.0 - v4.6.x then the DB will still contain the scheduled jobs.
You can remove them from the `apscheduler_jobs` table manually, or simply wait for the scheduler to clean them up, which will produce these 3 messages:

```
[error ] Unable to restore job "resume-workflows" -- removing it [apscheduler.jobstores.default]
[error ] Unable to restore job "clean-tasks" -- removing it [apscheduler.jobstores.default]
[error ] Unable to restore job "subscriptions-validator" -- removing it [apscheduler.jobstores.default]
```

If you upgrade from an older v4.x version, this does not apply.

If you want to recreate these 3 schedules by using the API, we have provided a command to do so:

```
python main.py scheduler load-initial-schedule
```

You can also choose to only schedule a specific task, or change when they run.
For now this can only be done through the API; in the future this will be possible from the UI. ([orchestrator-ui-library#2215])

For more information: [Scheduler API].


#### 2. I use the scheduler with both the default schedules AND my own schedules

**Short answer**

Check the actions for scenario 1.

No further action is required as your own schedules will continue to work.

**Long answer**

Please check scenario 1 regarding the default schedules.

For your own schedules, you may remove their definitions in code and schedule them using the [Schedule API].
However, this is not yet a requirement as the decorator-scheduled jobs can be used alongside API-scheduled jobs.
This will only be a requirement for orchestrator-core v5.0.0.

You can determine how jobs have been scheduled by running the [show-schedule] CLI command and inspecting the `source` column.
For example, an excerpt of the SURF schedule:

```
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┓
┃ id ┃ name ┃ source ┃ next run time ┃ trigger ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━┩
│ 6db5c270-8239-455c-97e4-14a6a865c68d │ Task Resume Workflows │ API │ 2025-12-11 11:07:42+00:00 │ interval[1:00:00] │
│ import-crm-to-ims │ Import CRM data to IMS │ decorator │ 2025-12-11 12:00:00+00:00 │ cron │
│ 70c0b5ce-88f5-48fb-ba5f-d19422628bef │ Task Clean Up Tasks │ API │ 2025-12-11 16:07:42+00:00 │ interval[6:00:00] │
│ import-customers-from-crm │ Import customers from CRM │ decorator │ 2025-12-11 19:00:00+00:00 │ cron │
```

#### 3. I do not use the scheduler

No action required!

### Workflow authorization callbacks change from `def` to `async def`

In [v4.7.0] the workflow authorization callbacks change from synchronous to asynchronous.
This was done for consistency with API authorization callback functions which are also async.
Additionally, it is always possible to run a sync function in an async context, but the other way around is much harder.

While this is a breaking change, we decided to add it in this minor release since workflow authorization is currently still a beta feature.

#### Example

Given a synchronous callback, like this:

```python
from oauth2_lib.fastapi import OIDCUserModel
from orchestrator.workflow import workflow, StepList, begin

def authorize_workflow_access(user: OIDCUserModel) -> bool:
...

@workflow("My workflow", authorize_callback=authorize_workflow_access)
def my_workflow() -> StepList:
return begin >> ...
```

The required change is to make the callback look like this:

```python
async def authorize_workflow_access(user: OIDCUserModel) -> bool:
...
```

!!! Info
**Important**: check your callback for synchronous I/O calls.
These block the eventloop, which has a negative impact on performance of the API.

For example, if your callback performs a `requests.get()` call or a sqlalchemy query, this will block the eventloop.
You can resolve this either using an asynchronous alternative that can be `await`ed, or by using [to_thread] to run the synchronous call in a thread.
Running in a thread is not as optimal as truly async code, but it's still a lot better than blocking the loop.

For more information see [Workflow authorization callbacks].

<!-- link definitions -->

[v4.4.0]: https://github.com/workfloworchestrator/orchestrator-core/releases/tag/4.4.0
[v4.7.0]: https://github.com/workfloworchestrator/orchestrator-core/releases/tag/4.7.0
[Workflow authorization callbacks]: ../../reference-docs/auth-backend-and-frontend.md#authorization-and-workflows
[Scheduler API]: ../tasks.md#the-schedule-api
[apscheduler]: https://pypi.org/project/APScheduler/
[show-schedule]: ../../reference-docs/cli.md#scheduler
[load-initial-schedule]: ../../reference-docs/cli.md#scheduler

[clean-tasks]: https://github.com/workfloworchestrator/orchestrator-core/blob/4.6.5/orchestrator/schedules/task_vacuum.py
[resume-workflows]: https://github.com/workfloworchestrator/rchestrator-core/blob/4.6.5/orchestrator/schedules/resume_workflows.py
[validate-products]: https://github.com/workfloworchestrator/orchestrator-core/blob/4.6.5/orchestrator/schedules/validate_products.py
[subscriptions-validator]: https://github.com/workfloworchestrator/orchestrator-core/blob/4.6.5/orchestrator/schedules/validate_subscriptions.py

[#1250]: https://github.com/workfloworchestrator/orchestrator-core/issues/1250
[orchestrator-ui-library#2215]: https://github.com/workfloworchestrator/orchestrator-ui-library/issues/2215

[to_thread]: https://docs.python.org/3/library/asyncio-task.html#asyncio.to_thread
Loading