Skip to content
This repository was archived by the owner on Mar 29, 2022. It is now read-only.

Commit 4242cea

Browse files
wjallenwjallen
committed
updates to abaco_custom_reactor page
1 parent 59313e2 commit 4242cea

File tree

1 file changed

+199
-75
lines changed

1 file changed

+199
-75
lines changed

‎docs/04.abaco_custom_reactor.md‎

Lines changed: 199 additions & 75 deletions
Original file line numberDiff line numberDiff line change
@@ -1,116 +1,240 @@
11
---
22
layout: page
3-
title: Abaco Command Line Interface
3+
title: Create a Custom Reactor
44
tagline:
55
---
66

7-
The Abaco CLI is a command line interface for working with the Abaco (Actor BAsed COntainers) API. Abaco containers can react to events like the successful upload of a file, or the completion of an agave job. Abaco helps with automation by triggering "reactions" do a defined set of events. Abaco reactors are designed to react quickly. If you are deciding between using an agave app or an Abaco reactor; if the computation will take place in under 1 minute, use a Abaco reactor, if the computation will need more than 1 minute to process, you should probably create an agave app.
7+
In this guide, we will demonstrate how to create an Abaco reactor from scratch.
8+
Abaco reactors react to events such as the successful upload of a file or the
9+
completion of an Agave job. Abaco reactors enable automation by automatically,
10+
rather than manually, triggering in response to defined events. If you are
11+
deciding between building an Agave app or an Abaco reactor consider the
12+
following rule of thumb: If the computation requires less than one minute and
13+
minimal resources, use an Abaco reactor. If the computation requires more than
14+
one minute and / or considerable resources, use an Agave app.
15+
16+
For an introduction to the Abaco CLI, please see
17+
[Abaco CLI Basics](04.abaco_cli.md).
18+
819

920
<br>
10-
#### Creating an Abaco container from Scrath
21+
#### Requirements
1122

12-
The Abaco cli comes bundled with the SD2E agave cli. To make sure you have the most current version, run:
23+
The Abaco command line tools needed to create a custom reactor are included with
24+
the [SD2E CLI](01.install_cli.md). To make sure you have the most current
25+
version, run:
1326
```
14-
sd2e info --upgrade
27+
% sd2e info --upgrade
1528
```
1629

17-
To use Abaco, you'll need the following installed on your machine:
30+
The following are also requirements:
1831

19-
1. [Docker CE](https://www.docker.com/community-edition) and you'll need to create a Docker Hub account
20-
2. [jq](https://stedolan.github.io/jq/)
21-
3. getopts (installed by default on macOS High Sierra)
32+
1. [Docker CE](https://www.docker.com/community-edition)
33+
2. [Docker Hub Account](https://hub.docker.com/)
34+
3. [jq](https://stedolan.github.io/jq/)
35+
4. getopts (installed by default on macOS High Sierra)
2236

2337

2438
<br>
25-
#### Create skeleton for Abaco Reactor
39+
#### Initialize a new reactor
2640

27-
To create an Abaco reactor, create a directory where you'd like to organize your reactors. Abaco reactors can be created from an existing docker container using:
41+
To initialize a new Abaco reactor in the simplest form, run the following
42+
command:
2843
```
29-
abaco init -n tutorial jturcino/abaco-trial:latest
44+
% abaco init -n my-project
3045
```
31-
or more generally:
46+
47+
After executing this command, you will see a new directory with the project
48+
(reactor) name. This directory contains all the required files to deploy an
49+
Abaco reactor:
3250
```
33-
abaco init -n Reactor_Name Docker_org/Image_name:version
51+
my-project/
52+
├── Dockerfile
53+
├── TEMPLATE.md
54+
├── config.yml
55+
├── message.json
56+
├── reactor.py
57+
├── reactor.rc
58+
├── requirements.txt
59+
└── secrets.json.sample
3460
```
3561

36-
After executing this command you will see a new directory get created with your Reactor name. This directory contains all the required files to deploy an abaco reactor:
62+
To build a functional reactor, modify the files as follows:
63+
64+
<br>
65+
##### `Dockerfile`
66+
The only mandatory line is the `FROM` statement, which, by default, is:
3767
```
38-
Dockerfile
39-
TEMPLATE.md
40-
config.yml
41-
message.json
42-
reactor.py
43-
reactor.rc
44-
requirements.txt
45-
secrets.json.sample
68+
FROM sd2e/reactor-base:python2
4669
```
70+
This docker image comes pre-loaded with the python libraries that contain the
71+
reactor functions. If you need to add any additional code, dependencies, etc.,
72+
to your reactor, this is the time for that. Other possible base images are
73+
described on the [base images page](06.base_images.md).
4774

48-
1. Dockerfile
49-
* All that's mandatory is the `FROM` statement. If you need to manually add
50-
additional code, dependencies, etc. to your Reactor, this is the place for
51-
doing that.
75+
<br>
76+
##### `TEMPLATE.md`
77+
This is a short description of the files necessary for creating a reactor.
5278

53-
2. Template.md
54-
* This description of the necessary files to create a reactor.
79+
<br>
80+
##### `config.yml`
81+
This file contains configurations to be passed into the reactor. For example,
82+
if the file has the following contents:
83+
```
84+
---
85+
myVar: value1
86+
myCategory:
87+
var1: value2
88+
```
89+
Then you will be able to retrieve values from an [`AttrDict`][] named '`settings`'
90+
in your `reactor.py`. For example, the above values could be called as:
91+
```
92+
> import reactors as Reactor
93+
> Reactor.settings.get('myVar')
94+
value1
95+
> stuff = Reactor.settings.get('myCategory')
96+
> stuff.get('var1')
97+
value2
98+
```
5599

56-
3. The function file (reactor.py)
57-
* The file `reactor.py` is where the code for your function can be found. If
58-
you need to add more files, extend the `Dockerfile`:
100+
<br>
101+
##### `message.json`
102+
This is the message template. To activate automatic validation of incoming JSON
103+
messages to your reactor, add a valid [JSON schema][] (draft 4+) by extending
104+
the Dockerfile:
59105
```
60-
ADD mycode.py /
106+
ADD message.json /
61107
```
108+
At present, the JSON schema *must* be named `AbacoMessage`
62109

63-
4. Deployment configuration (reactor.rc)
64-
* The CLI reads from `reactor.rc` to automatically set certain attributes of
65-
the Reactor. Of these, only `REACTOR_NAME` is mandatory.
66110

67-
5. Optional configuration files
68-
* We've tried to automate the repetitive parts of deploying functions to the
69-
by making Reactors runtime and build processes smart (but not too clever for
70-
their own good). The files described below unlock that automation.
111+
<br>
112+
##### `reactor.py`
113+
This file is where the code for your main function can be found. If you need to
114+
add other python files located in this same directory, extend the `Dockerfile`
115+
with:
116+
```
117+
ADD mycode.py /
118+
```
119+
An example of a functional reactor could be:
120+
```python2
121+
import reactors as Reactor
122+
import json
71123
72-
6. Function configuration (config.yml)
73-
* If you import the `reactors` module as show in some of the examples, you'll
74-
automatically gain access to an [`AttrDict`][] named `settings` populated by the
75-
contents of `config.yml`.
124+
def main():
125+
md = Reactor.context.message_dict
126+
Reactor.logger.info(json.dumps(md))
76127
77-
7. Message template (message.json)
78-
* To activate automatic validation of incoming JSON messages to your Reactor
79-
add a valid [JSON schema][] (draft 4+) by extending the Dockerfile:
80-
```
81-
ADD message.json /
128+
if __name__ == '__main__':
129+
main()
82130
```
83-
At present, the JSON schema *must* be named `AbacoMessage`
131+
This reactor simply logs the message that is sent to it. Examples of more
132+
interesting reactors with useful functions can be found [here](06.links.md).
133+
134+
<br>
135+
##### `reactor.rc`
136+
The `reactor.rc` file contains important deployment configurations. The Abaco
137+
CLI reads directly from this file when deploying a reactor to the Docker
138+
registry and to the Abaco API.
139+
```
140+
# Reactor mandatory settings
141+
REACTOR_NAME=my-project
142+
143+
# Reactor optional settings
144+
# REACTOR_DESCRIPTION=
145+
# REACTOR_WORKERS=1
146+
# REACTOR_PRIVILEGED=0
147+
# REACTOR_STATELESS=0
148+
# REACTOR_USE_UID=0
149+
# REACTOR_ALIAS=aka_reactor_demo
150+
151+
# Docker settings
152+
DOCKER_HUB_ORG=your_docker_registory_uname
153+
DOCKER_IMAGE_TAG=my-project
154+
DOCKER_IMAGE_VERSION=0.1
155+
```
156+
Ensure the `REACTOR_NAME` is set correctly to `my-project` (or whatever you
157+
would like to call it), and add your Docker Hub username to the `DOCKER_HUB_ORG`
158+
near the bottom.
84159

85-
8. Python dependencies (requirements.txt)
86-
* This is empty by default. If you have additional Python dependencies beyond
87-
those that ship with the Reactors base image, add them here and they will be
88-
included and built (if possible) at `deploy` time or when you manually run
89-
`docker build`.
160+
<br>
161+
##### `requirements.txt`
162+
This is a standard Python requirements file, and is empty by default. If you
163+
have additional Python dependencies beyond those that ship with the Reactors
164+
base image, add them here and they will be included and built (if possible) at
165+
`deploy` time or when you manually run `docker build`.
90166

91-
9. Ignore files
92-
* We've preconfigured .dockerignore and .gitignore files for you that are
93-
tailored towards preventing you from including sensitive information and/or
94-
build cruft in the Docker images and git repositories used in creating Reactors.
167+
<br>
168+
##### `secrets.json`
169+
If you create a file called `secrets.json` (from the example file called
170+
`secrets.json.sample`) it will never be committed to a git
171+
repository or included in a Docker image build, but `abaco deploy` uses it to
172+
populate the default environment variables for the Reactor. Values placed in
173+
this file are *NOT THAT SECRET* since they can be discovered by other users if
174+
you choose to share your Reactor.
95175

96-
10. Secrets file
97-
* If you create a file called `secrets.json` it will never be committed to a Git
98-
repository or included in a Docker image build, but `abaco deploy` use it to
99-
populate the default environment variables for the Reactor. vValues placed in this
100-
file are *NOT THAT SECRET* since they can be discovered by other users if you
101-
choose to share your Reactor.
176+
<br>
177+
##### Ignore files
178+
Also in this directory are preconfigured `.dockerignore` and `.gitignore` files
179+
that are tailored towards preventing you from including sensitive information
180+
and/or build cruft in the Docker images and git repositories used in creating
181+
Reactors.
102182

103-
#### Building and Deploying your Reactor
183+
<br>
184+
#### Deploy your reactor
185+
186+
Before deploying your reactor, first make sure to refresh the Agave token:
187+
```
188+
% auth-tokens-refresh
189+
```
190+
191+
Once you have added the necessary information to the files listed above, you can
192+
deploy your reactor by executing the `abaco deploy` command from the top level
193+
of your reactor project directory:
194+
```
195+
% abaco deploy
196+
Sending build context to Docker daemon 11.78kB
197+
Step 1/1 : FROM sd2e/reactor-base:python2
198+
# Executing 4 build triggers
199+
---> Using cache
200+
---> Using cache
201+
---> Using cache
202+
---> Using cache
203+
---> 1b7f8366e05f
204+
Successfully built 1b7f8366e05f
205+
Successfully tagged username/my-project:0.1
206+
The push refers to repository [docker.io/username/my-project]
207+
45080ac9c614: Layer already exists
208+
8a5132998025: Layer already exists
209+
...
210+
aca233ed29c3: Layer already exists
211+
e5d2f035d7a4: Layer already exists
212+
0.1: digest: sha256:997c7f59sa5aaaa07f2f4w1dwdbc1f0ar4frf3fbq7qb1d5dcdb0f2f42056d5f1e1a size: 6987
213+
[INFO] Pausing to let Docker Hub register that the repo has been pushed
214+
Successfully deployed Actor with ID: wmoxRE7rEqGPQ
215+
```
216+
217+
This process pushes a Docker image to the location specified in your `reactor.rc`
218+
file, and it deploys an Abaco reactor object into the API. If all goes well, you
219+
will find a "Success" message at the end, along with the reactor ID (as a
220+
13-character alphanumerical string). To run this particular reactor, send it a
221+
message with any contents:
222+
```
223+
% export MESSAGE='{ "myVar":"value1" }'
224+
% abaco submit -m "${MESSAGE}" wmoxRE7rEqGPQ
225+
kWmayogY84RPA
226+
{
227+
"myVar": "value1"
228+
}
229+
% abaco logs wmoxRE7rEqGPQ kWmayogY84RPA
230+
Logs for execution kWmayogY84RPA:
231+
[INFO] 2018-05-22T03:06:00Z: Returning a logger set to level: INFO for module: wmoxRE7rEqGPQ
232+
[INFO] 2018-05-22T03:06:00Z: {"myVar": "value1"}
233+
```
234+
235+
More descriptions on the various Abaco commands can be found in the
236+
[Abaco CLI introduction](04.abaco_cli.md).
104237

105-
Once you've added the necessary information to the files listed above, you can deploy your reactor by executing this command from your reactor directory:
106-
```
107-
abaco deploy
108-
```
109-
Or if you'd like to just build (but not deploy) the reactor, in order to test functions locally, you can use:
110-
```
111-
abaco depoly -R
112-
```
113-
Once deployed, abaco will return an 13-digit alpahnumerical hex which will serve as your Reactor ID.
114238

115239
---
116240
Return to the [API Documentation Overview](../index.md)

0 commit comments

Comments
 (0)