muledump-stack-updater

Launches an EC2 instance, runs muledump-asset-compiler, synchronizes with S3, and terminates.

How It Works

This script is designed to be ran with AWS Lambda but can be ran from anywhere. When running in Lambda, all authentication is handled using assumed the IAM role. If running without an assumed role, IAM credentials can be provided instead.

The function first builds a startup script using the supplied configuration. Then an EC2 instance template is requested from your S3 bucket.

An EC2 instance is launched and supplied the generated startup script.

The startup script prepares the instance OS, downloads Muledump Asset Compiler, and executes the program using the provided bootstrap script.

When the program finishes running, the instance shuts down and terminates automatically.

Requirements

The main program will run on any OS with NodeJS v18.

Currently, only Amazon Linux 2023 on x86_64 is supported for the runtime EC2 instances.

Other Linux-based AMIs are possible if you tweak the package manager references in both the user data and muledump-asset-compiler bootstrap scripts.

Setup

1. Create your IAM user with no permissions or groups
   1. This user account is utilized by the EC2 instance for all of its AWSCLI needs
2. Create an API access key for your IAM user
3. Create your IAM role with no permissions or groups
4. Create an inline permission policy for your IAM role using the template in docs/policies/iam-role-runInstances.json
5. Create your S3 bucket and add a bucket policy using the template in docs/policies/s3-bucket.json
   1. If using S3 for the source for muledump-asset-compiler then upload the zip to the preferred location (example: packages/muledump-asset-compiler.zip)
   2. Upload your customized runInstancesConfig.json from docs/runInstancesConfig.json.sample to config/runInstancesConfig.json
6. Create your Cloudfront distribution(s)
   1. Select your S3 bucket for the origin
   2. Select Origin access control settings for Origin Access and choose Create control setting with default values
      1. Be sure to copy the S3 bucket policy it provides you here
      2. If it isn't displayed here, you can find it by going to the origin settings in the distribution after creation
      3. This policy must be added as an additional statement in your S3 bucket policy
      4. If you are using one Cloudfront distribution then this policy must be updated to block access to logs/ and config/
   3. Choose your Cache Policy
      1. Select CacheDisabled if you want immediate access to new assets without manual cache purging
      2. Select CacheOptimized if you want to minimize S3 traffic
   4. If you are using two Cloudfront distributions
      1. Create one Cloudfront distribution for each of rotmg-assets and muledump-renders using the above settings
      2. After selecting your S3 bucket, set the correct Origin path (e.g. rotmg-assets/ or muledump-renders/)
7. Create your Secrets Manager secret and add the keys found in docs/aws-secrets-manager-secret.json
   1. The default secret name is muledump-stack-updater/config
   2. Add a permissions policy using the template found in docs/policies/aws-secrets-manager.json
8. Create your AWS Lambda function using Nodejs v18
   1. Upload muledump-stack-updater.zip to the Lambda function
      1. If you are building this from the Git repo, you must first npm install and include the node_modules folder in your zip archive
      2. You must update config.json in the archive before upload if you are:
         1. Setting up in a region other than us-east-1
         2. Using a secret name other than muledump-stack-updater/config
         3. Using IAM access keys instead of a role for access to S3 and EC2 permissions
   2. Select the role you created earlier, or create a new role and make sure you update its permissions as described above
   3. (optional) Create a trigger
      1. Choose type EventBridge (CloudWatch Events)
      2. Create a rate: cron(*/10 * * * ? *)
         1. This example is set to run every 10 minutes
      3. This trigger utilizes the build-info.json object generated by the muledump-renders-rotmg-version-monitor

EC2 Resource Sizing

Minimum requirements of 2 vCPU cores and 4GB RAM.

With the default settings and runInstances configuration, the whole stack runtime averages 3 minutes.

There are diminishing gains on using instances with more vCPU or RAM. For example, c6a.2xlarge runtime is 2 minutes 50 seconds, and c6a.4xlarge is 2 minutes 45 seconds.

Free tier instance sizes don't provide enough RAM to decompress the game assets.

Below is a table of tested instance types and their average stack runtime.

Instance Type	vCPU	Core Speed	Stack Runtime	$/Minute	$/Run
c6a.large	2	3.60Ghz	3:00	$0.00128	$0.00384
c6a.xlarge	4	3.60Ghz	3:00	$0.00255	$0.00765
t3a.xlarge	4	2.5Ghz	4:45	$0.00251	$0.01192
c6i.xlarge	4	3.5Ghz	7:35	$0.00283	$0.02146
t3.xlarge	4	3.1Ghz	8:35	$0.00277	$0.02378

Configuration

By default, this program utilizes AWS Secrets Manager to load the runtime configuration.

The following keys are utilized:

Key	Default Value	Description
access_key_id	""	AWS Access Key ID
secret_access_key	""	AWS Secret Access Key
s3_bucket	""	AWS S3 Bucket
s3_prefix_assets	"assets/"	S3 key prefix for all assets resources
s3_prefix_renders	"renders/"	S3 key prefix for all renders resources
s3_log_path	"logs"	S3 key prefix for all logging data
runInstancesPath	"config/runInstancesConfig.json"	EC2 instance configuration path in S3
buildInfoS3	"assets/build-info.json"	S3 path for build-info.json object
buildInfoHTTPS	"https://assets.muledump.com/build-info.json"	HTTPS path for build-info.json object
macLocation	"https://github.com/jakcodex/muledump-asset-compiler.git"	URI for muledump-asset-compiler
compareHashes	"true"	Compare currently built build hash vs latest detected build hash
shutdownOnError	"true"	Shutdown EC2 instance when an error occurs (takes precedence over shutdownOnComplete)
shutdownOnComplete	"true"	Shutdown EC2 instance when the task completes
force	"false"	Force EC2 instance launch (takes precedence over compareHashes)

The AWS Key and S3 bucket keys are required when uploading generated resources and runtime logs to AWS S3.

This package contains a config.json script. This configuration only needs to be changed if you need to:

1. Utilize AWS IAM credentials instead of a role for accessing AWS Secrets Manager and EC2
2. Utilize a region other than the default us-east-1

Support

Jakcodex operates its own Discord server at https://discord.gg/JFS5fqW.

Feel free to join and ask for help getting setup, hear about new updates, offer your suggestions and feedback, or just say hi. We love to hear from the community!

If you encounter a bug, have a feature request, or have any other feedback you can also check out the issue tracker to see if it's already being discussed. If not then you can submit a new issue.

Jakcodex License

Copyright 2023 Jakcodex

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.