AWS CloudFormation stack manager
Stax creates and manages CloudFormation stacks (aka stax) in AWS (Amazon Web Services). Several CloudFormation templates are provided with stax, take a look at them in the templates
directory to modify and create your own.
As an illustration, with the proper configuration and templates, stax
can create a set of entities in AWS like those shown in the diagram below.
Stax requires Linux (tested on Debian GNU/Linux 7/8 and Ubuntu 14.04/15.04) and Apple OS X (tested on 10.09/10.10). You'll also need an account with Amazon Web Services.
Install Homebrew for OS X, which '...installs the stuff you need that Apple didn’t'. Basically it's a *nix package manager like we have in Linux. It's easy to install via their setup script:
$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Visit their site to learn how to install it manually if you don't trust that line.
Install curl if you don't already have it installed (OS X has it by default, and most Linux distros have it by default) on your client.
$ apt-get install curl # Debian GNU/Linux, Ubuntu
$ yum install curl # Red Hat Enterprise Linux (RHEL), CentOS, Amazon Linux
Install aws-cli (Universal Command Line Interface for Amazon Web Services) on your client.
NOTE: the version in the Debian repo is out of date, but we'll need the latest version, to get that we'll first need to install the Python package manager pip
. These [install docs](http://www.cyberciti.biz/faq/debian-ubuntu-centos-rhel-linux-install-pipclient/] will help you get pip
installed, once complete, go to the next step to install awscli
.
$ pip install awscli # Debian GNU/Linux, Ubuntu
$ brew install awscli # Apple OS X ([via Homebrew](http://brew.sh/))
$ yum install awscli # Red Hat Enterprise Linux (RHEL), CentOS, Amazon Linux
After installing the AWS CLI, you will need to configure it with your AWS access credentials, specifically an AWS API key and secret. You can generate these credentials from your IAM user page on the AWS Console. Choose your username, scroll down to Security Credentials > Access Keys (Acess Key ID and Secret Access Key), and click Create Keys if you haven't created them yet. Running stax requires many permissions in AWS, so be sure the IAM user you are running stax as has sufficient permissions, and in an admin group (called 'wheel' by default). If you already have this group, and your user is in that group, continue to 'aws configure' - otherwise follow the next steps to setup the 'wheel' group.
NOTE: if this is a new install (the AWS environment hasn't been used before) you'll need to create the wheel group, and add your user to that group for the user to have permissions to build on AWS.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "*",
"Resource": "*"
}
]
}
Click 'Apply Policy' - Now apply your user to that group, click Users > Click your new user > Click Add User to Groups > select the 'wheel' group > Click Add to Groups
Run the AWS configure utility to enter your keys and regions.
$ aws configure
You will be prompted to enter your access key and secret access key, copy this from the AWS Console you opened above. Next you will be prompted to enter your AWS region, currently we have stax setup to use us-east-1
. Finally you will be prompted for the default output format, enter json
. The AWS CLI will store your configuration in ~/.aws/credentials
. If you need to change your configuration, you can either run aws configure
again or edit this file directly.
Two templates are provided with stax to get you started, below are brief descriptions.
The template template/simple.json
stands up a simple AWS environment. Since it is simple to comprehend and spins up quickly, this template is intended as a sandbox to stand up and test new CloudFormation entities. For example, the implementation of CloudWatch Alarms was first tested using this template. The config/simple.json
has minimal parameters and the default values are acceptable for all but the project/team parameters.
Project
REQUIRED: Name for the project with which this application ecosystem is associatedProduct
REQUIRED: Name for the end-user facing product service by this application ecosystemGroup
REQUIRED: Name of the group that created this stackTeam
REQUIRED: Name of the team that created this stackEnvironment
REQUIRED: Type of environment, e. g., production, test, branch nameKeepAlive
REQUIRED: Tag to avoid the reaper, default is falseInstanceType
: AWS EC2 instance type for NAT instances, default is t2.microOwner
REQUIRED: Tag for owner of the stack, if set to @OWNER@
stax will substitute the current user idKeyName
REQUIRED: AWS EC2 Key Pair to use when launching instances, if set to @KEY_NAME@
stax will use a key pair it createsSSHLocation
: CIDR IP range to allow SSH access to bastion host, if set to @SSH_FROM@
stax will replace it with the CIDR IP of the host running stax, default is 0.0.0.0/0The template template/vpc-default.json
creates an environment useful for deploying microservices as docker containers in AWS. The following entities are created:
The text in parentheses is the first part of the name given to the entity, with the latter part being the CloudFormation stack name.
This template takes the following parameters, see config/vpc-default.json.example
for the format.
Project
REQUIRED: Name for the project with which this
application ecosystem is associatedProduct
REQUIRED: Name for the end-user facing product
service by this application ecosystemGroup
REQUIRED: Name of the group that created this stackTeam
REQUIRED: Name of the team that created this stackEnvironment
REQUIRED: Type of environment, e. g., production, test, branch nameKeepAlive
REQUIRED: Tag to avoid the reaper, default is falseOwner
REQUIRED: Tag for owner of the stack, if set to @OWNER@
stax will
substitute the current user idKeyName
REQUIRED: AWS EC2 Key Pair to use when launching instances, if set
to @KEY_NAME@
stax will use a key pair it createsServiceELBSubdomainName
: Route 53 subdomain, leave blank to not
create oneServiceELBBaseDomainName
: Route 53 base domain name, leave blank
to not create oneServiceELBSSLCertName
: IAM name of SSL existing certificate, leave
blank to not use SSLDockerRegistryUser
: Private Docker registry user nameDockerRegistryPass
: Private Docker registry passwordDockerRegistryEmail
: Private Docker registry email addressDockerRegistryUrl
: Private Docker registry URL, defaults to https://index.docker.io/v1/
ServiceClusterSize
: Target size of Service Cluster auto scaling
group, default is 0GatewayClusterSize
: Target size of Gateway Cluster auto scaling
group, default is 2BastionInstanceType
: AWS EC2 instance type for bastion host,
default is t2.microNATInstanceType
: AWS EC2 instance type for NAT instances, default
is t2.microDockerInstanceType
: AWS EC2 instance type for CoreOS/Docker hosts,
default is m3.mediumCoreOSChannelAMI
: CoreOS channel (stable, beta, alpha) to use for
CoreOS instances, default is CoreOSStableAMIAdvertisedIPAddress
: Which instance IP address to use for
clustering, private or public, default is private, use public to
cluster across unpaired VPCsAutoScaleCooldown
: Auto scaling cooldown period, in seconds, after
an auto scaling event, default is 300AllowSSHFrom
: CIDR IP range to allow SSH access to bastion host,
if set to @SSH_FROM@
stax will replace it with the CIDR IP of the
host running staxAllowHTTPFrom
: CIDR IP range from which to allow HTTP access to
the Gateway ELB, if set to @HTTP_FROM@
stax will replace it with
the CIDR IP of the host running stax$ git clone https://github.com/MonsantoCo/stax
$ cd stax
$ cp config/vpc-default.json.example config/vpc-default.json
<changeMe>
.$ vi config/vpc-default.json
$ ./stax
Usage: stax [OPTIONS] COMMAND [COMMAND_ARGS]
Options:
-c,--config=CONFIG Use file CONFIG rather than config/vpc-default.json
-d,--debug Turn on verbose messages
-h,--help Output this message
-j,--jump=IP SSH through host with IP address IP
-m,--module=MOD Use config/MOD.json and template/MOD.json
-q,--quiet Mute status messages
-t,--template=TEMPLATE Use file TEMPLATE rather than template/vpc-default.json
-v,--version Print name and version information
-y,--yes Do not prompt for confirmation
If an argument is required for a long option, so to the short. Same for
optional arguments.
Commands:
add Add functionality to an existing VPC
add-logger Add logging functionality to an existing VPC
auto-services Lanch multiple services on fleet using a template/NAME.services file
check Run various tests against an existing stax
connect [TARGET] Connect to bastion|gateway|service in the VPC stax over SSH
create Create a new VPC stax in AWS
describe Describe the stax created from this host
delete Delete the existing VPC stax
dockerip-update Fetch docker IP addresses and update related files
fleet Run various fleetctl commands against the fleet cluster
help Output this message
history View history of recently created/deleted stax
list List all completely built and running stax
rds PASSWORD Create an RDS instance in the DB subnet
rds-delete RDSIN Delete RDS instance RDSIN
remove ADD Remove the previously added ADD
services List servers that are available to run across a stax
slack Post usage report to Slack, define hook in stax.config
sleep Turn on/off bastion host which allows ssh access into the VPC
start SERVICE Start service SERVICE in the fleet cluster
test Automated test to exercise functionality of stax
update Update an existing VPC with changes from Cloudformation
validate Validate CloudFormation template
For more help, check the docs: https://github.com/MonsantoCo/stax
$ ./stax --config=config/vpc-default.json --template=template/vpc-default.json create
It will take ~5 minutes to build, you can test if it's ready by trying to connect.
$ ./stax connect
The command will check if the cluster is built, and if it is, it will populate the bastion host with needed information and then connect you to it.
$ ./stax connect gateway[N]
$ ./stax connect service[N]
fleet
command:$ ./stax fleet
check
command:$ ./stax check
$ ./stax services
$ ./stax start SERVICE
replacing SERVICE
with the name of the service you want to deploy.
The service deployment is done using fleet
.
$ ./stax rds PASSWORD
replacing PASSWORD
with a password you will need to connect to the
database.
To delete the stack and the other entities stax creates as part of its management, e. g., the S3 bucket, EC2 Key Pair, and RDS instances, run the following command:
$ ./stax delete
An optional customization configuration file is available at config/stax.config
, which allows you to specify option parameters for your stax. Currently the file allows customization of the prefix stax uses to name your stax, an ssh jumpbox you might have to proxy through to get to your stax, and integration with Slack.
## stax.config - optional parameters specific to your environment
# the prefix for the name of your stax, default is "stax"
STAX_TAG_PREFIX=""
# the IP of a ssh 'jumpbox' to proxy SSH connections to your VPC
STAX_JUMP=""
# the webhook URL to allow stax to post to a slack channel (#stax by default)
# see https://slack.com/services/ for setup documentation
STAX_SLACK_WHOOK=""
cp config/stax.config.example ~/.stax.config
~/.stax.config
and enter your valuesNow, running stax will see that file and source it, using the found variables over any others in the config
directory.
The stax project started off with ideas from the following projects:
Thanks everyone, Open Source FTW!
# Modified BSD License
Copyright (c) 2015, Monsanto Company
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of the Monsanto Company nor the
names of its contributors may be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL MONSANTO COMPANY BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Stax, besides being a clever take on the word stacks, is named after the famous Stax Recording Studio in Memphis, TN. If you're ever in Memphis, visit the awesome Stax Museum... it's far more interesting than Sun Studios, but I digress.