boto and load autoscaling groups

In boto, get_all_groups has a max return of 100. to get past that, you use tokens:

asconn = boto.ec2.autoscale.connect_to_region(self.region_name)
# This will load all groups.
# Source: http://stackoverflow.com/questions/29317526/how-can-i-retrieve-more-than-50-autoscaling-groups-via-python-boto
all_groups = []
rs = asconn.get_all_groups()
all_groups.extend(rs)
while rs.next_token:
    rs = asconn.get_all_groups(next_token=rs.next_token)
    all_groups.extend(rs)
for asg in all_groups:
    print self.stack_name + " = " + asg.name

AWS / Ansible Dynamic Inventory

I’ve been working to manage dynamic inventory in AWS for Ansible deploys… then I came across this stack overflow link & ches’ answer.

Ansible looks for executables and flat files in a directory and merges their results.

=> tree inventory/staging
inventory/staging
-- base
-- ec2.ini
-- ec2.py
-- group_vars -> ../group_vars

The base file looks like:

=>  more inventory/staging/base
[localhost]
# I need to tell Ansible which Python on my system has boto for AWS
127.0.0.1 ansible_python_interpreter=/usr/local/bin/python

# The EC2 plugin will populate these groups, but we need to add empty entries
# here to make aliases for them below.
[tag_Stage_staging]
[tag_Role_webserver]

[staging:children]
tag_Stage_staging

[webservers:children]
tag_Role_webserver

You then just point to the directory for inventory:

$ ansible -i inventory/staging webservers -m ec2_facts
# OR
$ export ANSIBLE_HOSTS=inventory/staging
$ ansible webservers -m ec2_facts

AWS via Ansible – use private key

With AWS ssh, you need to use a private key. When working on a new script, I didn’t want to deal with my private account having a “build box” which was already on the VPC. So I was using my box & giving the destination a public IP. I know… totally insecure but considering I was killing the VM every few minutes I didn’t care.

So to call ansible-playbook & provide a private key:

ansible-playbook -i envs/localhost elasticsearch.yml -vvvv --private-key=~/.ssh/mykeyname.pem 

aws cmd line tool – use profiles

I’ve got a home account & a work account. I need to easily swap between the 2.
Add 2 sets of creds
~/.aws/credentials

[default]
aws_access_key_id=AKIAIOSFODNN7EXAMPLE
aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

[work]
aws_access_key_id=AKIAI44QH8DHBEXAMPLE
aws_secret_access_key=je7MtGbClwBF/2Zp9Utk/h3yCo8nvbEXAMPLEKEY

Add 2 sets of region / outputs
~/.aws/config

[default]
region=us-west-2
output=json

[profile work]
region=us-east-1
output=text

Then to use a profile:

export AWS_PROFILE=work

OR

aws ec2 describe-instances --profile work

SOURCE

Configure EC2 CLI Tools

If you haven’t done so already, configure the EC2 CLI tools. You can download the latest EC2 toolchain from here.
(It’s also available using homebrew “brew install ec2-api-tools”)
Now configure the environment variables:

export EC2_HOME=~/.ec2
export PATH=$PATH:$EC2_HOME/bin
export JAVA_HOME=$(/usr/libexec/java_home)
export AWS_ACCESS_KEY=
export AWS_SECRET_KEY=

Note: The Java path cannot be a symlink. It must be the actual path to the java home.

Test the EC2 CLI tools by running:

ec2-describe-regions
Posted in AWS