When signing up for Amazon Web Services, you end up generating all of the following lovely identifying information:
- Your Account Number
- An Access Key
- A Secret Access Key
- An SSL Certificate file
- The Private Key for your SSL Cert
And the various AWS command APIs and tools will require you to provide one more more of these secret-y pieces of information. But I’d rather not host all of this information on my instances. A connundrum indeed.
I chose to solve it with a Ruby toolkit using:
My needs are small-scale for the moment, so I can keep a hard-coded list of my instance IDs. The EC2 API allows me to fetch instance metadata, identify their public DNS (plus everything else), and
- open up an
- push data into a Channel’s environment,
- or upload files for bundling purposes
I learned a lot.
It looked like a good plan to do most of the remoting over an
Net::SSH::Connection::Channel, and sometimes asynchronously (as in the case of my Health Check script).
There are a lot of library shorthands – this “Hello World” example works great, and it is blissfully unaware of Channels:
Net::SSH.start("localhost", "user") do |ssh| # 'ssh' is an instance of Net::SSH::Connection::Session ssh.exec! "echo Hello World" end
But as with all examples, you soon find that surface-level tools won’t quite do the trick. The main thing you’ll need to have all the time is convenient access to the data come back from the Channel, asynchronously or otherwise. So create yourself a simple observer:
Then during and after executions, you’ll have access to all the core information you’ll need to make conditional decisions, etc.
Pushing Your Environment
Many of the AWS tools will recognize specific environment variables. The usual suspects are:
So, you can use
Net::SSH::Connection::Channel.env to inject key-value pairs into your connection. However, you’ll need to make some config changes first – and major thanks to the Net::SSH documentation for clarifying this in its own RDoc:
“Modify /etc/ssh/sshd_config so that it includes an AcceptEnv for each variable you intend to push over the wire”
# accept EC2 config AcceptEnv AMAZON_ACCESS_KEY_ID AMAZON_SECRET_ACCESS_KEY ...
After you make your changes, make sure to bounce the ssh daemon:
% /etc/init.d/sshd restart
Then your selected environment will shine through.
Running sudo Remotely
It’s reasonable to execute root-level commands using sudo, because you can provide a good amount of granular control.
The first thing that we’ll all run into is:
sudo: sorry, you must have a tty to run sudo
ch.request_pty do |ch, success| raise "could not start a pseudo-tty" unless success # full EC2 environment ###ch.env 'key', 'value' ###... ch.exec 'sudo echo Hello 1337' do |ch, success| raise "could not exec against a pseudo-tty" unless success end end
I’ve taken the approach of allowing
NOPASSWD execution for everything I do remotely, after making darn sure that I had constrained exactly what could be done under those auspices (HINT: pounds of caution, tons of prevention).
You can configure all of this by editing
# EC2 tool permissions username ALL=(ALL) NOPASSWD: SETENV: /path/to/scripts/*.rb ...
Also, if you intend to have those scripts consume the environment variables that you’ve injected into your Channel, you’ll also need to annotate the line with
SETENV, or they won’t export across into your sudo execution.
If you are more security minded, then there’s many other variations you can play with /etc/sudoers. I haven’t yet experimented within pushing a sudo password across the wire, but that may be where
Net::SSH::Connection::Channel.send_data comes into play.
I wanted to easily do SSH tunneling / port-forwarding against an instance, referenced by my local shorthand. So I wrote my usual EC2 instance ID lookup and started an Net::SSH connection. Here’s how you put yourself in ‘dumb forward’ mode:
[Ctrl-C] will break you out.
Inconsistent EC2 Marshalling
I’d gotten all of my EC2 data parsing working great, then I started to expand my capabilities with the Amazon S3 API gem (aws-s3). Suddenly and magically, the results coming back from my EC2 queries had taken on a new form. Moreover, this was not consistent behavior across all operating systems; at least I do not see it on WinXP, under which my health monitoring checks execute.
The Amazon APIs are SOAP-driven, so I guessed that amazon-ec2 (and aws-3) would both leverage soap4r. Well, that is in fact not the case; Glenn has commented below on this known issue. He uses the HTTP Query API, and the cause of the discrepancy are (as of this writing) still undetermined.
Regardless, our job is to get things to work properly. So for the meanwhile, let’s use a quick & dirty work-around. The two core differences are:
- the data coming back may be wrapped in an additional key-value pair
- single value result-sets come back as an Object (vs. an Array with a single Object)
So I crafted a little helper singleton, which which still allows me to detect between ‘valid’ and ‘empty’ requests (
That will address our immediate needs for the moment, until a benificent coding Samaritan comes along.