Using Jython to Manage EC2 Resources: Part 2
Posted on 2011-01-16 19:05:59 by JerIn part 1, we installed and configured jython and the AWS SDK, started a REPL and initiated our first connection to EC2. Now, we'll look at a real use case and work through the implementation. I'll attempt to show the flavor of working in the REPL to make incremental progress.
The TaskI want to spin up an instance of an AMI, create a number (6) of EBS volumes, attach them, do some work and shut down the instance to avoid recurring compute charges. The data does need to persist, hence the EBS volumes. Because I'm keeping the data around to use again, I want to be able to attach the same volumes to another instance later with the same mount points. This could be done manually, but it is incredibly tedious. Once is enough. Ultimately, I am going to want a number of these instances and the manual method would be completely untenable.
Let's get started. We need to spin up an instance first.
>>> c.runInstances('ami-e4a3578d')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: runInstances(): 1st arg can't be coerced to
com.amazonaws.services.ec2.model.RunInstancesRequest
And so it begins. We didn't call runInstances() correctly. Can we introspect runInstances() to find out how to call it?
>>> help(c.runInstances) Help on method runInstances: runInstances(...) method of com.amazonaws.services.ec2.AmazonEC2Client instance
Unfortunately not. We know we need a RunInstancesRequest. Let's create one.
>>> import com.amazonaws.services.ec2.model as model >>> req = model.RunInstancesRequest()
And try again.
>>> c.runInstances(req) Jan 16, 2011 9:29:01 AM com.amazonaws.http.HttpClient execute INFO: Sending Request: POST https://ec2.amazonaws.com / Parameters: (Action: RunInstances, SignatureMethod: HmacSHA256, AWSAccessKeyId: REDACTED, Version: 2010-08-31, SignatureVersion: 2, Timestamp: 2011-01-16T15:29:01.273Z, Signature: jZ0xmayF6y+QAeHV+bjyNtOkeOEZYNAeMZwYMtsle0s=, ) Jan 16, 2011 9:29:03 AM com.amazonaws.http.HttpClient handleErrorResponse INFO: Received error response: Status Code: 400, AWS Request ID: 23a77814-9344-41fd-b13f-36cd57300ab9, AWS Error Code: MissingParameter, AWS Error Message: The request must contain the parameter ImageId Traceback (most recent call last): File "<stdin>", line 1, in <module> at com.amazonaws.http.HttpClient.handleErrorResponse(HttpClient.java:519) at com.amazonaws.http.HttpClient.execute(HttpClient.java:215) at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:3804) at com.amazonaws.services.ec2.AmazonEC2Client.runInstances(AmazonEC2Client.java:267) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) com.amazonaws.AmazonServiceException: Status Code: 400, AWS Request ID: 23a77814-9344-41fd-b13f-36cd57300ab9, AWS Error Code: MissingParameter, AWS Error Message: The request must contain the parameter ImageId
OK, that was deliberately obtuse. We didn't specify the AMI to start. But you'll notice that a) we actually talked to EC2 this time and b) got a useful error response back. Nice! This time we'll look at the javadoc and setup the RunInstancesRequest correctly. We need to pass the constructor the AMI, minimum and maximum number of instances we want.
>>> c.runInstances(model.RunInstancesRequest('ami-e4a3578d', 1,1))
com.amazonaws.AmazonServiceException: Status Code: 400,
AWS Request ID: 7b2c61dc-9014-4d95-9fdf-01b899140c88,
AWS Error Code: InvalidParameterValue,
AWS Error Message: The requested instance type's
architecture (i386) does not match the architecture
in the manifest for ami-e4a3578d (x86_64)
Close, but no cigar. Starting now, I'm omitting the HTTP request logging and stack trace from errors or this will be the longest post ever. EC2 defaults to using an m1.small instance type. We're asking for a 64-bit AMI, so we need to specify an instance type of m1.large. Consulting the javadoc, we see that RunInstanceRequest has a setInstanceType() method. Perfect.
>>> req = model.RunInstancesRequest('ami-e4a3578d', 1,1)
>>> req.setInstanceType('m1.large')
>>> resp = c.runInstances(req)
>>> resp
{Reservation: {ReservationId: r-d19c9ebb, OwnerId: 923332273044,
RequesterId: null, GroupNames: [default],
Instances: [{InstanceId: i-459b4829,
ImageId: ami-e4a3578d, State: {Code: 0, Name: pending, },
PrivateDnsName: , PublicDnsName: , StateTransitionReason: ,
KeyName: null, AmiLaunchIndex: 0, ProductCodes: null,
InstanceType: m1.large, LaunchTime: Sun Jan 16 10:31:47 CST 2011,
Placement: {AvailabilityZone: us-east-1b, GroupName: , },
KernelId: aki-427d952b, RamdiskId: null, Platform: null,
Monitoring: {State: disabled, }, SubnetId: null, VpcId: null,
PrivateIpAddress: null, PublicIpAddress: null,
StateReason: {Code: pending, Message: pending, },
Architecture: null, RootDeviceType: ebs,
RootDeviceName: /dev/sda1, BlockDeviceMappings: null,
VirtualizationType: paravirtual, InstanceLifecycle: null,
SpotInstanceRequestId: null, License: null,
ClientToken: , Tags: null, }], }, }
Running instance, check. You'll notice we captured the return value of runInstances() in a variable. This will come in handy when we need to start working with the instance.
The next thing we need to do is create the volumes. I'll bet our client has a method to create volumes.
>>> 'createVolumes' in dir(c) False >>> 'createVolume' in dir(c) True
So, createVolume() it is. The javadoc also tells us that of course, but we can also introspect. It's a judgment call which is faster. Now we're going to start moving a little faster before I bore you to death.
We can see from the javadoc that we need to specify up to three parameters to create our volume: size, snapshot ID, and availability zone. To start with we only care about size and availability zone. We need to check the availability zone of our instance and make sure we create the volume in the same zone. Otherwise we could create it and be unable to attach it. Since we didn't specify a zone when we spun up our instance (oops), lets try and get it from our instance.
>>> instances=resp.getReservation().getInstances() >>> len(instances) 1 >>> i = instances[0] >>> az = i.getPlacement().getAvailabilityZone() >>> az u'us-east-1b'
There. Took a bit of work, but we got the availability zone. If you looked closely when we printed the repl object above, you may have noticed the zone in the Placement section. That and some javadoc perusal led to the getPlacement() method. Next time, let's specify that when we start the instance and save the trouble.
>>> vresp = c.createVolume(model.CreateVolumeRequest(1, az))
>>> vresp
{Volume: {VolumeId: vol-8219a1ea, Size: 1, SnapshotId: ,
AvailabilityZone: us-east-1b, State: creating,
CreateTime: Sun Jan 16 11:10:47 CST 2011,
Attachments: null, Tags: null, }, }
>>> vol = vresp.getVolume()
>>> vol
{VolumeId: vol-8219a1ea, Size: 1, SnapshotId: ,
AvailabilityZone: us-east-1b, State: creating,
CreateTime: Sun Jan 16 11:10:47 CST 2011,
Attachments: null, Tags: null, }
Now we've created a volume. Let's attach it to our instance.
>>> vol.getVolumeId()
u'vol-8219a1ea'
>>> i.getInstanceId()
u'i-459b4829'
>>> model.AttachVolumeRequest(vol.getVolumeId(),
i.getInstanceId(), '/dev/sdf')
{VolumeId: vol-8219a1ea, InstanceId: i-459b4829, Device: /dev/sdf, }
>>> req = model.AttachVolumeRequest(vol.getVolumeId(),
i.getInstanceId(), '/dev/sdf')
>>> resp = c.attachVolume(req)
>>> resp
{Attachment: {VolumeId: vol-8219a1ea,
InstanceId: i-459b4829,
Device: /dev/sdf, State: attaching,
AttachTime: Sun Jan 16 11:16:25 CST 2011,
DeleteOnTermination: null, }, }
There it is. We have an running instance with EBS volume attached.
We've gone a long way towards implementation of our use case. We have created an instance, created a volume and attached the volume to the instance. If you made it this far, well done - I'm sure you found it a bit of work if you followed all the examples. Hopefully, you got a good taste of the give and take of working interactively in the REPL. The important thing to remember is in actuality these iterations progress rapidly and you can build your code from the bottom up as you go.
Now that you've got the flavor, we'll start taking it in bigger chunks in the next episode and look at taking the code we've gotten working in the REPL and putting it together to finish implementing our use case.
blog comments powered by Disqus
About Me
Subscribe
to this weblog's
atom feedRecent Articles
Best Books of 2013How to Create a Histogram with SQL
Bulletproof Coffee Redux
On Being T-Shaped
Rudderectomy