Upon testing AWS Auto-scaling I came across a problem with an RHEL AMI that I created from a snapshot.
Basically the VM started up and everything seemed ok eg. database up, services started however when I tried to check for updates I received a 401 HTTPS error – Unauthorized from all the mirror servers. Quickly tested updates on other servers within the same VPC, Security Group and Subnet however they had no issues. It appeared an issue with this AMI only.
A quick search on the Amazon Forum revealed this is a bug that has been around since RHEL 5 at least.
Next up was to confirm this was indeed something to do with BillingProducts value being “Null” I queried the instance metadata on http://169.254.169.254/ to find this out.
Yep, it seems to be empty. I figured it could be somethign to do with the yum cache so issued sudo yum clean all however this made no change unfortunately.
Apparently there was not much else to change on the broken instance hence creating a new AMI was on the table. This time however rather than creating a snapshot and then the AMI I went ahead and created the AMI from the running instance.
Now please note this may be disruptive unless you choose not to restart the running instance but the new AMI image may not be consistent ending up with a broken image again.
Creating the AMI may take a little while so go get a cuppa. Once the job finished launch a new instance using the newly created AMI and test sudo yum update -y again – it should all work as expected.
As always send me a message or leave a comment if you found this helpful.