16 Jan 2014
While implementing graphite I found many examples of upstart scripts for the carbon cache agents. However, each of those examples failed to work on a system using upstart version 0.6.5. Most of them would start carbon up, then lose the pid and in turn lose the ability to stop and restart the process. I wanted upstart to start, restart and stop carbon cache agent processes, so I ended using the --debug option. (maybe that's a hack, but it does work)
Below is a working upstart script for starting, restarting and stopping carbon cache.
description "Graphite Carbon Cache"
start on runlevel [2345]
stop on runlevel [!2345]
env CARBON_CACHE=/opt/graphite/bin/carbon-cache.py
env PIDFILE=/opt/graphite/storage/logs/carbon-cache.pid
respawn
kill timeout 5
post-stop exec $CARBON_CACHE --pidfile=$PIDFILE --debug stop
script
ulimit -n 65000
exec $CARBON_CACHE --pidfile=$PIDFILE --debug start >> /opt/graphite/storage/log/carbon-cache/console.log 2>&1
end script
12 Dec 2013
As you can tell from my posts I use fog a lot and it works well in most situations. Here's another one, I want to load up a hosted zone and create a record for my EC2 machine:
require 'fog'
connection = Fog::DNS.new({ :provider => "aws", :aws_access_key_id => access_key, :aws_secret_access_key => secret_key})
zone = connection.zones.get("Z111111QQQQQQQ")
#Create a record:
record = zone.records.create({ :name => "mynewhost.local", :value => ["127.0.0.1"], :type => "A", :ttl => 300 })
while record.reload.status != "INSYNC"
sleep 2
end
#Find that record:
record = zone.records.get("mynewhost.local")
puts record.reload
You'll need to get the hosted zoneID from Amazon (i.e. Z111111QQQQQQQ)
08 Dec 2013
I stumbled upon a really interesting Ruby Cryptography library that uses NaCl library, but I won't explain it here; just go to the Github page for more details.
I decided to use it for mutual authentication, which has been done, but with different mathematical algorithms. RbNaCl does this by using private and public keys between 2 things like people, client/server, etc... Below I will show how to create a client key and a server key then decrypt a message from the client to the server.
Let's first create the keys for the shell client.
require 'rbnacl'
require 'base64'
shell_private_key = RbNaCl::PrivateKey
shell_private_key = shell_private_key.generate
shell_public_key = shell_private_key.public_key
shell_public_key_base64 = Base64.urlsafe_encode64(shell_public_key.to_bytes)
Create the keys for the server client
server_private_key = RbNaCl::PrivateKey
server_private_key = server_private_key.generate
server_public_key = server_private_key.public_key
server_public_key_base64 = Base64.urlsafe_encode64(server_public_key.to_bytes)
Encrypt a message from the client
shell_box = RbNaCl::Box.new(Base64.urlsafe_decode64(server_public_key_base64), shell_private_key)
shell_nonce = RbNaCl::Random.random_bytes(shell_box.nonce_bytes)
shell_nonce_base64 = Base64.urlsafe_encode64(shell_nonce)
shell_message = "This is a valid message from the client"
shell_ciphertext = shell_box.encrypt(shell_nonce, shell_message)
shell_ciphertext_base64 = Base64.urlsafe_encode64(shell_ciphertext)
Server needs to decrypt the message from the client
server_box = RbNaCl::Box.new(Base64.urlsafe_decode64(shell_public_key_base64), server_private_key)
shell_decrypted_message = server_box.decrypt(Base64.urlsafe_decode64(shell_nonce_base64), Base64.urlsafe_decode64(shell_ciphertext_base64))
puts "Valid message from client"
puts shell_decrypted_message
3 things to note:
1. The nonce is special and needs to stay random, do not use the same nonce for new messages as that will, according to the documentation make the cryptography weak.
2. I'm using base64 to be able to export and import private and public keys, as you cannot send over raw bytes to the server or client, so you need to give them some ascii to decode to use the server's public key.
3. I tried to see if there was a file size limit and got up to 100MBs with no issue in encrypting the file and decrypting it later. I didn't go higher as I had no use, but I didn't see any limits on how large of a file you can encrypt.
Here's how you can keep the same private key instead of generating a new one each time:
shell_key = "kPeIKHWcKqX2baXdoA8GO-8fDVBE_9qnRQn-IbcKd6E="
shell_private_key = RbNaCl::PrivateKey.new(Base64.urlsafe_decode64(shell_key))
08 Dec 2013
I needed to create a sortable list within a webpage that also made sure it updated the order of that list via the id of the form input. Below is the javascript and html page and this is the jsfiddle showing how it works.
Here is the html:
<h4>Order of your favorite Animals</h4><br />
<div id='sort-list'>
<div id='listItem_1'>
<li>dog<input id='1' type='text' name='1' value='1'/></li>
</div>
<div id='listItem_2'>
<li>cat<input id='2' type='text' name='2' value='2'/></li>
</div>
<div id='listItem_3'>
<li>mouse<input id='3' type='text' name='3' value='3'/></li>
</div>
</div>
Here is the javascript, you also need to use jquery and jquery UI (sortable)
$(function() {
$( '#sort-list' ).sortable({
stop: function (event, ui) {
var inputs = $('input');
var input_length = inputs.length;
$($("input").get().reverse()).each(function(idx) {
var newvalue = $(input_length - idx);
$(this).attr('id',$(newvalue)[0]);
$(this).attr('name',$(newvalue)[0]);
$(this).attr('value',$(newvalue)[0]);
});
}
});
$( "#sort-list" ).disableSelection();
});
Then you can drag and drop the list, you'll also see the values in the form update to the proper number the input is in. This way it helps when it is submitted.
02 Dec 2013
I've already posted how to use the ruby gem fog to create an RDS instance, now I will post how to create a manual snapshot of an RDS instance, in case once per day is not good enough. Below you'll find the code to do just that.
require 'fog'
connection = Fog::AWS::RDS.new(:aws_access_key_id => access_key, :aws_secret_access_key => secret_key, :region => "us-east-1")
result = connection.create_db_snapshot("my_database_instance", "my_database_instance-manual-1")
if result.status != 200
puts "RDS manual snapshot request failed"
end
#Now check the snapshot's progress
mydb = rds.servers.get("my_database_instance")
my_snapshot = mydb.snapshots.get("my_database_instance-manual-1")
while my_snapshot.reload.state != "available"
print "."
sleep 20
end
puts "Created snapshot successfully"
Just a couple notes, manual snapshots need to be cleaned up "manually" and you'll want to validate this backup (as always).