This is more of a reminder for myself because I forget this every time…

Please note the -Restart at the end. This will reboot without mercy

Discussion here… social.technet.microsoft.com/Forums/en-US/exchange2010/thread/0aefcc92-dc32-40cf-bf24-341d41974d0e/

Over the past week I threw together a “status board” that I’ve hooked into my crontab.  Since I don’t log into my linux server on a daily basis to check logs/mail output and i don’t always give my full attention to automated messages that come to my main mailbox, a status board is a good compromise.  If everything is in order I can review in a quick glance, if there is an error i know where i need to look into later.  Simple highlighting of errored jobs stands out in a different color.

For me this is also a learning exercise in Go (golang.org/) so the code could be vastly improved.  Essentially, what we are doing is taking command line options to either update a status or output the “database” to html.  The database is just a simple json file living in ~/.status. I left what is meant by “object” and “status” pretty vague to keep this tool flexible for future use – both are just strings.  When the html output is dumped it will highlight in green for a status of “Success” and red for “Fail”.  This could easily be extended to highlight different colors for different status, do a regex instead of a straight match, etc.

Integrating Into Crontabs and Scripts

To integrate into my crontabs i created a simple “if” wrapper that assumes an return code of 0 is good and anything else is bad.  Scripts are wrapped the same way

Crontab

00 06 * * * if someCommandThatReturnsAnErrorCode; then statusboard update "My Thing" "Success"; else statusboard update "My Thing" "Fail"; fi

Script

#!/bin/sh
tar czf /tmp/mytar.tar.gz /myfiles
TAR_RESULT=$?
if [ ! $TAR_RESULT ]
then
    statusboard update "Tar Operation" "Fail"
else
    statusboard update "Tar Operation" "Success"
fi

The Code

Statusboard - https://github.com/zpeters/GoStatusBoard

Improvements

Many improvements can (and probably should) be made to the code:

• Use a html template instead of hard-coded html
• regex scan strings for success or failure instead of hard-coded strings
• include a link for STDOUT and STDERR output of scripts that failed for review right in the html file

Background

When I initially setup my home ESXi server the UPS was an afterthought.  I vaguely knew that ESXi could be setup to trigger a shutdown from a UPS and i figured if I had Vmware Guest utilities installed on all my vm’s that would take care of safe shutdown.  Searching around most of the methods were either network shutdown (my UPS only has a USB connection) or homebrew.  If it’s homebrew already I might as well learn what this is all about and build my own solution.

Base setup

The heart of the UPS monitor/shutdown service will be a small vm running Linux. I chose Centos since it is reasonably light-weight feel free adapt this to your favorite distro-of-choice.

1. Download Centos 6.3
2. Create a new VM in ESX with the following:
   • 512 MB ram
   • 8 gig hdd – Thin provisioned
3. Install Centos as normal
4. Edit /etc/sysconfig/network-scripts/ifcfg-eth0 – set onboot to yes reboot
5. yum upgrade
6. After installation add the USB controller and pass-through device from ESX.  You should see something labeled “APC”
7. yum install usbutils
8. lsusb – see if apc is there

Install APC Stuff

The default package repo in Centos did not have the apcups packages so we need to add “epelrepo” (elrepo.org/tiki/tiki-index.php)

1. Add epel repo
   • rpm -Uvh ftp.osuosl.org/pub/fedora-epel/6/i386/epel-release-6-7.noarch.rpm
2. yum install apcupsd
3. chkconfig apcupsd on
4. edit /etc/apcupsd/apcupsd.conf and set the following options
   • UPSNAME – any name will do – UPSNAME myups01
   • UPSCABLE – usb – UPSCABLE usb
   • UPSTYPE – usb – UPSTYPE usb
   • DEVICE – leave this blank – DEVICE
5. service apcupsd start
6. apcaccess (to test). You should see something like the output below

Email alerts

Next we’ll setup UPS alerts so the UPS can warn you of power outages, battery status, etc. The detail and options available will be different for each UPS to make sure to review the apcupsd manual for details. Remember if you intend to got notifications of power outages your modems, routers, switches, etc. need to be powered at that time

• yum install sendmail
• chkconfig sendmail on
• service sendmail start
• Edit /etc/aliases and setup and alias for root, perferably an in internal and external user ex. root:  user,user@gmail.com
• Run /etc/apcupsd/changme this will simulate a “battery needs changing” alert and should trigger an email to be sent

Install Vmware Tools

At this point we have the apcupsd tools installed, we can monitor the battery and have the default alerting rules setup.  Our plan will be to have a manually specified list of vms and the order to shut them down in.  The purpose for these is we may be running vms that don’t support Vmware Tools, thus can’t do a “safe shutdown” when ESXi shuts down.

When apcupsd triggers a shudown (senses loss of power) we’ll but using our Centos install to control the shutdowns of all of our vms and the ESX server itself.  However we don’t have a good way to tell our vm to shut itself down (it is possible we could add our instance to the end of the shutdown script or do a simple shutdown -h now … this seemed like a cleaner solution at the time)

• ESX Console -> Vm -> Guest -> Install Tools
• mount /dev/cdrom /mnt
• cp /mnt/VMwareTools* /tmp
• cd /tmp
• tar xzvf VMWareTools...
• cd vmware-tools-distrib
• ./vmware-install.pl (choose all defaults)
• reboot and confirm that ESX summary for the vm shows vmware tools running
• Test “guest” reboot through VM -> Power -> Restart Guest

Test Default UPS shutdown

• Safely shutdown all ESX vms “just in case”
• Leave the Centos vm running
• Unplug the power
• tail /var/log/apcupsd.events and watch the UPS events. By default when 3 mins left will initiate a shutdown

Shut Down Everything

At this point we have tested all the pieces of our glorious UPS shutdown system.  Time to put the final pieces in place to do the actual work.

Setup SSH access

To accomplish the shutdowns we will need to run some commands on the ESX server, which means our Centos VM will need ssh access to the server.

Following this guide create a ssh key and add it to the ESX servers authorized keys.  Make sure to follow this guide since the authorized_keys file isn’t in the usual location

 Shutting down the VMs

In ESXi there is a built in command language to allow us to do some of our regular maintenance tasks from the command line.  One of these is to send a “shutdown signal” to the vm.  Since we want to control the order our VMs are shutdown we need to get a list of the VMs and their unique IDs first

Log on to your ESX server and run the vim-cmd vmsvc/getallvms command and note the VM names, IDs and the order you want to shut them down in.

Next create a script similar to the following and place it somewhere on your ESX server.

https://gist.github.com/3909385

Wrapping It All Up

After you’ve tested remote access to the ESX server (run something like ssh root@1.1.1.1 "ls" and making sure you get output) and you’ve done a test run of your shutdown script. It’s time to tell apcupsd to run this instead of the traditional ‘shut myself down’ command.

• edit /etc/apcupsd/apccontrol and find the line for doshutdown
• Replace the traditional “shutdown -h ….” line with ssh root@1.1.1.1 "/scripts/shutdown-all-vms.sh"

And that is pretty much it.  You have a small Centos instance monitoring your UPS battery state, capable of initiating shutdown (or any other commands you wish) on a power outage.  Timing for power outages, how long you wait to shutdown the host, etc. are up to you, however the defaults are reasonable.

Have fun and enjoy.

Bonus – CGI interface

As a bonus I’m throwing in instructions on how to setup the CGI interface that will allow you to see the UPS/Battery status in real time.

• yum install apcupsd-cgi
• chkconfig httpd on
• service httpd start
• edit /etc/sysconfig/iptables (add a line to allow 80 in – copy the line for ssh/22)
• service iptables restart
• edit /etc/httpd/conf.d/apcupsd.conf (comment out Allow from lines and add Allow from all)
• service httpd reload
• Browse to 1.1.1.1/apcupsd

References

• blog.shvetsov.com/2011/11/ssh-into-esxi-5-host-using-public-key.html – How to add SSH keys to ESXi
• kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1014165 – vim-cmd command reference

I’ve recently had the pleasure of testing out Windows 8 on my primary laptop.  Initially I was filled with a bit of dread and foreboding due to some major UI changes.  Having spent a few weeks on it, at this point I have things functioning smoothly and the few UI quirks are no longer a hinderance.

Overall, Windows 8 seems to have taken a stab at the “multi-level” interface in and some way gotten this very right.  By “multi-level” interface, what I mean is an interface that is discoverable by the novice (tiles that function like we expect from iPads and Android devices) but does not hinder the “expert” (most apps/functions can be accessed via shortcuts and “search-as-you-type”, the tiles are just eye-candy).

I don’t consider myself by any means an expert in Windows 8, however after years of various OS’s and a proclivity towards command line interfaces I feel I can bypass the tile interface and start to optimize my keyboard usage.

One of the first mods I make in any environment (Windows, Linux, etc) is to remap the Caps Lock key to be another Control key.  Back in “Killing Caps Lock” (thehelpfulhacker.net/2012/02/15/killing-caps-lock/) I reviewed various methods for accomplishing this.  Sadly my go-to tool, Ctrl2cap does not appear to work in Windows 8.  Luckily, after some searching I did find a slightly more featureful applicaton that will do this – and more.

KeyTweak ( webpages.charter.net/krumsick/ ) does present a bit of a mystery.  I’m not able to find any other official source then what appears to be someones personal site at Charter.  I won’t go over much detail on mapping the actual keys, once you open the interface it is dead simple.  This does allow you to tweak several other keys/settings so have fun.  From everything I can tell this was last updated in 2009.  As a service to everyone out there, I would like to host a copy of the exe and pdf in case the Charter site ever goes down.

Original Exe: webpages.charter.net/krumsick/KeyTweak_install.exe

Original Manual: webpages.charter.net/krumsick/KeyTweak%20Manual.pdf

Hosted Exe: media.thehelpfulhacker.net/bin/KeyTweak/KeyTweak_install.exe

Hosted Manual: media.thehelpfulhacker.net/bin/KeyTweak/KeyTweak%20Manual.pdf

MD5 Exe: 0437c75eb8b510b7d3715f047b6a84ff

MD5 Manual: 700e1d4bf276b4d155a68113c09c71ba

Today I’d like to share a productivity secret with you.  This is nothing mind-blowing, just a simple tweak I make on my systems to speed up my typing.

In nearly all OS’s and major applications the Control key plays a large roll.  Whether it’s CTRL-C to copy, CTRL-L to enter a URL or CTRL-x CTRL-s to save – the Control key get’s a lot of use.  Unfortunately on most standard keyboards it’s in a fairly awkward position, causing you to use your weakest finger to stretch away from the home-row to somewhere very uncomfortable.  We all know the more uncomfortable or unnatural something is the more likely it will be abandoned.

The thought here is simple – get rid of a really annoying and arguably useless key (Caps-Lock) and replace it with the more commonly used Control key.  At first getting used to reaching to the left instead of down will be a difficult habit to break, but I guarantee you after a few days it will becoming totally natural and you will never look back.  Having a larger “target” to hit will your pinky will help with accuracy!

Windows

The solution for Windows is actually rather straight-forward.  Sysinternals has a nice piece of software called Ctrl2cap.  As promised it moves your Control key to the Caps-lock position.  Since it works as a kernel mode device driver you will not have any issues with compatibility between apps, video games, virtual environments, etc.  They all will see the CapsLock as the true Control key.

Download Ctrl2cap

Mac

In recently modern versions of OS X (> 10.4) swapping the caps lock is relatively simple.  Just go to System Preferences, then to Keyboard, select Modifier Keys and change the drop-down by Caps Lock.

Linux

The Linux Philosophy makes swapping Caps Lock and Control a little different.  Unlike Windows and Mac who grew out of a single user environment (one dedicated mouse and keyboard for one user) Linux has a multi-user heritage.  Owing to this there is the possibility of having local and global keymaps for both text-mode and X Windows.  In addition many Window Managers will let you customize your key layout as well.

Rather than list all of the different possibilities for Linux I’ll point you to a RemapCapsLock on the venerable c2 wiki