Urania

One of the most popular posts I made on this site was back in 2009 when I wrote about how I got  the increasingly dated PGPLOT package put together by Tim Pearson running under Snow Leopard.  I decided to update that entry to how I approach things today, using homebrew as a package manager under Mountain Lion.  Honestly, these days I do most of my programming and plotting with Python and matplotlib, but I still have old code that needs to be run sometimes, so getting PGPLOT running under Mountain Lion allows me to keep that old software running.

1. Install X11: You will need to install XQuartz in order to have XWindows support for PGPLOT.  Grab the package installer and install it.
2. Install Homebrew:   Intall the homebrew package manager.
3. Install GFORTRAN: The first thing you need to do after installing homebrew isinstall Gnu Fortran (aka gfortran).  This is fairly simple.  All you need to do is say:

   brew install gfortran

   There is a glitch here.  For some reason, the home-brew version of gfortran doesn’t bother to link the gfortran libraries from /usr/local/lib.  To workaround this,  symbolically linked them using:

   ln -s /usr/local/Cellar/gfortran/4.2.4-5666.3/lib/gcc/i686-apple-darwin11/4.2.1/x86_64/libgfortran.a /usr/local/lib/libgfortran.a
   ln -s /usr/local/Cellar/gfortran/4.2.4-5666.3/lib/gcc/i686-apple-darwin11/4.2.1/x86_64/libgfortranbegin.a /usr/local/lib//libgfortranbegin.a
   ln -s /usr/local/Cellar/gfortran/4.2.4-5666.3/lib/gcc/i686-apple-darwin11/4.2.1/x86_64/libgfortranbegin.la /usr/local/lib//libgfortranbegin.la

   
   The exact path to the original gfortran libraries may change if you install a newer version of gfortran! 
    
4. Once they are installed, you simply have to make sure /usr/local/bin is in your $PATH.
    
5. Install PGPLOT: I grabbed the PGPLOT source code from Tim Pearson’s website (here) and untarred the tarball into /usr/local/src/pgplot, the default location the code expects to be in (based on the instructions in the install-unix.txtfile included in the source code tarball).
   

   sudo tar -C /usr/local/src/ -xzvf pgplot pgplot5.2.tar.gz

6. Install sys_macosx configuration: The PGPLOT source code has various compiler configurations stored in “configuration directories” but it doesn’t come with one for Mac OS X.  I ended up hacking the sys_macosx/ configuration directory to include a gfortran configuration. I have made a tarball of that configuration directory available in the pgplot5.2_macosx_addition.tgz that you can download and unpack into the PGPLOT source directory using:
   

   sudo tar -C -xzvf /usr/local/src/pgplot5.2_macosx_addition.tgz

7. Compile the PGPLOT binaries: At this point, if you follow the instructions in the install-unix.txtfile in the PGPLOT directory you will be fine, baring in mind the configuration you want to use is the “maxosx gfortran_gcc” configuration. However, I will outline the steps I used below.
    
8. Create a PGPLOT library/binary directory:Create a directory to contain the PGPLOT libraries. I created /usr/local/pgplot using the command:
   

   sudo mkdir /usr/local/pgplot

9. Choose the graphics drivers you want:Copy the drivers.list file from the source directory to this new pgplot directory and edit it to match your needs:
   

   cd /usr/local/pgplot
   sudo cp /usr/local/src/pgplot/drivers.list .
   sudo vi drivers.list

   
   (You can replace this last step with emacs or whatever text editor you prefer). You make a graphics driver part of the compilation by removing the leading exclamation point from the line. I choose to activate the X-Window drivers, the GIF drivers (to create GIF images), and the PostScript printer drivers (which you can use to create PostScript versions of plots for publication). Be aware PNG support requires libpng be installed.
    
10. Create the makefile: We now need to create the makefile using PGPLOT’s makemake script. Within the /usr/local/pgplotdirectory execute:
    

    sudo /usr/local/src/pgplot/makemake /usr/local/src/pgplot  macosx gfortran_gcc

    
    which should result in the following output
    

    Reading configuration file: /usr/local/src/pgplot/sys_macosx/gfortran_gcc.conf
    Selecting uncommented drivers from ./drivers.list
    Found drivers GIDRIV NUDRIV PSDRIV XWDRIV
    Creating make file: makefile
    Determining object file dependencies.

11. Create all the binaries: Now you just have to create all the binaries, which is a simple makecommand:
    

    sudo make

    
    Assuming everything proceeds without error, you should then see at the end of the output
    

    *** Finished compilation of PGPLOT ***
    
    Note that if you plan to install PGPLOT in a different
    directory than the current one, the following files will be
    needed.
    
           libpgplot.a
           libpgplot.dylib
           grfont.dat
           rgb.txt
           pgxwin_server
    
    Also note that subsequent usage of PGPLOT programs requires that
    the full path of the chosen installation directory be named in
    an environment variable named PGPLOT_DIR.

    
    At this point, you should (if you are going to use PGPLOT within perl or C compile the C library as well using
    

    sudo make cpg

    
    Finally, clean out all the temporary object files, you don’t need them. Do this using the makefile by typing
    

    sudo make clean

    
    If you want to test if things are working you can run one of the PGPLOT demo programs created in this directory. However, the pgdemo* executables seem hard coded to expect the pgplot libraries in the /usr/local/lib directory, to it might be a good idea to do the following step before trying the demos. 
     
12. Link library and header files: This step is optional, but since most programs (including the pgdemo* executables) don’t look in /usr/local/pgplot for library and header files, I symbolically linked the versions in the /usr/local/pgplot directory to /usr/local/lib and /usr/local/include respectively using
    

    sudo ln -s /usr/local/pgplot/libcpgplot.a /usr/local/lib/libcpgplot.a
    sudo ln -s /usr/local/pgplot/libpgplot.a /usr/local/lib/libpgplot.a
    sudo ln -s /usr/local/pgplot/libpgplot.dylib /usr/local/lib/libpgplot.dylib
    sudo ln -s /usr/local/pgplot/cpgplot.h /usr/local/include/cpgplot.h

13. Making sure I use these PGPLOT binaries: Since I am using SciSoft OSX, I modified my ~/.tcshrcfile to change the PGPLOT related environmental variables after loading SciSoft’s environment

    setenv PGPLOT_DIR /usr/local/pgplot/

    . If you are not using Scisoft, you can place these lines anywhere in your ~/.tcshrc file. If you stick to using bash, then the corresponding lines in the ~/.bashrcfile that you need to create (after setting up Scisoft, if you are using that) are:

    export PGPLOT_DIR=/usr/local/pgplot/

    At this point you have a working PGPLOT set of libraries installed. 
     
14. Installing PGPLOT support in Perl: You can stop here if you just want to use PGPLOT from C or FORTRAN source code. If you want to use PGPLOT from within Perl, you need to go further.
    1. Install the ExtUtils:: F77 perl module: In order to install PGPLOT support, you need to install ExtUtils:F77 first. You can download ExtUtils::F77 here and once you untar the tarball,
       

       tar xzvf ExtUtils-F77-1.17.tar.gz

       
       It can be easily compiled using the following standard perl module compilation steps:
       

       cd ExtUtils-F77-1.17
       perl Makefile.PL
       make
       sudo make install

    2. Install the PGPLOT perl module: You can download PGPLOT here. Untar the tarball.
       

       tar xzvf PGPLOT-2.21.tar.gz

       
       We start as we usually do for Perl modules, creating the makefile using the Makefile.PL script:
       

       cd PGPLOT-2.21

       
       Unfortunately, the Makefile.PL script will create a makefile this creates doesn’t work because it doesn’t call gfortran so we have to change the Makefile.PL script to know about gfortran. So load Makefile.PL and edit the line that reads
       

       use ExtUtils::F77;

       
       to read
       

       use ExtUtils::F77 qw{Darwin GFortran};

       
       Once you have done that, create the makefile using
       

       perl Makefile.PL

       
       Once you have done that, you can finish installing the perl module using:
       

       make

       
       you will see some warnings related to missing type specifiers and non-void functions.  Ignore them and continue
       

       make test
       sudo make install

       
       I was able to get the make test to work once I had the proper environmental variable settings for PGPLOT_DIR (see step 13).

So that is it, I now have working PGPLOT installations with perl support on my Mountain Lion Macs.

I haven’t been doing much image processing in recent months (something that will be changing as I take charge of the MSUM observatory for the next year).  I just noticed that Nor Pirzkal placed a new version of the 64-bit SciSoft OSX online here.  If you are still transitioning from 32-bit Scisoft, this version of SciSoft OSX doesn’t install over the old 32-bit version, so a simple edit of your start up files can allow you to switch between the two versions.

This version makes a variety of updates to IRAF and pyraf and the underpinnings of those systems.  Specific changes I noticed versus the 2012.3.1 beta I was using

• IRAF updated to version 2.16 64bit (from version 2.15 EXPORT) which was released on April 13, 2012.
• STSDAS and TABLES updated to version 3.15 from version 3.13
• pyraf updated to 1.11 (from 1.10)
• Python package changes include

• The ipython interactive interpreter updated to 0.13 (from 0.10)
• The matplotlib plotting package has been updated to version 1.1.0 (from 1.0.0)
• The scientific computing packages NumPy and SciPy have been updated.  Numpy from version 1.5.1 to 1.6.1 and SciPy from 0.8.0 to 0.10.0.
• stsci_python updated to 2.12 (from 2.11)
• The astronomical ephemeris calculator package pyephem was updated to 3.7.5.1 (from 3.7.3.4)
• ScientificPython updated to 2.9.1 (from 2.8)
• The python Monte Carlo stats package pymc was updated to version 2.1a (from 2.0).
• The python database interface package egenix-mx updated to version 3.2.2 (from 3.1.3)
• Cython updated to 0.15.1 (from 0.13)
• New Python packages Quantities (version 0.7), uncertainties (version 1.8), and pymzq (version 2) added.
• Nor also noted in his release notes that “This version fixed a missing link for SuperMongo and updated the STScI Synphot tables.”

Before installing this version, I backed up the old version of SciSoft OSX using the command line

sudo mv /usr/local/scisoft /usr/local/scisoft_old

(you may not need to ‘sudo’ if you have write permission for /usr/local).  Once the backup was done,  I installed this version of SciSoft OSX by double clicking the installer package.  

Minor Tweaks I Made to My SciSoft Installation After the Installation

1. Fixing Owenership Issues: To fix ownership issues with some of the files, I performed a

   sudo chown -R root:admin /usr/local/scisoft
    

2. Updating SAOImage DS9: I noticed that SAOImage DS9 installed by this Scisoft OSX is still at version 6.1 whereas version 7.0 is the current version, so I manually installed the updated version by downloading the MaxOSX 10.7 (Lion) version  DS9 from here.  Once unpacked, there were two files there, ds9 and ds9.zip.  I replaced the symbolic links to the SciSoft OSX ds9 binaries in /usr/local/scisoft/bin with these binaries using the commands when I had the terminal in the ds9 download directory:

   sudo rm /usr/local/scisoft/bin/ds9*
sudo mv ds9 /usr/local/scisoft/bin/ds9
sudo mv ds9.zip /usr/local/scisoft/bin/ds9.zip

3. Living with Enthought Python Distribution: I have been using the Enthought Python Distribution for the last few month, quite happily I might add.  However, SciSoft OSX prefers to use the system’s python distribution and so I have taken to only loading the SciSoft OSX packages into my path when I need them by defining the following command in my ~/.tcshrc (and I use C shell, the native shell for IRAF out of habit):

   alias load_scisoft "test -r /usr/local/scisoft/bin/Setup.csh && source /usr/local/scisoft/bin/Setup.csh"
    

Well, that sucked.  I had my computer start locking up when severing web pages.  Not a hard lock up, but all the httpd processes would become ‘uninterruptible’ and hang.  Eventually this prevented the server for feeding out webpages and required me to hard reboot (aka, push the power button) on the server to reboot it.

As such, I decided to do some spring cleaning and built up the server from scratch, luckily I had backups of everything.  Steps thus far:

1. Break up a mirrored and stripped RAID array of four 2TB drives.  Given the mirroring didn’t seem to be getting me much more than having a simple second striped RAID of two 2TB drives as backup, I switched to two 4TB partitions.  Had to copy the data back onto the new RAID setup from the backup drive. Total time: 7 hours for first copy, then 7 more hours to make backup that used to be mirrored.  Net gain: I now have 2GB of additional drive space available, even with the backups.
2. Backup the old boot drive to a backup partition.  Total time: 5 hours.
3. Wipe a drive and install OSX Lion Server on it. Total time: 1 hour
4. Copy user files and Applications from old drive to new boot drive.  Total time: 4 hours (but ran simultaneous to the second copy of the RAID mentioned above).
5. Set up new server to clone itself to backup every day and let those backups run.  Total time: 3 hours.
6. Setup new web server, including figuring out how Apple sets up server security certificates.  Total time: 3 hours.

It may seem like a lot of time invested, but really, most of the time has been spent copying data from backups… so I multitask and work on other things on my laptop while that is going on.  The big lesson I take away from this.  Having automatic nightly backups is ABSOLUTELY ESSENTIAL to making this kind of changes relatively quickly.  My resistance to making this kind of change would have been much larger if I thought I might lose data and didn’t think I could just switch boot drives and go back to my old set up in case I had screwed things up.

So at this point, I have checked, all my old applications seem to be running and the website is back up.  The only problem I have is that PHP doesn’t seem to be properly loading the SSH2 extension.

A Quebec-er informed me that certain sites in Quebec were not showing up in my Clear Sky Clock widget.  An investigation revealed that JavaScript doesn’t understand Unicode characters in its regular expressions.  A bit of thinking revealed a much simply regular expression that would match the site names from Attilla Danko’s Clear Sky Chart website.  So I have fixed the widget to support all accented characters, and for that matter, any name that the Clear Sky Chart presents.  The accented characters may appear as ‘diamonds’ in the menu, but selecting the sites will work now.

You can download this MacOS X widget from here.

If you have the old version installed, you can simply quit the widget, then replace the installed widget with this one.  You may have to log out and log back in for the new version of the Clear Sky Clock to be loaded after installing the new version.

In mid 2008 I spent some time learning JavaScript well enough to update and fix the Clear Sky Clock widget for the OSX Dashboard. This widget, developed by Joshua Lynch, pulled Attilla Danko’s Clear Sky Chart images (example shown above) and displayed them on the Mac Dashboard. However, it was not parsing the locations properly. I fixed the location parsing and added a built-in legend and the ability to resize the widget and released it under a GPL license.

This morning I was alerted by Bruce Krobusek to the fact that the widget was no longer allowing users to select a location, at least with the current Snow Leopard and Lion releases. I had not seen this problem because once you had the widget configured (as I did under Leopard), the settings were saved. However, new users could not change the location from the default of Minneapolis, Minnesota without a bit of hacking on the preferences files. After a bit of digging around, I found the glitch in the location parsing. I am not certain, but I believe this failure was occurring due to a change in JavaScript. There were no changes to Attilla’s website where the widget gets its list of locations. In any case, I have fixed the bug and everything seems to be working again.

The new widget, version 1.4, is available at my Clear Sky Clock widget website. If you have the old version installed, you can simply quit the widget, then replace the installed widget with this one. You may have to log out and log back in for the new version of the Clear Sky Clock to be loaded after installing the new version.

New version of Scisoft OSX 64-bit beta available

I just noticed today that, over a month ago, Nor Pirzkal placed a new beta of the 64-bit SciSoft OSX online here.  I have not had as much joy with the 64-bit version of IRAF as with the older ones.  That said, this version of SciSoft OSX doesn’t install over the old 32-bit version, so a simple edit of your start up files can allow you to switch between the two versions.

I backed up the old version of scisoft using the command line

sudo mv /usr/local/scisoft /usr/local/scisoft_old

and then I installed this version

sudo tar -C / -xzvf scisoft.current.tar.gz

(you may not need to ‘sudo’ if you have write permission for /usr/local).  I’ll report back if anything interesting changes.  The CONTENTS file only noted that ncftp 3.2.5 had been added.  Not sure why an FTP client was added unless it was needed by some other package.

One note is that I have been experimenting with Enthought’s Python Distribution and have it over-riding the python packages installed by the Scisoft OSX installation, so I may have to play around a bit to get PyRAF working again.  Or I may drop SciSoft OSX altogether (since its Python interferes with Enthought).

STScI makes MacOS X Package Installer version of IRAF available

In the process of looking at PyRAF, I was pleasantly surprised to find that STScI now has a package installer available for installing IRAF+STSDAS/TABLES and all the corresponding files needed for STSCI Python in a single click.  It is only for 64-bit Macs.  I will have to look this over in the future, but it apparently installs everything in /usr/scisoft so it will not collide with anything. However, it also requires a new installation of gfortran which I already know can break my PGPLOT installation.

I run several blogs as well as some WebDAV servers on my MacOS X web server, all of which use passwords to authorize access.  I recently decided it would a good idea to support HTTPS (secured, encrypted HTTP) on my web server so that my passwords for accessing my blogs and webDAV server were not sent plaintext and unsecured over the internet.

Initially I had balked at this because the typical SSL Certificate, needed to encrypt and decrypt the traffic from the web server, cost on the order of $100 per year if signed by a Certificate Authority (one of the companies that all the web browsers come set up to ‘trust’ as signers of SSL certificates).  Such a digital signature is necessary if you want your https connections to be recognized as trusted by all web browsers.

However, I recently learned that it is possible to simply ‘self-sign’ your SSL certificates and so long as you only need the encryption for yourself (for example, when using a password to log in as an admin on a blog), this is a free solution that works.  The certificate will not be marked as ‘trusted’ by any authority, but if you can trust it, it will allow you to encrypt the traffic to and from your web server.

The solution I outline below is partly based on a hint provided at MacOSXHints for getting a full Certificate Authority signed certificate installed (here) and a hint from the CentOS wiki (here) for self-signing your certificates.

I’ll outline the procedure I used below, but as a warning, you need to have a familiarity with the command line and the use of sudo.  I am also assuming you are using Snow Leopard (MacOS 10.6.x).

1. Create Directory to store certificate: Start by creating a directory you want to place the certificates in and then “cd” into that directory on the command line.  I used /etc/apache2/local_certs/ as my directory.
2. Generate the Private Key file: Next, from the command line, generate a private key:
   openssl genrsa -out ca.key 1024
   during the command the output should look like this:
   

   Generating RSA private key, 1024 bit long modulus
   ........++++++
   .......................++++++
   e is 65537 (0x10001)

3. Generate a Certificate Signing Request file: Next you need to generate a certificate signing request. You won’t ask anyone else to sign this, but you need it in order to create the public key file. So from the command line type:
   openssl req -new -key ca.key -out ca.csr
   and you will be asked a series of questions. Your answers should be as follows:
   1. For Country Name (2 letter code) answer with the two letter code for your country, for the United States it was “US”.
   2. For State or Province Name (full name) I answered “Minnesota”.
   3. For Locality Name I entered my city.
   4. For Organization Name I entered my name (Juan Cabanela) since this is a server run by my school, but not managed by my school.
   5. For Organizational Unit Name I just hit “enter” and left it blank, your answers may vary.
   6. For Common Name I entered by server’s URL iparrizar.mnstate.edu.
   7. For Email Address I used my email address. It appears in the certificate so I suppose it could lead to spam, but I figured it was better to give people stumbling across the certificate an email address they could contact for questions.
   8. For A challenge password and An optional company name, I hit enter. I don’t expect to need to get the certificate signed.
4. Generate a Public Key file: The last step in certificate creation is you need to generate the self-signed public key which will actually be what users get when they access your server. The user uses this public key which decrypts the traffic the web server will encrypt with the private key (only accessible to the web server).  To generate the public key file, use the command line to run:
   openssl x509 -req -days 365 -in ca.csr -signkey ca.key -out ca.crt
   This will only make your public key valid for 365 days, you might want to change this if you don’t want to have to generate a new public key every year.
5. Set up the web server: Now you have to edit your MacOS web server configuration to support encrypted HTTPS traffic.  This comes down to editing two files. ou will need ‘admin permission’ to edit both of these files, so you will either need to use sudo with a command line text editor (like vi or emacs) or you will need to use a text editor that supports editing files with ‘admin’ permission. The changes you need to make are the following:
   1. Open the main web server configuration file, /private/etc/apache2/httpd.conf, in a text editor and uncomment (remove the leading ‘#’ from) the following line (it’s line 473 in my installation):
      Include /private/etc/apache2/extra/httpd-ssl.conf
   2. Edit /private/etc/apache2/extra/httpd-ssl.conf and make sure the following lines are set as follows:
      • SSLCertificateFile points to /private/etc/apache2/iparrizar_ssl_new/ca.crt or where ever you have your ca.crt file you created in step 4. SSLCertificateKeyFile points to /private/etc/apache2/iparrizar_ssl_new/ca.key or where ever you have your key file you created in step 2.
      • Make sure the lines containing SSLCACertificateFile and SSLCARevocationPath (these are two seperate lines) are commented out (add a leading ‘#’ to each line). Since these certificates will not be signed by a certificate authority, you have no need to point people to CA certificate files or revocation paths.
6. Restarting the web server: Now all you should have to do is restart you apache web server and see if everything works. From the command line type:
   sudo apachectl restart
   This should restart the web server.  Assuming no error messages appear, you can next test the connection.
7. Testing the web server: See if you can go to your web server using https://your.domain.name.here/ as the URL in your web browser. You will be presented with the ‘untrusted’ certificate and asked if you want to accept it.  In Safari on the Mac, accepting the certificate adds it to the Keychain.  I am not sure about where Firefox or Chrome store the certificate, but they should hold on to it for future use, ensuring your future connections to the web server are secure (if done via “https”).
8. Bonus Step for iOS device users: If you want to add these self-signed certificates to your iOS device, you can d