Yosemite Upgrade Changes Open File Limit

uncategorizedbash, node.js, osx, yosemiteDan

OSX has a ridiculously low limit on the maximum number of open files. If you use OSX to develop Node applications -- or even if you just use Node tools like grunt or gulp -- you've no doubt run into this issue.

To address this, I have this line in my $HOME/.bash_profile:

ulimit -n 1000000 unlimited

And a corresponding entry in /etc/launchd.conf:

limit maxfiles 1000000

That solved the problem until I upgraded to OSX Yosemite, after which I began seeing the following error every time I opened a terminal window:

bash: ulimit: open files: cannot modify limit: Invalid argument

Oy.

Luckily, I a little Google foo yielded this Superuser post (and answer).

So it was a quick fix:

$ echo kern.maxfiles=65536 | sudo tee /etc/sysctl.conf
$ echo kern.maxfilesperproc=65536 | sudo tee /etc/sysctl.conf
$ sudo sysctl -w kern.maxfiles=65536
$ sudo sysctl -w kern.maxfilesperproc=65536
$ ulimit -n 65536 65536    

Then I updated my $HOME/.bash_profile to change the ulimit directive to match that last command, above, and I was back in business.

View all 3 comments

Easily prune your ssh known_hosts file

uncategorizedaws, bash, ec2, programming, sshDan

At some point, you've probably seen this message when you try to log in to one of your servers:

spacer

This is really common when you have Amazon EC2 instances behind Elastic IPs because the IP address stays the same (and probably the hostname, too), but as new instances replace old instances, the new instances' ssh keys are probably different.

But if you look carefully, you'll see that the failure message tells you how to resolve this problem:

Offending RSA key in /Users/[username]/.ssh/known_hosts:5

That means that line no. 5 of the known_hosts file contains the problematic key. So, assuming that you are sure this is NOT in fact a security breach, you can remove that line.

It's a bit of a pain-in-the-butt to manually edit this file, though. You can use sed to do it easily, but if you're like me and you don't use sed all the time, you need to look at the man pages every time you want to use it. That's why I wrote this quick bash script to do it automatically.

Drop that in your PATH and make it executable. Then you can simply type ssh-purge-host 5 to remove line 5 from your known_hosts file.

Hope that's useful!

Leave a comment

Could JXCore Be An Awesome Deployment Tool?

uncategorizedjxcore, node.jsDan

JXCore allows you to turn Node.JS applications into stand-alone executables. One possible use case would be to package up your entire application in an executable and deploy it to production servers, skipping the usual dance with git and npm. If performance is good, this could make for an interesting deployment tool. Deploy by Dropbox? Yup, you could! It's on my list of things to try with a hobby project.

View all 2 comments

New git alias: git last

uncategorizedbash, development, git, proDan

I made a new git alias I'm loving. Maybe you have something similar.

I've added this to my .gitconfig:

[alias]
    last = rev-parse --abbrev-ref @{-1}

This gets the name of the branch you had checked out prior to the current branch. It's like git checkout -, but you can use it all over, such as:

$ git merge `git last`

Resetting NTFS files security and permission in Windows

Resetting NTFS files security and permission in Windows. Life saver for someone (me) who still reluctantly maintains a Windows machine.

icacls * /T /Q /C /RESET

I'm Adopting an Open Source Project. Now What?

uncategorizedoauth, open source, python, twitterDan

On June 11, Twitter retired version 1.0 of their REST API. It was announced well in advance, but I had other things to do; checking to see if my Twitter tools would still work was too low a priority. Until they broke.

When I refer to my "Twitter tools," I mean a Python command-line script that I use to post to Twitter from bash scripts, PHP programs, Node.js programs, etc. That Python script is little more than a wrapper around an abandoned open source library that broke when Twitter turned off API v1.0 -- twitter-oauth. The repository seems to have been emptied, too. Luckily, the library is still available in the Python Package Index: pypi.python.org/simple/twitter_oauth/

So, I grabbed the library, updated it so that it works again, and put it up on GitHub: danmactough/twitter-oauth.

Now what?

Is there a protocol for taking over an abandoned project?

Fargo as a Fluid App

uncategorizedfargo, fluid, mac, osx, outliningDan

I thought it would be cool to have Fargo as a separate Fluid App. For those who don't know, Fluid let's you make a web app into what feels like a regular desktop app (for Mac OS X only). For me at least, this will make it easier to find Fargo among my many open windows and browser tabs -- now it's got it's own icon in my Dock and it's own spot in the Cmd-Tab switcher. It makes Fargo feel like the first-class Mac App that it is.

Here my switcher now -- there's our beloved dino!
spacer

If you want to set this up too, know that I had to jump through a few hoops to get it to work. The difficulty appeared to be a by-product of Fargo needing to get my authorization to connect to my Dropbox account (in technical terms, OAuth is a pain in the ass). The workaround was to get this authorization before creating the Fluid App.

  1. Open Safari
  2. Go to fargo.io to launch Fargo
  3. Follow the prompts to authorize Fargo to connect to your Dropbox account
  4. Close Safari (optional)
  5. Create your Fargo Fluid App (follow the instructions on fluidapp.com/)
  6. Launch your new Fargo Fluid App

Why did that work?!

I'm using a free version of Fluid (for now -- still kicking the tires). The free version doesn't have separate cookie storage -- it shares it with Safari (Fluid is built on Apple's WebKit, like Safari). So, by setting everything up in Safari first, Fluid was able to piggyback on that setup when it created the Fargo app. Neat trick -- but not a long term solution. For example, I want to pay for the full version of Fluid (to support the developer), but I suspect this trick won't work in the full version.

RabbitMQ Queue Auto-Delete

uncategorizedamqp, queues, rabbitmqDan

Breadcrumbs for myself.

Queue config options:

  • exclusive: delete when declaring connection closes. No other
    connection can even access this. Only queues can be exclusive.

  • autodelete: delete when the last downstream thing (i.e. consumers for
    a queue, or queues for an exchange) goes away. Note that this isn't
    bound to channels or connections at all really.

  • exclusive is more frequently useful than autodelete

More about auto-delete:

  • auto_delete queues will delete themselves when their consumer count drops to 0 from a higher number. Otherwise
    they would delete themselves as soon as they were declared...

  • The downside of exclusive queues is that they can only have consumers on the connection that created them. So this doesn't work for shared queues, or queues created by some app that are subsequently to be consumed from by some other app.

  • Auto-Delete queues not getting deleted

Other options:

  • Queue TTL

Fragile Tools

uncategorizedfree, miscellany, softwareDan

Not as much fun as Fraggle Rock.

I'm currently dependent on the following fragile tools:

  1. Twitter
  2. Google Reader
  3. To a lesser extent, Firefox and Chrome

What do I mean by "fragile tools"? Consider Twitter: It started as a fun ecosystem for development, where anyone could whip up an interesting alternative way to interact with the service. Now? Not so much. Remember "track"? Yeah.

And Google Reader seems to have only one developer -- just babysitting it. The API is not documented and can change at any moment, without notice. Plus, Google has been sunsetting services left and right. Having killed off virtually all competitors, when will Google shut down Reader?

Firefox and Chrome are both moving to narrow their feature sets. Both seem incredulous that I might want to enter the URL of an XML file and view that file in my browser window. On the plus side, both are light-years ahead of Internet Explorer or Safari in usability and reliability.

Any of these tools could very quickly stop offering key features or go away entirely. The precariousness of my position as a user of those tools leaves me in a constant state of anxiety.

P.S. This post primarily is intended to test a one of the plugins I'm using. So, it's really just a bunch of conclusory statements without any references or argument.

Making FiOS Actiontec Router and Apple Bonjour Services Play Together

uncategorizedactiontec, apple, bonjour, fios, fixes, mDNS, networking, routers, verizonDan

I have Verizon FiOS at home. It's very fast, but if you want optimal speed and all the features the service offers, you must use the router supplied by Verizon. This week, Verizon replaced my old router with a new one: Actiontec MI424WR-GEN3I. Unlike the older model it replaced, the new router is a Gigabit LAN router and a 802.11n wireless router. That's great! But with the new speed also came a new problem: Bonjour services were't working properly on my home network. Specifically, my MacBook (using WiFi) could not find other computers and printers on the network.

Apparently, this is a common problem.

First, I tried this supposed solution: disable 802.11b mode. No luck.

Then, I saw this suggestion: create an ACL whitelist entry for 224.0.0.251/255.255.255.255 in the IGMP proxy settings on the Actiontec router. That sounded ridiculous, so I kept looking.

Finally, I came across this: disable the IGMP proxy on the Actiontec router. I am loathe to disable a default setting that I don't understand, but this fixed my problem instantly, and I have yet to see any negative consequences.