Sandboxing
The recent release of xScope 3.0 is our first product to use the new application sandbox that will soon become a requirement for submission to the Mac App Store. I’d like to share some experiences and advice on how to use it in your own products.
First off, Ivan Krstić and the rest of the team at Apple have done a great job in making the whole process easy to implement. Adding entitlements and signing your code will be the least of your worries as you transition to the new sandbox.
Of course there are some applications that have a harder time than others: primarily if those apps require access to all or part of the filesystem (think about syncing data with Transmit, for example.) Apps that make use of AppleScript for inter-app communication will also have a difficult time: this includes our Take Five app. Apple is actively listening to developers who are encountering these types of issues, so if you haven’t filed a Radar yet, quit bitching.
Speaking of Radar, we encountered a fairly nasty problem after launching xScope. Many of our customers are designers and developers who love SSDs. It’s common to use a symlink in your Home folder to put big datasets like Pictures, Music and Movies on a separate hard drive. When you do this, folder access in the application sandbox container breaks. A small number of users who use symlinks are also getting crashes after launching the app that was downloaded from the Mac App Store:
xpchelper reply message validation: sandbox creation failed: 1002
Container object initialization failed: The file couldn’t be opened.
We also encountered a problem when using Sparkle to update an app running in a sandbox: an app can’t update its own binary. Changing Sparkle so that it uses an XPC service is a major architectural change, so we decided to remove the sandbox for the version we distribute on the website.
Besides being the path of least resistance, it also gives us a version of xScope that doesn’t run into the sandbox bugs reported above. I highly recommend that you give yourself this option for any customers that experience sandbox related problems.
All things considered, adding an application sandbox has been a fairly smooth transition. But it’s also clear that we’ve only just begun putting the genie back in the bottle.
Updated January 27th, 2012: The bug reported above is a duplicate of Radar 9865143.
Posted on January 23rd, 2012 in Advice, Development
Homebase
A lot of people I know and respect have been commenting on problems associated with the iPhone mute switch:
John Gruber – On the Behavior of the iPhone Mute Switch
Andy Ihnatko – Unmuting on The Mute Question
Marco Arment – Designing “Mute”
Guy English – Mute This
Both sides of the argument have valid points-of-view. This really is a situation with no right answer given the current mechanisms.
That got me thinking that there might be something missing that’s causing this ambiguity. I’ve come to the realization that this is a problem bigger than just alarms going off at inopportune moments. What we really want is for the devices in our pocket to behave differently depending on where they’re physically located.
Let’s imagine a new feature in iOS called “Homebase”. A user would be presented with a simple UI that lets them select a location that’s a “safe” environment. After the setup is complete, your Homebase would be recognized by GPS coordinates and/or available Wi-Fi networks. The important thing here is that the user gets to define where they feel safe with their device.
With that information developers can make smarter decisions:
- Alarms that go off while the mute switch is on make noise at Homebase and just vibrate elsewhere. There’s no need to worry about alarms going off in public places (such as concert halls) and you won’t oversleep when you go to bed with a mute switch on.
- The lock screen doesn’t need to display a Passcode lock at Homebase. People who use the Remote app with their Apple TV will no longer be annoyed by an unnecessary security precaution, nor will folks forget to turn their Passcode lock back on when they leave for the local bar (where they’re certain to get a Poopin’ tweet.)
- Apps, like Find My Friends, could use cached Apple ID credentials at Homebase and avoid asking the user for them over and over and over and over again.
Of course, this feature is needed most by people who don’t even know the Settings app exists. It’s my opinion that if developers are careful with this additional knowledge about the user and device, default behavior can be adjusted appropriately without additional confusion. It’s analogous to the Energy Saver on the Mac: people don’t question why the screen dims when the power cord is removed because it just “makes sense”.
The examples above use Apple’s own apps, but the Homebase status would be useful for third-party developers, too.
If you’d like to see something like Homebase in iOS, please be sure to file a duplicate Radar.
Posted on January 16th, 2012 in Development, Miscellaneous, Observation
Un-Trusteer-ed
The bank we use for our business account recently mandated the use of a product called Trusteer Rapport while accessing our information online. Although Mac OS X doesn’t have any problems with “increasingly sophisticated malware in the online environment”, I do need to periodically check our accounts and transactions so I proceeded with the installation.
The first warning sign was after starting the Installer: I was prompted for an administrator password indicating that this software wanted to run from protected areas of my system. Being a developer, I immediately dug into the installer scripts and configuration files to see that the app is placing items in the Rapport/bin, PreferencePanes, LaunchDaemons and LaunchAgents folders of the main system Library folder. The launch folders indicate that the software will be run whenever my Mac is restarted and will be able to do things a normal user would not (because of elevated permissions.)
I placed my security concerns aside as I need to access my bank website, so I went ahead with the installation. Afterwards, I was directed to a web page describing the new software.
Again, as a developer, my first thoughts were suspicious ones. From experience, I know that it’s not easy to modify Safari’s user interface in the way that Trusteer was doing. My guess that there would be a new, always active, background process was confirmed by the presence of “rooksd” in my process list.
However what happened next really opened my eyes. Safari crashed.
Of course that, in and of itself, isn’t the end of the world. But I was surprised to see a new library named RapportUtil1 while looking at the Safari crash report. It was pretty clear that the new Trusteer software caused the crash. But how?
As a longtime Objective-C developer, I know a thing or two about “method swizzling“. In a nutshell, this allows a developer to replace the functionality of code they don’t have direct access to (typically in the system or other frameworks.)
Seeing “_nsapplication_sendEvent_override” tells me that Trusteer is using this technique to change the behavior of Safari. The function that is being affected is -sendEvent: — the part of every Cocoa application where mouse, keyboard and other input is routed.
Method swizzling is a dangerous activity. You have to make assumptions about how some other code, that you’ve never seen, is behaving. You also need to think about how that code might change in future versions. There are extreme cases where this technique can be effective: overriding the default behavior of my web browser is not one of them.
It’s clear that the folks taking control of my browser aren’t as clever as they think. Do you see a common theme when you search Apple’s discussion forums for “RapportUtil1“?
Even more troubling is the method being overridden: every key press or mouse movement is first being sent to Rapport and then forwarded onto Safari. Since this happens often, the intruding software can do pretty much whatever it wants, whenever it wants. And remember that part of this package is running with elevated permissions in the background.
After mentioning my findings on Twitter, I got back some very interesting replies. Graham Lee (@iamleeg) pointed out that I’m not the first developer to question the technical merits of this software. But then Peter Hosey (@boredzo) dropped the real bomb. Trusteer tacitly admits to recording my password. That’s easy to do when you take control of -sendEvent:.
Essentially, my bank is asking me to install is a keylogger. Just so they can warn me not to use the same password on suntrust.com and playboy.com.
Hopefully, the engineers behind Rapport are smart enough to be using hashed passwords rather than clear text. And hopefully none of the personal information Safari has access to is being forwarded to the Trusteer servers. And hopefully they’re not recording how many times I visited playboy.com last month. But that’s beside the point, because as a closed source product, no one can audit their activity. That’s not true with Safari.
Oh, and there’s one other thing: the Rapport software isn’t supported on Lion. One of the tenets of method swizzling is to test your software early and often with any new release of the system or framework that it’s modifying. As a developer, you need to be proactive about fixing any problems that pop up in the code you are overriding. Not doing so is irresponsible and puts your users at risk. The last update for Rapport was in 2009.
(One could speculate that the new privilege separation architecture for Safari in Lion is causing Trusteer’s developers a lot of headaches. The other tenet of method swizzling is to remember that it’s not a matter of if your hack will break in the future, but rather when it will break and how painful it will be to fix.)
Needless to say, I have uninstalled this software and will never be installing it again. I would recommend this course of action to any end user.
But that leaves me with a problem: how do I access my bank’s website? I have three options:
1) Find another bank. This is a difficult choice, as there are many systems that are hooked up to this account: ACH transactions for sales via iTunes, bi-weekly payroll, automatic payments for services, etc. I’d also like to give SunTrust a chance to reconsider their position in requiring this software (they will be getting a copy of this report.)
2) Use the telephone. I can call the bank when I need the information. Sure they’ll get tired of hearing from me, and it will cost them more for customer service, but it’s their choice to require Trusteer Rapport.
3) Run the Trusteer Rapport software in locked down environment. Once it’s supported on Lion, it should be possible to create a virtual machine that that will only be used to access the bank website. Needless to say this is inconvenient, a waste of resources, and severely limits my ability take advantage of my bank’s services.
In closing, I’ll leave you with one final irony: I will never be able to access my bank’s website from what is arguably the most secure computing device in existence today. That’s because the iPad is not a supported platform. Apple only allows third-party applications to run in a secure sandbox where they can’t affect other applications or the operating system. What you’ve seen above is exactly the reason they’ve done this.
Posted on August 1st, 2011 in Advice, Development, Miscellaneous, Observation
The Rise and Fall of the Independent Developer
I’m old enough to remember a time before the Internet. I know what it’s like to develop software both with and without a worldwide network.
Little has changed with the process of software development since the 1980’s. Of course there have been improvements in our tools and techniques, but the basic act of creating software products is much the same. What has changed dramatically in the past 30 years is how we distribute our creations.
In the days where software was distributed on magnetic media, such as reels of tape, cassettes, or floppy disks, it cost a lot of money to get the product to a customer. As a result, large companies and software publishers were the only ones with the financial resources to get these applications into a retail channel. There were very few independent software developers; and those who did exist were very small operations.
Then along came the Internet and everything changed. Distribution was suddenly cheap.
I remember a conversation with my good friend Cabel Sasser a few years ago. He and I were reminiscing about our first foray into online distribution and were surprised that we had the same initial reaction: “Holy crap! We can put our software on the Internet and people will actually buy it!”
Many other developers had this same experience and began leaving large companies to work on their own. Making a good living while having the freedom to work on their passion was a great life.
Now distribution is going mainstream with the App Store. And it’s already begun changing the lives and businesses of independent software developers. On the surface, it all looks good. There are more customers, increased revenues, and many great new products.
But this expanded distribution is also putting our business at risk: there are people in this new market who claim a right to a part of our hard work. Either by patent or copyright infringement, developers are finding this new cost of litigation to be onerous.
The scary part is that these infringements can happen with any part of our products or websites: things that you’d never imagine being a violation of someone else’s intellectual property. It feels like coding in a mine field.
From our experience, it’s entirely possible that all the revenue for a product can be eaten up by legal fees. After years of pouring your heart and soul into that product, it’s devastating. It makes you question why the hell you’re in the business: when you can’t pay salaries from product sales, there’s no point in building it in the first place.
So, just as in the days of magnetic media, the independent developer now finds him or herself at a point where it is again becoming very expensive to distribute their products to a mass market. This time the retail channel itself is very cheap, but the ancillary costs, both financially and emotionally, are very high.
And, of course, only large companies and publishers can bear these costs. My fear is that It’s only a matter of time before developers find the risks and expenses prohibitive and retreat to the safety of a larger organization. We’ll be going back to square one.
Over the years many of the top selling apps have been created by independent developers, starting with Steve Demeter and Trism at the App Store launch, and continuing to this day with titles like Tiny Wings by Andreas Illiger.
Losing that kind of talent and innovation to a legal system is the real crime.
Posted on July 13th, 2011 in Business, Development, Opinion
Predators
Dear Steve,
I’m one of the developers that is affected by the Lodsys patent infringement claim. I’m writing not to beg for your support, but rather to give you a better idea of how this legal action affects the average iOS developer.
We’re a small company. We have 12 employees that have created 14 products for Mac and iOS. We have been incorporated in the state of North Carolina since 1999. We won an Apple Design Award in 2008.
We’ve been doing product development long enough to know that legal expenses are just a part of doing business. But as we both know, the costs of patent litigation can be staggering. As a small company, we don’t have the resources to defend ourselves, so that leaves us with one option: to pay a licensing fee.
And that worries us and every other iOS developer we know.
In and of itself, paying half of a percent of our App Store sales to Lodsys isn’t going to put us out of business. The fear we have is that this is the first step on a very slippery slope.
It’s well known that the top titles in the App Store can earn tens of thousands of dollars per day. There are many predators with dubious patents who see dollar signs when they look at the flock of iOS developers.
What these predators don’t realize is that for every developer who’s earning millions, there are many thousands who are earning much less. This backbone of the iOS ecosystem is doing well with work we love, but that is very much at risk with increased legal costs. We wonder what happens when these predators discover that the earnings from these apps are much lower than they expect. Will the licensing fees increase as a result? Will our next infringement be 5%, 10%, or more?
Of course, this is also a slippery slope for Apple. Taking on a legal burden for an entire platform is not something we would want to do, especially when the root of the problem is a screwed up patent system.
We love developing products for iOS and the Mac, but this legal mess has already started killing that enthusiasm. Apple has revolutionized the distribution of software via the App Store and that has been a great boon for smaller developers. It makes us furious that these greedy predators can put all of that at risk with patents.
Thanks for your time,
Craig Hockenberry
Posted on May 23rd, 2011 in Business, Development, Miscellaneous
A disappointment…
When we released Chameleon at the end of last month, we thought we’d try something new to raise funds for the project. Selling T-shirts and stock icons seemed like a pretty good way. Boy, were we wrong.
Since March 22nd, we’ve sold 17 shirts. If you do the math, that will pay for less than a week of our time. And considering that we’ve already spent more than a week getting things rolling on Github and preparing a talk at the VTM conference, we’re already running at a deficit.
In summary, we’re very disappointed with how things have turned out. Not because of the funding, but because there’s some potential here that will never be realized. We’ll continue to add things we need for our own products, but don’t expect to see any documentation or bug fixes that don’t affect our own code. Any changes or fixes will get pushed out to the community on a schedule that suits us best: probably at the end of minor release cycles (every few months.)
To those of you who were so generous to contribute to the project, we appreciate your support. Unfortunately, we’re sorry to say that such a low volume of T-shirts doesn’t warrant the time and energy needed to get them printed up. We’ll make sure that the credit card orders are cancelled.
Damn.
Updated April 19th, 2011: To be clear, we’re not disappointed with the Mac and iOS developer communities. The source of our disappointment is that we can’t afford to spend time working on general improvements to the framework.
Updated April 20th, 2011: We’re hearing a lot of “let the community help”—which is great and has already started happening. We will continue to support this aspect of Chameleon: we’re not giving up on the project.
But the area where the community is not equipped to help is with documentation. We have nine months of experience with porting an iOS app to the Mac: no one else is presently in a position to help other developers achieve the same goal. We also know that writing is a long and difficult task (my book took seven months of full-time work.) A lot of time is going to be wasted without this transfer of information: so be it.
So much of the open source community focuses on the details (code) without looking at the big picture (how people are going to use that code.) And if you try to think beyond the status quo, you get called names.
We’re also painfully aware that we’re new to managing open source projects. It’s likely that 13 years of selling software is tainting our decisions, but our hearts are in the right place: we want to help the iOS and Mac developer communities.
Finally, we’ve added a more traditional Donate button on the Chameleon Project page. Thanks in advance for any contribution you can make.
Posted on April 19th, 2011 in Development
Revealing
To date, the Iconfactory’s contributions to open source have been fairly small. That just changed: in a big way.
I’ll be honest here—we’re a bit nervous about this release. We’re not used to revealing our work before we’re completely happy with the results. We’ve spent over nine months getting this far, but there’s so much more to do. We’d like to continue work on this project and get the developer community involved. And that’s going to take time and money.
So please take a moment to check out the Chameleon Project website and buy an exorbitantly priced T-shirt to help us achieve our goal. Thanks!
Posted on March 22nd, 2011 in Development
Great writing, terrible reading
Apple has recently released Xcode 4—a major part of this release is an overhaul of the user interface. Change in your development environment is always a bit disruptive, but overall I think the move towards a single-window environment that adapts to different working modes is a good thing.
But this post is not to debate these changes to the programming environment. Rather, I’d like to discuss the new documentation viewer and how it has become unsuitable for both Mac and iOS development.
Apple’s technical documentation has always been top-notch: well written with just the right amount of technical detail. Unfortunately, the documentation viewer that we use to read this valuable information has been declining in ease of use over the past few releases.
It has gotten to the point where frustration with usability overshadows the excellent content. The best way to describe these annoyances is by example: I often get the feeling that the writers who create this prose don’t understand how we use it. Hopefully, this critique will help Apple create a viewer that’s just as good as the information it holds.
A corrupt index
A developer coming from Xcode 3 will have a terrible first experience with the new documentation viewer. Any previously installed documentation sets are incompatible with Xcode 4. Methods that you know exist just don’t show up:
There are also problems with the Jump Bar navigation stack not being recorded correctly and the browsing history being unavailable (the back button isn’t available when it should be.)
Presumably, there is a corrupt or incompatible index. The workaround is to delete and re-install the documentation set, but this is far from obvious.
Since I currently have three different versions of Xcode installed (and will continue to use Xcode 3 for the foreseeable future), I’m wondering if this corrupted/incompatible index will continue to be a problem. Fingers are crossed, but at least now we know what to fix if it breaks.
Popup hell
When you hold down the option key and click on a symbol in Xcode, you see the following window:
For novices, this window has some utility—it provides a simple way for them to dig into what is probably unfamiliar territory (”What’s a UIWindow anyway?”).
The problem is that this window becomes a roadblock for experienced developers. We know damn well what a UIWindow is: we need to dig into the details of this important class. Maybe we want to know more about the rootViewController instance or look at some of the methods in UIResponder (because we know it inherits from that.) This helpful popup quickly becomes a hindrance.
In previous versions of Xcode, holding down the shift key along with the option key gave you a quick way to avoid this popup help. In Xcode 4, that shortcut is gone.
Considering that this feature can get in the way hundreds of times per day, this is truly popup hell.
rdar://9149588
No methods
Once you get the documentation index in working order and actually make it past the popup help, your next hurdle is to locate the information you seek. Let’s say we’re looking for some background on what happens when a new -rootViewController instance is assigned. We’ve got the page of documentation, but there aren’t any controls to show the methods for the UIWindow class:
Besides being a pain in the butt, this is wholly inconsistent with the behavior in the code editor:
(Note that typing “ro” is enough to select “rootViewController” in the code editor’s popup menu. That, followed by the enter key gets you to the code of interest.)
From a developer’s point-of-view, the header files and the documentation page go hand-in-hand. Make the UI affordances the same and we don’t have to think about whether we’re looking at code or the words that describe it.
With a little more digging, you’ll find that you can get to the rootViewController documentation with the Jump Bar. Unfortunately, it takes a lot more effort than in the code editor: you have to click on the class name, and then move the mouse until the subcategories appear. Choose “Instance Methods” and wonder why rootViewController isn’t there. Then move the mouse back and try Properties.
Bingo (but you don’t feel like a winner.) And forget about navigating these lists quickly and easily with the keyboard as you can with the code editor.
rdar://9149638
Unmanaged complexity
Our final navigation problem is reading chapter-based documentation. These are the crown jewels of Apple’s developer documentation. Titles like The Objective-C Programming Language, iPhone Human Interface Guidelines, and the Cocoa Fundamentals Guide are essential reading for all developers, both beginner and advanced. As I began learning about Xcode 4, of course I turned to the excellent User Guide.
These guides typically span many chapters when sections that cover a wide range of topics. And this is how you navigate through those chapters:
Managing complexity, indeed.
The pity here is that someone in developer documentation has forgotten that a Table of Contents tells a much more important story than the individual chapters. A roadmap lets you visit the destinations efficiently.
To get an idea of how painful this is, try finding the recommended Singleton implementation in the Cocoa Fundamentals Guide using the Jump Bar. I’ll wait. (For extra credit, count how many menus you open in the process.)
Of course the documentation viewer has a search function, but even that’s a bit laborious because you have to click on a lot of disclosure triangles to find the right item in the results. Why aren’t the relevant results opened automatically? (And, yes, option clicking the disclosure triangle can be used to achieve this goal, but the question still remains: why isn’t this the default action?)
Updated March 29th, 2011: Matt Neuburg has discovered that for some documents, search results don’t show where your term occurs; you’re shown a higher-level page, but not the actual page.
The root of the problem here and with the method names in the class documentation, is that a deep hierarchy is too hard to navigate. Present the information in a single list and it becomes much more useful. Imagine how bad the code editor navigation would be if it presented a hierarchy based upon classes, properties and methods. It’s flattened into a single menu for a reason: and those same reasons exist in the documentation viewer.
The Jump Bar is a great addition to Xcode, but it’s true power lies in having a predictable end point. With code, that end point is a function, property or method. With documentation, that end point is elusive: it varies depending both with the type and the structure of the documentation you’re viewing. And that’s a real problem when you’re looking for something.
rdar://9149683
ePub, not PDF
While we’re on the subject of this long-form documentation, why isn’t more of it available in the ePub format used by iBooks? It’s pretty safe to assume a huge majority of Mac and iOS developers have an iPad and like to use it for technical documentation. Searching for “Apple Developer Publications” in iBooks results in only six books. That’s a great start, but there is still a lot of documentation available only in PDF.
PDF is, of course, an option for iBooks. But turns out to be unsuitable because there is no back button. If you click a link in the PDF file, it’s a one way proposition. And for technical documentation, that’s a deal killer.
ePub also has the advantage of better font control and image viewing.
rdar://9149845
Some good news
Fortunately, it’s not all bad news. This new version of the documentation viewer seems to keep track of its place on the page much more reliably than in the past. Gone are the days where hitting the back button put you back as the top of the page (instead of the method or property you were looking at previously.)
This one simple fix will save developers a huge amount of time. Thanks!
Ingredients
This situation with the Xcode document viewer has gotten so bad two developers, Alex Gordon and Jean-Nicolas Jolivet, have taken matters into their own hands. This ultimate workaround is an application called Ingredients.
Ingredients parses the HTML files used by Apple’s own viewer and persists the information with Core Data. The result is quick access to the documentation you need with advanced options to filter and sort to your liking. Recent work by Troy Gaul added an item to the Services menu so a keyboard shortcut can be created to view the selected symbol from any text editor (include Xcode.)
If the problems mentioned above affect you adversely, take a look at this alternative documentation viewer. And please take a moment and file duplicate bug reports using the Radar links above. This is the best way to give Apple an idea of how much this is affecting our daily work. Thanks!
Updated March 22nd, 2011: The developers of Ingredients are now accepting donations.
Posted on March 17th, 2011 in Development, Observation, Opinion
Twitterrific firsts
Why are third parties important in the Twitter ecosystem?
Let Twitterrific count the ways:
- First use of “tweet” to describe an update (see page 86 of Dom Sagolla’s book.)
- First use of a bird icon.
- First native client on Macintosh.
- First character counter as you type.
- First to support replies and conversations (in collaboration with Twitter engineering.)
- First native client on iPhone.
And more.
Posted on March 11th, 2011 in Observation
Mac App Store guide
It’s no secret that the Mac App Store is a terrific new distribution channel for developers. Apple also provides plenty of documentation on how to prepare your app for submission.
Unfortunately, there’s not much information on how to create a product that can also be distributed through more traditional channels, such as your own product website. This guide will help you update your Xcode projects to make it as simple as possible to create products for both channels simultaneously.
Introduction
The basic strategy is to create two build targets: one that creates everything for your own website, and another that creates the stuff Apple requires for the Mac App Store. The examples were all written using Xcode 3, but can be adapted to newer versions as needed.
The build target for your own website will include the Sparkle framework for doing updates. The one for the Mac App Store will create signed code that does a cryptographic check of the license receipt.
Note that these techniques can be used even if you’re only doing distribution on the Mac App Store: beta testers will benefit from builds that don’t require a receipt before launching.
To take you through the entire process, I’m going to use a real world example: I originally wrote these instructions while preparing our Flare product for release. When you see “Flare”, think “MyApp”. Likewise, “Iconfactory” will be “MyCompany.”
Certificate Setup
Before setting things up, make sure that you have two certificates in your keychain:
3rd Party Mac Developer Installer: The Iconfactory
3rd Party Mac Developer Application: The Iconfactory
If you don’t already have these installed