Laughing Meme

I met Mr. Chaturong this morning. It was 6:30am and he was waiting with his taxi and his 7yr old son to drive us to our dive spot. It was a peaceful morning, up a small mountain, some dew still on the broad green leaves, away from much of the bustle of Thailand (if Koh Tao can be said to have any bustle at all). He might have been about 40 years old, I didn’t ask. I did ask if he was from Koh Tao, I hadn’t yet met someone from Koh Tao in the 4 days we’d been here, but I kept asking, here is what he answered.

“I’m from Koh Tao, people come from lots of places, all over, to find work with the tourists, but it was different when I was young. When I was young there were only 700 people on the island, it was quiet, no trucks, no motor bikes, we walked everywhere, it was nice, my dad was a fisherman, sometimes made coconuts, everyone was a fisherman or made coconuts. It was quiet.”

If you need a taxi in Koh Tao his number is 089 0049117.

I gave a talk last October at the First Round CTO Summit on what we’d learned at Etsy about hiring great engineers and in particular great women engineers for our team, and a little bit about the promising results we’d seen from the Etsy + Hacker School program.

The video went up last week, it’s 18 minutes long, it was given in a venue that was originally off the record and aimed at CTOs, and talks primarily about the work that Marc led. Folks seem to find it interesting, which is deeply gratifying.

NB: also when I say “81 of them were women”, obviously I meant “81 of them were men”. This is the problem with giving talks on back to back days on different topics in Tokyo and San Francisco, you aren’t at your most polished.

The slides are on slideshare.

The Atlantic wrote up their interpretation of the talk.

When I first started sketching out the “Minimal competence” blog post in my head, I imagined it as part of a series. The series had three real functions:

1. To layout a theory about our obligations as builders of services and consumers of those services
2. To selfishly hilight the hard problems that Flickr and Etsy had tackled and others had shied away from
3. To contextualize my frustration that Twitter’s search lacked two features that, while hard, were core to the way I wanted to use it, and the retreat from which felt like cowardice.

On point #3 those two missing Twitter features were:

1. The ability to search over all time, not just a fluctuating window of 12 – 72 hours.
2. To scope those searches to people you follow

Now I’ve built both of those features for much smaller, slower moving corpuses (after all Flickr is only up to photo id 8 billion and something, and most of Etsy’s items expire after 4 months), so I both knew it was possible, and also could only speculate on how you’d do it on top of a corpus like Twitter.

Which is all a long way of saying, Twitter now has both of these features, socially scoped searches that go back over years of tweets. Plus the ability to download your archives.

Good fucking work Twitter! (here’s hoping others follow)

I was talking with Aaron and Blackman a week or so ago about the state of reverse geocoding. This is the business of turning a lat/long into a named place. Besides a neat party trick it turns out that named places have a few benefits over floating point pairs.

1. while technically the space is infinite and the lat/long space is finite, in practice the names we use to call places converge rapidly to a very small set (in any given region), and for whatever reason (natural or historical) seem to have an affinity for being hierarchical. Both good properties for clustering, compression, and discovery.

2. humans don’t tend to think in floating point pairs.

At Flickr we spent a while working on turning the point where a photo was set on map (or whose GPS coordinates were shoved into the EXIF) into a place. The work of reverse geocoding is about taking a point, and finding out which polygon its in. This is a well solved problem. With two caveats:

1. places don’t have neat boundaries, but overlap all over each other. And people disagree about the overlaps

2. even if places had neat boundaries, and people agreed on them, availability of information about those boundaries is variable at best.

Also it turned out not many people cared, and the ones who did tend to care extraordinarily when we got it wrong. And they cared most passionately about the things that were hardest to get right, points near borders, contested neighborhoods, etc.

Most folks dealing with geo presently bypass this whole problem, and instead go straight to human places, named things, a bar, a restaurant, a rest stop on the side of a highway. And Foursquare has done an amazing job of aggregating and sorting out human attention (and intention) around these places.

But that doesn’t mean the problem is solved, merely deferred. Not all of life’s interesting moments take place in bars (just most), and when aggregating information across multiple steams collapsing onto human labelled places is challenging.

Plus, it’s just dumb that a 100mil+ people carry GPS device in their pockets and we have to buy expensive proprietary data to find out about the shape of where we live.

Anyway, here’s the pitch

Are you shipping a location based mobile app? Would you like to increase engagement? Consider adding a quiz to your app asking people whether they’re in Williamsburg or Northside? Are they in the Bay Area or the East Bay? Is this the Burlingame or the Pennisula? And then publish that aggregated data. People love quizzes, Flickr released a piece of software for turning those answers into shapefiles, the answers change over time, and paying someone for this info is silly. But seriously, if you’re presenting people geographic data, ask them sometime if you’re getting it right.

“People think entrepreneurship is about sitting in a dreamy studio gazing out at nature whilst sipping a latte. In reality, it is a place like your studio, or sitting in your dining room in your pajamas at 10 am looking for something better than a crayon to write shipping weights on your packages with. ” – Caroline of Pease Blossom Studio regarding No Sleep Till Christmas: Etsy Sellers’ Haggard Holidays

“I’d like to find a site that invites you to populate your own archive – a bit like Facebook does for you on its timeline, but in a more customisable way, and in a way that allows you to share with people you choose. Anyone know of such a thing?” – from the comments

We got married two weeks ago today. It was the wedding we always wanted. After 13 years you don’t expect it to change much about your daily lives. And it doesn’t change much, but there are subtle (awesome) changes. For one, I’m wearing a ring.

I’m just now adapted to where I’m no longer aware of the sensation of wearing the ring. I don’t feel it on my finger unless I focus on it. This is an odd reminder of how much of what we take for objective reality is the subjective pastiche of our brains survival oriented signal prioritization routines. Once or twice a day I flash panic that I’ve lost my ring when I realize I can’t feel it on my finger, milliseconds of active querying my finger for touch sensations followed by a quick glance.

I wonder when I’ll internalize the odds sufficiently to realize that even though I can’t feel it, the ring is there. I wonder what other things I’ve come to assume exist without any sensory confirmation. I all of a sudden realize how people lose their rings. There seems to be a metaphor in there for relationships that I’ll leave unexplored today.

Do other people’s wedding rings have magic powers? Am I suggestible to the idea that a ring should be magic because the literature is so full of magic rings, or is the literature so full of magic rings, because rings are inherently suggestive of being imbued with magic?

Mine stores memories. A small handful of scenes, glimpses really, mostly from our wedding day. Glimpses that come back to me in full immersion when I focus on my ring. Is that normal?

Is it normal that as I say these words to myself I’m wondering what the storage density of something the size and shape of a ring would be? You need to make sure to leave space for a Bluetooth antenna because putting a jack of some sort on a ring is going to be awkward and uncomfortable. How far out is the system that allows me to access memories stored in metal (high information density already) visually? (without the benefit of magic)

Anyway.

Married.

I could get used to this. But hopeful not too used to it.

It’s conventional to sneer about “people living to take photos” rather then living to live. Going to a party/concert/art/explosion to photograph it rather then experience it.

I’ve never really bought that. (see also: working at Flickr) I tend to be skeptical by default of activities that arise (rather then are marketed) on the edges and are denigrated by folks with cultural weight.

But it struck me tonight that I think it might be part of a larger shift around identity, fluidity, and information flow.

Here is the early 21st century we’re ravenous information omnivores. We also live in uncertain times, where normalcy is rapidly overthrown and little is predictable. You can (almost) imagine a world in which you could predict your life rolling out before you, I’ll live here, I’ll do that, I’ll know these people the rest of my life, we’ll have the same shared set of stories. Given the retreat from that world, the documented individual experience aren’t bragging so much as tentacles we extrude to catch moment of shared sameness.

But what about all those photos we shoot that we don’t even share, what’s happening there?

I’m not entirely sure, but reading 24 Hour Bookstore tonight, and having to flip back a few pages to hilight a passage that I was still thinking about, I realize that this bookmarking behaviour is probably a form of wayfinding and partially a form of commons. I think as a generation we’re actually pretty good at imagining the idea that small contributions of order actually improve the world, and that also small bits of bookmarking will help us find our way back through the river of information we’re deluged with.

Or at least something is making it deeply compelling, and bragging is an insufficient answer.

“Let me tell you about the kitchen, it’s small, everything has to be made fresh, that guacamole, we make it when you order the sandwich, it hasn’t been sitting around. The sauces, the sauces are all less then two days old.” … “I work here, I should know everything about this place. I should know where we buy our produce, and where the wood of that table came from.” … “It came from a single tree in upstate New York, all the wood in this bar came from that tree.” … “What type of tree is it? You see that tall skinny guy at the bar, with his mother? He’s the owner, I’m going to go ask him, I’ll be right back.” – The Randolph

Ads are an ugly business. You barter away functionality, aesthetics, privacy, and performance for a marginal money maker predicated on using manipulation to get people to spend money they don’t have on things they don’t want. If you’ve ever experienced an old favorite website slowly descending into monetization (my canonical example is Alta Vista), you’ve experienced this viscerally, an old favorite slow selling off bits of itself for a few more hits of cash.

Then Google came along, and they went deep, they created a narrative of transcendental advertising. Advertising so good you wanted to see it. Advertising that was net positive. Advertising that would cause you turn off your ad blocker. And if you’re in an advertising supported business you probably even believe the narrative at some level. Ignore the data about who clicks on ads and why, or the insane degradation of most revolutionary communication medium since the printed word into SEO/SEM spam farms. Transcendental advertising, advertising as liberator, advertising for advertising’s sake, advertising as a higher calling. This is what I call “business transcendental”. A philosophy that is tied to your paycheck.

Watching folks responses to the iPhone 5 “Lightning” connector got me thinking about this. Apple has beautiful, breath taking reasons for launching a new connector. It’s innovative, it opens up previously unexplored options that most of us can’t even imagine yet. It’s the product of R&D by some of the best and brightest in the business, like the touch sensing pixel screen or the new thinking aluminum case. But it’s also planned obsolescence. Planned obsolescence is an ugly business. Uglier then advertising. I think, unlike advertising, most of us still recoil in disgust at gratuitous examples of planned obsolescence. Which is why transcendental planned obsolescence is so gut wrenching. Planned obsolescence as innovation, planned obsolescence as the pursuit of perfection, planned obsolescence as identity politics. Google is in the business of biz-transcendental advertising, Apple is in the business of biz-transcendental planned obsolescence. But the underlying business is as optional, and ugly as it ever was, and the transcendence is an illusion.

I backed App.net. The current iteration isn’t really compelling enough for me to use it, but I recently was deeply frustrated by the lack of edit on Twitter flipped over to App.net hoping that maybe it had added this most humanizing of features, but I was disappointed.

The Twitter that exists today is only one of the various Twitters that were posited over the years, it would be a shame if App.net cargo cults this Twitter rather exploring the space of Twitters.

Features from a handful of alternate Twitters:

• the ability to edit a tweet. There are several patterns in community software for handling the “I responded and then you changed what you said” pattern. One of then is versioning. The other is a short window of edits. It’s a question of balancing how much you prefer the conversational integrity vs the benefits of a little hypocrisy to a person’s self expression.

• archives. is this the record of your life and of culture, or a system for transmitting unrelenting now-ness? Dated archives are key toward setting the expectation that these items you share won’t disappear behind an event horizon of fuzzy human memory, available only in the vast archives of folks trying to sell you things. also, lose the relative dates after a little time has past, say like 2 days. Twitter uses relative dates because it was the preferred method for displaying dates in the Rails community, and it seemed pretty slick in 2006. (for point of reference in 2006 you were still using Myspace)

• personal context/useful search. is it a platform for brands/celebrities/robots/blowhards to broadcast or is it a place to connect with a more intimate group? Twitter’s brilliance is in mixing these, but their bread is very much buttered by the brands. This is surfaced both in their disinterest in giving you ways to organize your view (lists) and also in not providing search context (within folks I follow, my tweets, my favorites, etc). of course Twitter did roll out search from people you follow. I love it. So thank you whomever got that out.

• privacy. I was an early thorn in Twitter’s side about supporting the privacy settings. But honestly it was just always too much work to respond manually to follow requests or to maintain two separate accounts. Per status privacy and per status geo-privacy would go a long way towards changing the nature of what people share on Twitter away from re-publishing Mashable headlines.

• annotations. the ghost feature that lived it’s too brief days in the Sun. Twitter works alright as a “magic word distribution system” (to steal Aaron’s description), but briefly there was the promise of it working exceptionally well. That was annotations. Structured data that could flow along side a tweet indicating that this was me discussing what I was listening to, what I was eating, two robots discussing the weather, part of a larger narrative arc, etc. Metadata that could be displayed or ignored by clients as needed, consumed by listeners as they desired. It makes status casting into something with the potential to surprise you with it’s uses rather then to re-tread. To the extent that Twitter extracts entities like URLs, identities and hashtags already you get a sense of how powerful this could be. It both upends and plays well with the OpenGraph inspired resurgence of microformat based structure data sharing.

• federation. just going to leave this one here.

• heterogeneous sources of statuses. a tweet is in someways the irreducible atom of web content. if in the era of monied interests (everyone since Postel) we can get interop on anything ever, it should be on tweets.

Those we a few I’d had my heart set on a various teams over the last 6 years. I’m sure there were others. Re-building Twitter without exploring any of the explored alternatives seems like a waste.

Openness has always been my favorite trend.[1] At Etsy we talk about it as part of our “generosity of spirit” value. Just wanted to call out how much I’m loving seeing this trend on the blogs and engineering blogs across the industry.

Our What Hardware Powers Etsy.com?, was in part inspired by 37Signals Behind the Scenes: The Hardware that Powers Basecamp, Campfire, and Highrise. I’d love to see more teams posting in this series.

And our recent post taking a non-technical approach talking about a series of recent outages, felt a little less lonely, and risky, knowing out there with Soundcloud’s Shoot yourself in the foot with iptables and kmod auto-loading, and Simple’s Transparency.

This is how we get better as an industry.

1. Though sustainability is a close second. Sometimes they flip flop. But that wouldn’t have been relevant to this blog post.

I backed Postcards from Erik the Red’s House a while back. I like the small projects on Kickstarter, they make me happy, as does Iceland, as does someone writing their first novel.

Today my postcard arrived. It was a sunset in Vik. In particular it was of Reynisdrangar, 3 black basalt stacks just west of Vik. I’ve seen Reynisdrangar. In fact I woke up on a black sand beach, surrounded by sheep, looking out at Reynisdrangar.

We pulled into Vik late in the day. We’d been on the road for 5 or 6 days or so at this point in our circumnavigation of Iceland. We’d planned to drive on further, but as the road climbed up out of Vik we were exhausted. Also Jasmine had bought a funny hat at Vik wool. I’m not sure if that’s relevant to the story. We turned off on a side road at the top of the hill looking back over Vik, and drove until the road turned to sand, and we could see waves in the last of the day light. We pitched the tent by head lights, and went to sleep.

And woke up the next morning, unzipped the tent and found a flock of sheep, and Reynisdrangar.

Here’s the postcard

Camping on that beach is one of my favorite memories. Nice to be reminded of it. This is the network, making the world demonstrably better. That is all.

oldtweets is a search engine for the first year of Twitter.

A bunch of folks asked about the how. The Twitter API provides a method for fetching a tweet by ID. So to build an index of the first year of Twitter you need call the api for each ID in the range of IDs 1-20,000,000. 20 million API calls at the rate of 150 calls per hour. Or roughly 15 years of elapsed API time to index year one.

It also helps to know that Twitter is, and has always been, a MySQL shop, and that in the early days there was a theory about scaling databases by using large auto-increment offsets. (I don’t remember what the logic of that was) That started about 6 months in, was turned off for a while, and periodically drifted. So good news the 20 million ID space is very sparse, which significantly cuts down on the elapsed API time. You just need to send tracers into the space to map it.

From there it’s just a question of patience.

The whole things runs on a very small EC2 instance, and it’s on this week’s todo list to get the index running under Upstart, but it hasn’t happened yet. So if it goes away….

Why

I think our history is what makes us human, and the push to ephemerality and disposability “as a feature” is misguided. And a key piece of our personal histories is becoming “the story we want to remember”, aka what we’ve shared. I just wanted my old tweets, as a side effect I got all of them.

Providing an interface to the whole corpus was motivated by the desire that folks would investigate where the social norms arose, exactly like Rabble’s @-reply investigation.

Year 1

I thought year one was a meaningful symbol. It maps to the time when we were figuring out how to use Twitter, and maps to the time when I felt like the service was working best for me and mine as an “ambient intimacy” service.

Additionally after SxSW 2007 the rate of tweeting increased significantly, making the brute force approach even slower.

Something is going on here, but I’m not sure I can figure out what

from: Sarah Thompson sarahjthompson5@gmail.com

An idea for a blog post: Science and Design Behind Music Production

Hi ,

I’m getting in touch with you because I’m interested in writing an article for your blog. I came across your blog post laughingmeme.org while writing for a website on music production. During my research, I’ve found an increasing focus in terms of design as the tools and technology available today improve our ability to customize how create music and collaborate as musicians

Please let me know if you’d be interested in an article this topic. Thanks, and I look forward to hearing from you soon.

Best,

Sarah

And then a bit earlier

from: Brianna Meiers briannameiers8@gmail.com

An idea for a blog post: Aesthetics of Learning

Hi Kellan,

I found the information on your blog laughingmeme.org/ insightful as I was scouring the web for research on historical topics that are relevant to issues in higher education today. One of the biggest hurdles in completing an enriching and useful college degree (especially with different organizations such as online schools bombarding students with options) has been the lack of innovation in learning today. I believe that for students today to truly retain useful knowledge, it’s important to design curriculums keeping in mind aesthetics and learning theory.

I’d love to write a post for you that perhaps blends this topic with something deeper you are interested in for your blog. What do you think? Thanks, and I really look forward to hearing from you.

Best,

Brianna

It’s cool to see a project I backed on Kickstarter, Ai Weiwei: Never Sorry up on the Apple trailers page. But also deeply ambivalent as the promised digital download never arrived, which leaves me feeling used and kind of cheated.

One of the strangest genres of tech blog post is the: “we abandoned our old clearly suboptimal solution, and we have a new solution which we’re excited about, and we’ll assert our reasoning as to why this solution is clearly awesome, even though we don’t yet have any data on how it performs, or any discussion of why our decision making process for picking this solution was different then the decision making process for picking the last solution.”

You know that argument you’ve gotten into about whether pushing out an experimental feature is a good idea that goes something like, “If we offer this we can never take it back, so let’s be careful”? It’s akin to the argument, “It needs to be perfect, because first impressions are everything.” They’re reasonable fears that often morph into stasis, because we forget that change should feel risky and scary, and if it isn’t then we’re moving to slow.

I imagine if you were going to roll out your own currency system that people exchanged real dollars for that would be an example of something you need to make sure you got right because you could never take it back.

One of the reasons Facebook is where it is is because they don’t subscribe to that mental trap. Refreshing to see.

Daniel Abraham’s fantasy trilogy “The Dagger and the Coin”: The Dragon’s Path, The King’s Blood, and forthcoming “The Poisoned Sword” are the most satisfying fantasy I’ve read in recent memory.

It’s familiar and unexpected, engaging and unchallenging, swords and sorcery and fallen empires, and deep history, but done well. Economics, and race dynamics, and religion, and archaeology all play roles. Solid multi viewpoint story telling add some novelty to the heroes journeys.

Also a quite reasonable 12-14 months between new books, the 2nd book lived up the promise of the first, and they’re available on Kindle. Really everything you could ask for if you’re looking for a good fantasy novel. Recommended.

I also tried reading some of this “Poet Books” (officially called the “Long Price Quartet”). They dragged considerably more, but a couple of interesting ideas in there.