This is a repost a response I wrote on a private forum to the CEO of a small but growing company looking to make their first marketing hire. As the co-founder of Interstate, a company that sells multi-touch attribution software to marketers, I interact with lots of marketers at companies of all different sizes every week. Previously I saw a lot of marketing organizations from the inside while doing freelance growth consulting with companies like Twitch, Teespring, Everlane, etc. Based on those experiences, here are my thoughts on making a first marketing hire.

Your first marketing hire can be much harder than hiring an engineer, especially now that marketing is becoming more quantitative and marketing and product organizations are starting to blend together. You really need to be careful since there are a lot of folks who can sound like they are good but are actually terrible. First you need to figure out who you are hiring and at what level. Are you looking for someone who is a VP/Director or are you looking for a junior level marketing person to execute on marketing tasks? If you’re looking for a junior level person, what are the things you want them to be doing.

Here is a rough breakdown of skill sets:

• Content Marketing
• SEM
• Social Advertising (Facebook, Pinterest, Instagram, Twitter, LinkedIn, etc)
• Display Advertising (Retargeting, Direct Buys, Programmatic)
• SEO
• Email/Push Notifications
• Analytics
• User/Market Research
• Copywriting
• Positioning
• Landing Page/Onboarding Optimization

You’re going to be really hard pressed to find someone who is good at all of these. High quality junior level folks are probably going to be good at one or two. High quality VP/Director level folks are probably going to be excellent at one or two, good at a few more, and able to intelligently discuss all of them. The worst thing you could do is to hire someone who is mediocre at all of these. Rand Fishkin from moz.com wrote a good overview of this concept of a ‘T Shaped Marketer’ here: https://moz.com/rand/the-t-shaped-web-marketer/

If you’re small you’re probably not going to be able to hire the VP/Director level person right out of the gate ($200-250k+/yr plus significant stock is what the really high quality folks are making now). I’d probably try to focus on hiring a great junior level person who you can hire a VP/Director above as you grow. To do that you’re going to first need to figure out which of the skill sets above you care the most about getting, then tailor your job post accordingly.

I would structure my interview questions in this way:

General Questions

• Test for a general understanding of math. Make sure they grasp statistical significance and LTV. Make sure they will make the right tradeoffs between CPA/Retention on paid acquisition. (You will be shocked how many marketing managers who have run $25k+/mo budgets have no understanding of statistical significance and no ability to analyze data correctly)

• See how they think about making tradeoffs. They should think like a PM – what is the highest leveraged thing I can be doing with my resources – all the time. This is where scenario based interview questions can be great.


• See how they think about growth. (test for a bias towards analytical thinking) Ask them an open ended question about how they would go about improving growth and/or marketing for a particular product (generally not asking about your own product is the best since you’ll be more objective). You should be looking for someone who has a bias towards understanding the metrics around what are making something perform well or poorly before they jump right into making you recommendations. If you pick a product that is truly terrible though as an interview question its probably fine if they throw out some recommendations right out of the gate on first principles, as long as they are able to also explain those first principles.


• Figure out what kind of marketer they are (possibly before the interview). Of the skill sets listed above, which are they good at and which are they bad at? How good are they at the ones they say they are good at?


Specific Questions

• Dive into the areas they say the candidate says they are great at one by one and test their knowledge. This will probably require you to do some research beforehand on what questions to ask and how to dig deep unless you have a lot of experience in that particular area yourself

Good luck!

Paid marketing is a powerful tool, it’s one of the few ways to scale user growth but if not measured correctly it’s also one of the best ways to drain your bank account, fast. If you’re going to use paid marketing as a sustainable growth mechanism you have to know a lot more than if it’s driving new visits, you have to know if its profitable. It seems like something that is simple enough to calculate, but most marketers actually get it wrong. Let’s explore why.

How To Calculate Profitability

Most marketers have settled on a simple metric for measuring how profitable their channels are called return on ad spend (ROAS). ROAS is defined as the revenue a user generates divided by the advertising costs. It is usually calculated in two ways, across all channels together and for each channel individually. To understand the performance of our entire marketing budget for executive level reporting we would calculate it across all channels, but to make marketing decisions about what channels are working and which ones are not we would calculate it for each channel individually and compare the results. In equation form it can be stated like this:

ROAS = (Revenue)/(Marketing Spend)

Back to Algebra Class
This equation looks simple but is actually tricky because Revenue and Marketing Spend use different units. If you think back to your Algebra classes I’m sure you’ll remember having to solve problems that combined units like miles/minute and miles/hour into a single equation. In order to get the right answer, you first needed to change miles/minute to miles/hour. The ROAS equation is similar. One unit is Cost-Per-Visit (Marketing Spend) and another is Revenue-Per-Conversion (Revenue). In order to get the right answer we need to convert revenue-per-conversion to revenue-per-visit.

Calculating Revenue Per Visit
The Marketing Spend part of our equation above is easy to solve since we already know our cost-per-visit but Revenue will require some work. We know our revenue-per-conversion, but to turn that into revenue-per-visit we need to give a value to each visit based on the conversions it contributed to. Visits that never led to a conversion are easy to value, they get a value of $0. The difficult visits to value are those that did lead to a conversion. In order to give these visits values we need to come up with a set of rules that determines how to to value them. We call this set of rules an Attribution Model, it’s the key to calculating ROAS.

Attribution Models

Attribution models are a set of rules that determine how we convert Revenue-Per-Conversion into Revenue-Per-Visit so we can fairly compare our revenues to our marketing costs. There are two basic types of attribution models.

Single-Touch – The Old Way
A single-touch attribution model is one that gives credit to only one visit for a conversion. This is the most common way marketers today attribute revenue. It is easy to calculate but not very accurate.

Some common single-touch models are:
– First Click (give credit to the first visit)
– Last Click (give credit to the last visit before conversion)

Multi-Touch – The New Way
A multi-touch attribution model is one that gives credit for a conversion to a number of visits rather than just the first or last visit. This is the new way that smart marketers are using to attribute revenue. It is harder to calculate than single-touch attribution but is far more accurate.

Some common multi-touch models are:
– Uniform (give credit evenly to all visits)
– Parabolic (give credit to all visits but more to the first and last visits)
– Time Decay (give credit to all visits with more credit going to later visits)

Attribution – An Example

In order to understand why multi-touch is the “new way” lets consider a specific example. We’ll take a last-touch attribution model and a multi-touch attribution model and compare the differences in ROAS and channel performance.

Assume we have 5 users who visit our site for the first time on May 1st or May 2nd, these users then each make a $200 purchase on May 9th with a number of other visits in between their first visit and purchase. The user visits look something like this, with each visit channel color-coded for convenience (Facebook is blue, AdWords is green, Retargeting is purple):

We also have some advertising across different channels over each of those days:

Single-Touch Attribution – Last-Click Model
Now lets check out what last-touch attribution looks like in this scenario. We’ll ignore dollar values for now and say that each conversion has a value of 1 to make this simple. Let’s see how our different marketing channels are credited here. (Facebook is blue, AdWords is green, Retargeting is purple). If we add up the credit for each channel it looks something like this:

As you can see, the channel that each user visited last gets 100% of the credit for the conversion. In this case Facebook was the last channel two times, AdWords was the last channel two times, and Retargeting was the last channel once.

Now let’s include the spend and see what it looks like. Remember each conversion is worth $200. The ‘reporting window’ just means we’re only looking at what is happening during those dates. For example, when in Google Analytics you select to look at the last 7 days, those 7 days are your ‘reporting window’ even though there may be other user activity happening outside that timeframe. Since we only have the 5th to the 9th selected as our reporting window we’ll add up all the marketing spend and all the attributed conversions in that time period. The result looks something like this:

This looks great! From the 5th to the 9th we spent $600 on marketing and made $1000 in revenue on the users acquired from those marketing efforts for an ROI of 1.67x our marketing spend. Based on our last-touch attribution model for this sample of users it appears that AdWords and Facebook are equally good at driving conversions, and each are twice as good at
driving conversions as retargeting is on a per-dollar basis. Before we go tell our boss we need 10x more marketing budget and start allocating more spend to Google and Facebook than Retargeting lets give multi-touch attribution a shot.

Multi-Touch Attribution – Uniform Model
As we did before with last-touch attribution we’ll ignore dollar values for now and say that each conversion has a value of 1. Lets see how our different marketing channels are credited here. (Facebook is blue, AdWords is green, Retargeting is purple). In this case, instead of giving value just to the last click we’ll split it up evenly between every visit that drove a conversion. If we have one conversion for a user and 4 visits as we do in this case, each visit will get 1/4 of the credit for the conversion. The way it works out looks like this:

Now lets include the spend again and see what it looks like. Remember each conversion is worth $200, so we just multiply the partial credit amount (0.25 for all visits in this case) for each visit by the conversion value for each conversion ($200 for all users in this case). The result looks something like this:

That is quite a bit different than our last result, not only is the overall ROI worse 0.83x vs 1.67x but the per-channel numbers actually tell a completely different story. Here’s how last-touch and uniform attribution compare overall:

Our more robust multi-touch model tells us we actually have 50% of the ad performance we thought we did, crazy! Here is how the it looks if we compare by marketing channel:

Our single-touch model would suggest we move budget away from Retargeting and towards Facebook and AdWords since they have almost twice the ROI (2x vs 1x and 1x respectively), but our multi-touch model actually tells a different story with Retargeting performing 33% better than Facebook and AdWords (1x vs 0.75x and 0.75x respectively). It also suggests that we shouldn’t be expanding our budget at all yet, at least not without more in depth analysis on user lifetime values since we’re still losing money on our advertising.

The takeaway here? The attribution model we select can have large effects on how our spend is performing, using a naive model (or no model at all) can leave us in the dark about how our ad spend is performing, leading us to think we are making money while we’re actually losing it.

Implementing Attribution

Prescribing which specific attribution model is the correct one for your company is out of the scope of this post, but any model that gives credit to each visit along the path to conversion (multi-touch) is going to be much more accurate than one that only gives credit to a single visit (single-touch). Moving from a single-touch attribution model to multi-touch one can have big impacts on spend allocation decisions and can significantly improve advertising ROAS. As we saw in the example above.

It’s a lot of work to build out accurate multi-touch attribution, but there are third-party solutions out there that can simplify the process. My company Interstate makes a pretty good one in my opinion! Regardless of whether you build or buy, if you’re going to use paid marketing as a sustainable growth mechanism you absolutely must have a clear understanding ROAS. In order to that you must move past single-touch attribution, without it you might as well be guessing.

In the same way under-investment in technical rigor can result in long-lasting negative effects on a codebase when left unchecked, under-investment in metrics can lead to long-lasting negative effects in your product organization.

At Quint Growth we work with companies every day to improve how they measure, understand, and act on data. We review many metrics setups and talk to companies about how they approach the problem of understanding and optimizing growth. We find that key questions that companies need answers to in order to make important product and business decisions are often either poorly understood or require quite a significant amount work to answer. This doesn’t vary by company stage, we’ve seen this just as much in small companies as we have in large successful ones. I’ve never spoken to a single company who has complained about their over-investment in metrics, its chronically under-invested in by all post-product-market-fit companies.

There are a number of effects that under-investment in metrics causes:

[Read more…]

This was originally posted as a guest post on Andrew Chen’s blog here.

—

There are too many companies asking, “How do we acquire more users?” that should instead be asking “How do we get better at keeping the users we already have?”.

Its easy when approaching the problem of growth to think that you just need to get more users, after all that seems to be the very definition of growth. However, if you take a step back though and think about growth as the maximization of user-weeks over time, it quickly becomes apparent that focusing on retention has a much larger effect than topline growth. This is also much more of a sustainable growth mindset. Rapid user growth followed by rapid user attrition is an indicator of unsustainable growth. Strong retention of users over time is a good indicator of product-market fit, something you’re hopefully looking to achieve anyway.

Viral Factor and Retention

At a high level, retention is more important than virality because if your users don’t stick around they are not able to invite others to your product over an extended period of time. If you have high retention and no virality you will sustainably grow your user-base over time. If you have high virality and no retention you will not. Between these two extremes it gets a bit more complicated. In order to explain in detail we first need to review a couple of terms: viral factor and retention.

It will help as you follow this post to use our in-house growth model to play with the numbers yourself, the graphs we reference later on in this post are derived from it.

Viral Factor
This describes the growth rate of a site or app based on invitations from existing users of the service. This is often called k factor.

i = number of invites sent by each customer
c = conversion rate of those invites (#signups/#invites)
k = i * c

Viral factor on a weekly basis usually looks something like the graph below. This varies for different products, but I’ve seen this shape again and again across the products we consult on. It is front-loaded like this for three reasons:

1. The effectiveness of onboarding invitation flows.
   Onboarding is one of the few times you have a high level of user attention towards completing a specific goal (signup) and when you instruct users to invite others they often will without thinking much about it.
2. Users’ level of excitement.
   Humans are most excited by new things, this applies to internet products as well. The excitement users get when trying a new product leads them to share more, but this sharing tapers off as product becomes normal in their everyday lives.
3. Low invite saturation of users networks.
   When a user first starts using a product they know many more people that don’t use the product than do use it. Over time they share your product with others they know. Eventually, even a user who is very passionate about your product has nobody left to share it with that hasn’t already heard about it leading to lower virality as time goes on. This can also be an issue if your company grows very large, but that is a good problem to have.

Retention
This is the number of users that stick around from one time period to another. There are two ways to express retention, overall retention and week-to-week retention:

1. Overall Retention
   Overall retention is cumulative over time. If you have 30% overall retention in Week 3 it means that 30% of your users who started at the beginning of Week 1 are still around in Week 3. This is how companies normally express retention when discussing it internally.
2. Week-to-Week Retention
   For growth purposes its often useful to look at retention on a week-to-week basis instead. Week-to-week retention is how many users move from one week to the next. If Week 2 has 40% overall retention and Week 3 has 30% overall retention than our week-to-week retention from Week 2 to Week 3 is 75%. If week-to-week retention is below 100% it means we’re still losing users.

Retention on a week-to-week basis usually looks something like the curve below. It is lowest from the first week to the second week and approaches 100% as time goes on.

Why is retention so important?

In order for virality to be more important than retention your viral factor must be greater than your overall retention up to that point in time. We’ll prove this mathematically later in this post. The math is hard to simplify exactly, but there is a basic rule you can follow which approximates it. If you take only one thing away from this post it should be this:

Do not focus on improving virality unless your overall retention is stable, not continuing decrease after some reasonable period of time.

To help illustrate this, lets look at a few examples:

1. Your product has a very high immediate viral factor.

   If your product is front-loaded with invites that are sent out and accepted at a high enough rate, it is possible to achieve an immediate k > 1 viral factor. In this case, if your invites have an acceptance rate that is quick enough, your monthly active user numbers will continue to grow even if you have zero retention past the first-use of your app/site. Growth of the high virality, low retention type is almost always unsustainable as the viral loops being exploited to attain quick k > 1 virality eventually expire. This has been the downfall of many companies that appeared to grow fast, raised a lot of money on that growth, then quickly died. Like Viddy…

   

Viddy growth model with viral channels working looks like this.

Viddy growth model with viral channels broken looks like this.

2. Your overall retention is high and decreases slowly as time goes on, but you have strong virality.

   In this case, products actually do see long term benefit from focusing on increased virality, but it’s often a false signal. Its only worth focusing on virality if you think you can improve virality more than overall retention. If you switch focus too early it will lead to sub-par growth metrics. This is because the compounding effects of retention improvement are much stronger than those of virality improvement.

   We can illustrate this easily with our retention/virality simulator and setting week-to-week retention and virality to be equal. We can then measure the effect of proportional changes to one of the other in terms of number of users that we have at some point in the future. In the real world virality is not likely to ever be equal to week-to-week retention, but for purposes of this example please disregard that as it helps illustrate our point in the clearest way possible.

   Base Case – Equal Virality and Retention: ~88k users total in Week 7, 44k from retention and 44k from virality as seen in the stacked bar chart.

   20% Increased Week 1 Virality: ~110k users total in Week 7, 53k from retention and 57k from virality.

   20% Increased Week 1 Retention: ~125k users total in Week 7, 65k from retention and 60k from virality.

   As you can see here, changes in retention have long term effects that have a greater effect than equivalent changes in virality.

3. Your overall retention is high and stable.

   If you have maximized your retention to the point where you think you can increase virality more than overall retention, even accounting for the compounding effects of retention, then it makes sense to focus on virality.

Prove It!

It’s easy to model growth in Excel and there are some great models that have been shared online to help do this. The one I use is available for download at bit.ly/growthmodel. It’s a slightly modified version of this great one from Rahul at Rapportive. It will give you a nice overview of how you can expect to grow if you plug in some numbers. From a user accounting perspective this is great, but its a bit hard to conceptualize how growth actually works by looking at it that way. To get a bit of a different perspective, building a tree to see exactly where users that exist in a given week come from is quite helpful.

In the tree below w0 represents some set of users that start at time 0. Each level of the tree is a week in time. At each subsequent level you get users that stick around from retention or are invited via user virality. The number of users at a given node is just the product of all the nodes leading to that point. The coefficients represent the viral and retention factors over time, v2 is the viral coefficient in week 2 from the above virality graph, r3 is the retention coefficient in week 3 from the above retention graph, etc.

The number of users at any given level can be simplified into a recursive equation.

As you can see, this matches exactly what we saw in the tree graph above. The leading viral factor (current virality v_i) matters relative to overall retention (trailing product of r_n‘s).

Visualizing Retention and Virality

For the client work that we do at Quint Growth we built a tool to plug in numbers and visualize the retention/virality tree. Its the same tool we mentioned in the beginning of the post and used in Viddy example above. We’ve found it incredibly useful for visualizing how much more important retention is than virality (and the few cases in which it is not). We’ve made it available at quintgrowth.com/growthmodel.html and hope you find it as useful as we do!

Special thanks to Isaac Hodes for help with the d3 visualizations for the retention/virality visualizer.

In my last post I discussed how an understanding of consumer psychology can be extremely helpful for understanding how to design funnels that convert. In this post I’m going to give a concrete example of how I did this at Lookcraft to drive 18% conversion through a funnel with 18 discrete questions and 4 major steps. I got 18% of users who hit our homepage to enter their email and password, information about their fashion taste, and full sizing information. I’ve used these same tactics across multiple different products to similar effect with other companies I’ve worked with or consulted for and hope they will be useful for you as well.

[Read more…]

Over the last few years the startup community has really gotten behind A/B testing and hyped it up quite a bit. There is a more nuanced point about the downsides of A/B testing that needs to be understood: A/B tests are very very expensive for most startups at the time when they matter most, early in their formation.

[Read more…]

The next few years of e-commerce growth are going to fundamentally change the way soft-goods retail works today.

E-commerce has been growing rapidly for the last 6 years with equally rapid projected future growth. To date the growth has been mostly powered by existing brands moving rapidly online. Great direct-to-consumer brands like J. Crew realized this early, got out ahead of the curve, and are reaping the benefits. New companies selling other brands (e.g. Zappos) were notable as well. For the most part these companies built revenues through leveraging existing audiences (in the case of existing direct-to-consumer brands) or taking advantage of SEO and paid acquisition opportunities (in the case of wholesale retail). There is still some growth left to be had in this area, but there is one segment in particular that will drive e-commerce growth over the next few years: existing designer brands who previously only sold wholesale moving into direct-to-consumer online.

[Read more…]