Facebook shuffles the deck

, Thursday, August 30th, 2007

Just two days ago, I wrote a post about how some applications were cheating their way to the top of the “most installed” list on Facebook. It seems that they’re already taking action to curb these sorts of abuses, starting with changes to the code (someone with more programming About this applicationknowledge can probably fill in more here – if you’ve got a sense of what the code changes mean, post a link in the comments). But what I find more interesting is a new feature added today that tracks stats not solely by installs but by engagement. If you look at the application summary page today, you’ll see that instead of total installs, it now shows “Daily Active Users.” This change completely turns the application rankings on their head, and will likely have a major effect on how applications are developed and what gets installed.

So what is a “Daily Active User?” And what’s the percentage? Dave Morin explains on the Facebook Blog:

With a change this foundational to Facebook Platform’s measurement, we want to make sure that you completely understand how we will be measuring engagement. We define engagement as the number of users who touch your application every day (measured from midnight to midnight each day).

These touch points are:
- Canvas Page Views
- Link Clicks in FBML
- Mock-Ajax Form Submission
- Click-to-Play Flash

The number of engaged users is calculated by putting all of these touch points together. We display this as the number of “Daily Active Users.” Next to it we also show what percentage that is of the application’s total number of users.

So you can still figure out the gross reach of an application by taking the daily active users and dividing by the percentage. This leads to some new questions about what’s the most useful gauge of how much real influence an application has.

As I see it, there are three arguments that could be made:

1. Total installs remains the most important metric, because it represents potential users that could be activated. This is the argument that will almost certainly be thrown around by the install-and-run crowd who create the spam apps I railed against earlier this week. Because the application is installed all over the place, there’s a chance every Election08day that some event could cause people to use it. Plus there’s no way to know if today’s active users are the same as yesterday’s, so there might be a much larger active user segment if you used a week-long graph. I’m skeptical of this argument – I install lots of apps to test them (here at Beaconfire, we call it “work” – one of the reasons I love my job. Here’s where you can get one like it.), but keep them off my profile because I’m not interested in cluttering it up. Total installs would count me as a user, which I’m really not after the initial install. One thing that does make this a useful metric is for those applications whose primary value is not in interacting with them. Election08 and Where I’ve Been are just two examples of applications that are useful, but don’t really need to be clicked to be used, making total installs the best gauge of their popularity.

2. Daily Active Users are the most important metric, because it shows the number of people actually taking advantage of your application. This argument tends to Free Giftsfavor apps that are most useful when engaged on a daily basis – remember that since we’re measuring by day, each user can score a point once per day. This metric hurts both apps that you install once and forget about and apps that involve doing a lot of things in bursts. The first is a feature, the second is a bug, in my opinion. A slightly fairer system would factor in the number of “touches,” but this would unfairly prejudice against apps that are easy to use, and I don’t think we want to incentivize bad user interface.

3. Percent of total is the most important metric, because it tells users how likely they are to find this application useful after installing it. This argument seems to Chessmake sense on the surface – logically, if an application is useful, people will use it, and it doesn’t matter how many people have it installed (after all, every application started with just one install), it’s how often those people use it. The problem here is the lack of a baseline – every user who installs your application, then doesn’t log on to Facebook on a given day is a liability to it’s percent of total. A fix to this method would be to take the number of users who used the app divided by the number of users with the app installed who logged in. But that would involve significantly more calculation for servers that are already sometimes having trouble coping with the load – and school’s just getting back in. Percent of total helps apps like Chess that involve maybe one or two clicks per day consistently and encourage you to stop by Facebook regularly to check on it. Chess shows only 10,000 or so Daily Active Users, but 17% of all installs were active yesterday – that’s about 2-3 times the average!.

So what’s the perfect metric? I could cop out and say that it’s one of the metrics I noted above that didn’t exist (number of touches or percent of those who logged in). But I’m not going to do that. Instead, I’m going to cop out by saying it’s

4. They’re all important and need to be analyzed together to come up with the whole picture. An app with a huge total install count has a lot of potential, an app with a lot of Daily Active Users is being engaged regularly by the community, and an app with a high percent of total is probably useful. They’re obviously intertwined mathematically, I am Greenbut more subtlety so as well – a new app that’s very cool might have a low total install count, not a lot of active users, but a high percentage. As it becomes more popular, its reach goes up, active users goes up at a slower scale, and percentage starts to fall, as either the original users get bored or new users don’t find it as cool as the hype around it. An app that can defy these trends and keep percentage steady while total installs and active users go up is one to watch. Similarly an app may be useful for a niche audience (developers, environmentalists, Jedi) and not want a large number of total installs as much as a high rate of retention.

Now with this change, the black hats of development will undoubtedly be retreating to their lairs, twirling Artist??s Renditiontheir mustaches as they scheme new ways to spam their way to the top. Maybe it’s a code exploit that fires a pop-up every time you log in, making you take a seemingly innocuous action that counts as a touch. Maybe it’s changing their incentive systems to encourage you to come back once a day (and see more ads while you’re bumping up their rank). Maybe it’s just to keep spamming even faster so that you’re always bringing in new people to deal with attrition. Whatever it is, I’m sure Facebook will put a stop to it (and hopefully Karma will make short work of their ill-gotten gains), but this sort of thing is inevitable in any open community.

The bigger question is what metric the community will use to determine what to install. Facebook tried to make a major change by suggesting alternate metrics for what makes a popular and useful app – now it’s up to users to determine if they’ll go along with it.

One Response to “Facebook shuffles the deck”

  1. John Brian Says:

    Just so you don’t think our blog is all Facebook, all the time, here are some great recent posts about other things.

    Words you never realized were spamTim
    New Google Maps embedsEric
    HP’s new “Cloudprint” – can you really print from your phone? -me
    Beaconfire’s doing tech planning for the African Wildlife FederationLynn

    And of course, you can sign up for our RSS feed to get the latest Beaconfire Wire posts right away – be the first to dazzle your office with new insights (we won’t tell your co-workers where you got them).