One of our programmers told a story about how is 80 year old mother sent him an invite to ‘connect’ on LinkedIn:   This elderly lady unfortunately fell for a well known trap where LinkedIn logs into your email account and collects all of your contacts and indiscriminately sends every one of them an invitation to ‘connect on Linkedin’. You can read about this unethical practice and the class action law suit at the New York Times.

Social Network users:  Quality vs. Quantity vs. Non-Existent

Linkedin no doubt counted this 80 year old mother and each one of her vulnerable, elderly contacts who clicked on an Invite as an actual ‘user’, even though she won’t use the site again (and neither will her contacts). Users acquired under such unscrupulous circumstances would explain the sky-high user numbers Linkedin and Facebook reports each quarter, and would also explain their high valuations.

Facebook also uses similar tactics to increase their user base. Their approach isn’t quite as aggressive, but it’s equally unethical.

This raises the question: What is the actual number of ‘quality’ Facebook’s users?

Facebook’s user base can be likened to a gold mining operation: Users are an untapped, unexplored, unproven resource. Facebook, like a mining company, is in the business of extracting dollars from those reserves… and like a mining operation, the projected number of extractable dollars is highly speculative at best, where the quantity and quality of reserves is never accurately known.

Also like the mining industry, it is easy for social media companies to spin their numbers in a positive way, since they are in charge of their own reporting of exploration activities. But the mining industry is bound by regulations governing how they conduct their gold assays to determine quality and quantity of the resource.  In contrast, social media companies like Facebook and Linkedin are self-governing.   This means an unscrupulous social media company could outright lie about their user base and never be found out as they are not bound by regulations that require a forensic analysis of user quality and quantity.

Facebook has developed their own definition of quality users they call ‘Active Users’, the concept of which is explained in the company’s 10-K filing as part of an 800 word essay on page 4 (excerpted):

…monthly active users (MAUs), daily active users (DAUs), mobile MAUs, and average revenue per user (ARPU) are calculated using internal company data based on the activity of user accounts.

there are inherent challenges in measuring usage of our products across large online and mobile populations around the world. For example, there may be individuals who maintain one or more Facebook accounts. We estimate, for example, that “duplicate” accounts may have represented approximately 5.0% of our worldwide MAUs.

…estimates are based on an internal review of a limited sample of accounts and we apply significant judgment in making this determination… our estimation of duplicate or false accounts may not accurately represent the actual number of such accounts. We are continually seeking to improve our ability to identify duplicate or false accounts and estimate the total number of such accounts, and such estimates may change due to improvements or changes in our methodology.

Facebook uses a computer algorithm to filter out duplicate and fake profiles that violate their terms of service, however new fake and duplicate profiles are created daily, many with the intention of spamming groups and pages and other users or creating fake ‘likes‘ to a fan page. The battle against these profiles is an ongoing cat/mouse game for which is very difficult to stay ahead of the curve, and investors have no detailed insights into how they are fairing.

Even Mobile users – which would seem to be a quality metric (only real people own mobile phones right?) are susceptible to foggy metrics as per Facebook’s own 10-K:

Some of our historical metrics… have also been affected by applications on certain mobile devices that automatically contact our servers for regular updates with no user action involved, and this activity can cause our system to count the user associated with such a device as an active user on the day such contact occurs.

So the Facebook mobile app is installed by default on a number of new mobile phones, and when these phones are turned on the app automatically contacts Facebook. This activity counts in the Mobile Active User numbers, even if the mobile user did not initiate it.

Reading further, metrics become even more unclear:

The methodologies used to measure user metrics may also be susceptible to algorithm or other technical errors. For example, in early June 2012, we discovered an error in the algorithm we used to estimate the geographic location of our users that affected our attribution of certain user locations for the period ended March 31, 2012. While this issue did not affect our overall worldwide MAU number, it did affect our attribution of users to different geographic regions. We estimate that the number of MAUs as of March 31, 2012 for the United States & Canada region was overstated as a result of the error by approximately 3% and these overstatements were offset by understatements in other regions.

To believe a social media company’s metrics whole heartedly at face value is difficult, but when the methods of calculating those metrics are flawed, it has to be a leap of faith.

Indeed, it seems now that the SEC is starting to question how these companies measure users, and when you have regulators starting to probe the very foundation of your company’s stock valuation, this is never a good thing.

The real test of the quality of Facebook’s user reserves is going to be how much money they can get from advertisers on a consistent basis, which is going to depend on the returns advertisers receive on their dollars spent. This means real users with a pulse will not only have to click, but they’ll also need to buy something.. however we are already seeing signs of this being a challenge.

It’s unfortunate that investors will never see the algorithm that filters out fake users so they can draw their own conclusions as to its effectiveness.

It’s also unfortunate that forensic-level advertising metrics are not released publicly to investors (this would be the equivalent to metallurgical assays in the mining industry). We would love to see how many advertisers are paying for CPM vs. CPC, and the conversion metrics for each, but this is competitive information and thus a closely guarded secret.

Our guess is that large advertisers are experimenting with CPM-based branding campaigns as part of a larger social media strategy, and may not be concerned with conversions (for now) as they are simply build their brand and/or use Facebook users as a focus group to gauge reactions.

If this is what is making up the bulk of Facebook ad revenue, then for them it’s a race against the clock to mine new ad dollars out of the current user base before old ad dollars dry up as advertisers figure out how many of the ‘active users’ they are paying to show their ads to are fake, or bots, or uninterested spammers, or simply uninterested users.

Facebook, like Linkedin and other social media plays, certainly has a solid active user base, and we’re sure a solid business model to mine these users for their dollars is there somewhere, but unfortunately a true measure of market potential lays somewhere between fake profiles and the company’s own secretive methods for classifying and taking the pulse of ‘active’ users.  For now we only have the company’s revenue numbers to work with, but these numbers simply indicate how they’ve done in the the past, and certainly don’t paint a clear picture of the future for such a relatively young company.