TechNation recently called for feedback into how the Top 100 Aussie Web Startups is calculated and at the CTO meetup earlier this month we discussed some options that could improve the index. As with many things it became very clear, very quickly that defining this list is not as trivial as you would first think.
What is an Australian startup?
Defining an Australian startup is perhaps more difficult than defining whether it is successful.
Some companies work in startup mode for years; perhaps it’s taking a long time to get traction in their product, sometimes just because that is all the growth the business can sustain. Some people complain there are companies on the list that should not be, as they are no longer startups. The closest definition I can come up with: “A company is a startup if it has less than 20 employees, isn’t rolling in cash, and calls itself a startup”. This definition may seem ambiguous but I believe adding more criteria would be too restrictive.
What is a successful startup ?
Number of page views – If your business model is based on advertising revenue, this metric is a measure of potential success but revenue-per-impression may be more relevant in the longer term.
Number of users – often the holy grail in defining success but users are easy to get — keeping them interested and coming back is harder — so I much prefer the metric of number of active users.
Active users – If a user comes to your website once a week or more, on a regular basis, I would define them as active. This can be a useful metric for determining a ‘top 100’.
Length of operation – If a startup is sustainable and is surviving in a tough climate then that in itself is a metric for success.
Revenues – Measuring success based on revenues doesn’t sit well with me. If a company is making significant revenues then to me they are moving away from being a startup anyway. Although a useful metric in some ways I wouldn’t base the Top 100 on it.
Page ranking – TechNation is currently using page ranking from two sources to determine its Top 100. This can be a good definition of success but Compete metrics are only gathered in the US so a startup focused in the Australian market can lose out on this metric.
Number of blog posts, twitter posts etc — I believe the ‘social traction’ of a startup should be pulled into the calculation. If your company is being talked about on the Interweb then that is a measure of success and something that can be captured relatively easily.
Changes to the index.
Let’s start gathering more data and iterate and refine as we go – The agile approach 🙂
Many of the metrics that are defined above are only available internally to a startup. These are things that they should be tracking in order to measure their own growth and success. But information disclosed outside the company can be useful intelligence for competitors. Startups are understandably unwilling to make these statistics public on a regular basis, if at all.
How I would change the index:
1. Use the other metrics that are currently being gathered but not used in calculating the index – Quantcast and Hitwise.
2. Calculate the number of twitter posts about the startup in the previous month and add this as a column
3. Calculate the number of blog posts about the startup in the previous month and add this as a column
4. Allow each statistic to be given a weighting. (each of the 4 page ranking stats may contribute 20% to the total with 10% each to twitter and blog posts)
5. Allow results to be filtered on geography.
6. Provide tools for users to remix the results and weightings, gather their feedback and iterate again.
What would you change to improve the index?
Really inspiring to be part of the new technology coming out of Australia. I think Aussies have suffered under a talent drain to foreign shores for too long, and now people are realising you can be successful without leaving the country.