Datawocky

Why SMS GupShup is Bigger than Twitter

Matt Marshall at VentureBeat liked my post on SMS GupShup, and asked me to write a follow-up guest post for VB. That post appears on VentureBeat today. Leaving aside questions of technology and scaling, I ask why SMS GupShup is bigger and growing faster than Twitter. My hypothesis:

Microblogging is a nice-to-have in developed economies, like the US. It's a must-have in developing economies like India, China, and Egypt.

In essence, microblogging is semi-synchronous publish-subscribe messaging. It’s publish-subscribe because it decouples senders and their reader(s), who can choose which senders to follow at any point in time. It is semi-synchronous because readers can choose either to follow it synchronously (via various desktop tools, or their mobiles), or read it later. In the Western world, the penetration of PCs is almost universal, so we have other PC-dependent messaging options such as blogging (asynchronous publish-subscribe); email (asynchronous point-to-point); instant messaging (synchronous point-to-point). Yes, none of them offers quite what Twitter does, but the majority of people in the majority of situations can make do with the conventional options.

Contrast this with the situation in third-world nations: PC penetration is incredibly low, while mobile penetration is incredibly high. For example, India has about 40 million PCs but 10 times as many cell phones. This makes short text messages sent via SMS the main written communication mechanism. Blogging, email, and IM are just not options, so microblogging becomes the main form of publishing, communication, and self-expression.

You can read the full post on VentureBeat.

India's SMS GupShup Has 3x The Usage Of Twitter And No Downtime

I recently started using Twitter and have become a big fan of the service. I've been appalled by the downtime the service has endured, but sympathetic because I assumed the growth in usage is so fast that much might be excused. Then I read this TechCrunch post on the Twitter usage numbers and sympathy turned to bafflement - because I'm intimately familiar with SMS Gupshup, a startup in India that boasts usage numbers much, much higher than Twitter's, but has scaled without a glitch.

I'll let the numbers speak for themselves:

• Users: Twitter (1+ million), SMS GupShup (7 million)
• Messages per day: Twitter (3 million); SMS GupShup (10+ million)

Actually, these numbers don't even tell the whole story. India is a land of few PCs and many mobile phones. Thus, almost all GupShup messages are posted via mobile phones using SMS. And almost every GupShup message is posted simultaneously to the website and to the mobile phones of followers via SMS. That's why they have the SMS in the name of the service. Contrast with Twitter, where the majority of the posting and reading is done through the web. Twitter has said in the past that sending messages via the SMS gateway is one of their most expensive operations, so the fact that only a small fraction of their users use the SMS option makes their task a lot easier than GupShup's.

So I sat down with Beerud Sheth, co-founder of Webaroo, the company behind GupShup (the other founder Rakesh Mathur is my co-founder from a prior company, Junglee). I wanted to understand why GupShup scaled without a hitch while Twitter is having fits. Beerud tells me that GupShup runs on commodity Linux hardware and uses MySQL, the same as Twitter. But the big difference is in the architecture: right from day 1, they started with a three-tier architecture, with JBoss app servers sitting between the webservers and the database.

GupShup also uses an object architecture (called the "objectpool") which allows each task to be componentized and run separately - this helps immensely with reliability (can automatically handle machine failure) and scalability (can scale dynamically to handle increased load). The objectpool model allows each module to be run as multiple parallel instances - each of them doing a part of the work. They can be run on different machines, can be started/stopped independently, without affecting each other. So the "receiver", the "sender", and the "ad server" all run as multiple instances. As traffic scales, they can just add more hardware -- no re-architecting. If one machine fails, the instance is restarted on a different machine.

In read/write applications, the database is often the bottleneck. To avoid this problem, the GupShup database is sharded. So, the tables are broken into parts. For e.g., users A-F in one instance, G-K in another etc. The shards are periodically rebalanced as the database grows. The JBoss middle-tier contains the logic that hides this detail from the webserver tier.

I'm not familiar with the details of Twitter's architecture, beyond knowing they use Ruby on Rails with MySQL. It appears that the biggest difference between Twitter and GupShup is 3-tier versus 2-tier. RoR is fantastic for turning out applications quickly, but the way Rails works, the out-of-the-box approach leads to a two-tier architecture (webserver talking directly to database). We all learned back in the 90's that this is an unscalable model, yet it is the model for most Rails applications. No amount of caching can help a 2-tier read/write application scale. The middle-tier enables the database to be sharded, and that's what gets you the scalability. I believe Twitter has recently started using message queues as a middle-tier to accomplish the same thing, but they haven't partitioned the database yet -- which is the key step here.

I don't intend this as a knock on RoR, rather on the way it is used by default. At my company Kosmix we use an RoR frontend for a website that serves millions of page views every day; we use a 3-tier model where the bulk of the application logic resides in a middle-tier coded in C++. Three-tier is the way to go to build scalable web applications, regardless of the programming language(s) you use.

Update: VentureBeat has a follow-up guest post by me, with some more details on SMS GupShup. Also my theory on why SMS GupShup is growing faster than Twitter: Microblogging is a nice-to-have in places with high PC penetration, like the US, but a must-have in places with very low PC penetration, like India.

Disclosure: My fund Cambrian Ventures is an investor in Webaroo, the company behind SMS GupShup. But these are my opinions as a database geek, not as an investor.

Removing Friction at the Bottom of the Pyramid

Dharavi is a slum of over a million souls packed into a single square mile in the heart of Mumbai, India. It holds the dubious distinction of being the largest slum in Asia.  Families of 15 crowd into 300-square-feet tenements, sharing the space with many more mice. It's hard to imagine the people of Dharavi as consumers moving up the economic ladder, but that is precisely what is happening.

Kashyap Deorah moved back to India from the US recently, and has spent the last several months visiting Dharavi to understand its micro-economy. Every street in Dharavi is home to an electronics dealer. The main business is used cell phones and prepaid SIM cards; India now has over 246 million cell phone subscribers, with the number growing at a scorching pace. One of the hottest items is -- hold your breath -- used flat screen LCD televisions! Surprising, yet clear enough when you think about it: space is at a premium, so slum-dwellers behave rationally in opting for flat screen televisions.

This anecdote illustrates the demand for used consumer durables of all kinds among the upwardly-mobile masses in India. McKinsey has published a fantastic study on how the rapidly expanding Indian economy is creating new consumers. This study divides Indian households into 5 segments based on household income: Globals, Strivers, Seekers, Aspirers, and Deprived. The Globals are the super-rich elite; the Strivers and Seekers constitute the middle class; and the Deprived are the destitute  outside the pale of consumption.

The most interesting class are the Aspirers: these are not quite destitute or middle-class, but are upwardly mobile and aspire to enter the middle class. Today, only 5% of Indian households are in the middle class, 41% are Aspirers, and 54% are Deprived. In 2025, the study expects the middle class to have swelled to 41% of households, the Aspirers to remain steady at 36%, while the Deprived drop to 22% of households. What is happening is a massive shift of households from Aspirers to middle class and from Deprived to Aspirers. The people of Dharavi are among today's Aspirers and tomorrow's middle class.

Aspirers cannot afford new cell phones, or televisions, or washing machines. But there is huge demand for used cell phones, televisions, and consumer durables of all kinds. In India today, the market for used consumer durables is extremely inefficient, and relies primarily on word-of-mouth. A free flow of information about demand and supply can make the market efficient, and also help millions of people take their first steps to becoming consumers.

Given the almost universal penetration of cell phones among Aspirers in India, the natural solution would seem to be a solution that uses mobile phones to help people buy and sell used goods. Kashyap has started a company named Chaupaati Bazaar (named after a famous beachside bazaar in Mumbai) to do just this. The problem is challenging: create a used goods market, make it work entirely through SMS and voice (no web interface), make it work for a semi-literate user base speaking many languages, and figure out the business model. Not a challenge for the faint-hearted, but something that could really make a difference if it can be made to work. I'm proud to join Chaupaati's Board of Directors as its lead investor.

Dr. C.K. Prahalad is famous for coining the phrase "fortune at the bottom of the pyramid." The bottom of the pyramid consists of the poorest section of the world's population, who are not viewed as a viable market by most consumer products companies. There are 4 billion people at the bottom of the pyramid, out a world population of 6 billion. This is an opportunity to use technology to eliminate friction at the bottom of the pyramid, enabling at least some of those people to climb the rung to the middle class.

Can SMS be a publishing medium?

When we think of SMS (Short Message Service), we think of short  text messages sent between friends, or to small groups via services like Twitter. This is how the Internet started too -- primarily as a communication medium. But soon the World-Wide Web made the internet a publishing medium as well.

The question is, can SMS messages to mobile phones become a free, ad-supported publishing medium? It seems unlikely, given the limitations: text messages must be no longer than 160 characters. In addition, they also cost both the sender and the receiver.

Necessity, however, is the mother of invention. By a coincidence of three factors, India seems to have the perfect conditions for the emergence of SMS publishing:

• Huge cell phone penetration -- 246 million and counting.
• Very low internet penetration -- about 20 million internet connections.
• A cost structure where senders pay for text messages but recipients don't.

I met today with Rajesh Jain, whose company Netcore offers a service called MyToday in India that is is effect the first SMS publisher. Rajesh Jain is an Internet pioneer, having started India's first internet portal, indiaworld.in, back in 1994 -- he sold it for over $115 million in 1999, in what was perhaps the first big Internet deal in India.

MyToday "publishes" a number of "MyToday Dailies", on topics such as News, Cricket, Health, Gossip, Local News, and so on. You subscribe to one one of these dailies by sending a text message to a published phone number from your cell phone. Subsequently, you will receive a daily (or sometimes, more frequent) text message "articles" on the topic you indicated interest in. The articles are editorially assembled, and also include an embedded ad. How, you might well ask, can you fit in a story and an ad within the 160-character limit? There are two parts to this answer:

1. You can do a lot within 160 characters, if you try really hard and your expectations are not very high. I remember fitting very playable video games into 16K of RAM on my Sinclair Spectrum when 48K was way too expensive. I looked at some of the MyToday examples, and I'm impressed.
2. It's possible to split stories across multiple text messages; more and more phones have the ability to handle messages larger than 160 characters; the SMS transport breaks them up into 160-character chunks and the phone re-assembles the message.

The real economic key to this whole enterprise, though, is the cost structure of SMS in India. There you don't pay to receive a message, only to send it. So for subscribers the service is entirely free, after the initial message to subscribe. And the numbers prove it. MyToday has over 3 million unique subscribers, each with 3 subscriptions on average. They send more than 10 million text messages a day, making them India's largest sender of text messages.

For MyToday, ad rates today are still below the cost of sending a text message, but text messaging rates -- especially for huge bulk purchases -- are falling dramatically, so it's not hard to see the crossover happening next year. And that could usher in the era of the SMS publishers.

Traveling: In India this week

I'm on vacation, traveling in India. First stop Mumbai -- the main event is Lakme India Fashion Week. My wife, Kaushie Adiseshan, is a fashion designer. She is showing the latest collection from her line, Pink Elf, on the runway today. She's a veteran -- having shown several times at LIFW and San Francisco Fashion Week -- but there are aways butterflies before the event.

I'm looking forward to the parties after! Last time the entire Indian cricket team showed up -- let's see what we get today.

Update: The show was fantastic, as was the party hosted by Neeta Lulla at Henry Tham's last night. Bollywood superstar Aishwarya Rai was at the party!

Update 2: By popular request, here are some photos from the after-party.