Message System & WhatsApp

Scenario (features):

• log in, sign up
• contacts
• two user send msg to each other
• group chatting
• user is online/offline

Estimation are important!

WhatsApp:

• according to report:
• 1B active monthly users
• 75% DAU/MAU
• 750 M DAU
• based on this, let's design a 100M DAU whatsApp

QPS:

• assume a user sends 20 msg, avg
• avg QPS = 100M * 20 / 86400 ~ 20K
• peek: 20k * 5 = 100k

storage:

• assume 20 msg/user/day
• assuming each message on average has 30 characters, that result in 60GB assuming no metadata
• expected storage size:
  • 5 years
  • 60 * 365 * 5 = 100TB
• 2B in total, 30bytes/msg => 100M * 20 * 30byte = 60GB

design goal:

• Latency - Is this problem very latency sensitive (Or in other words, Are requests with high latency and a failing request, equally bad?). For example, search typeahead suggestions are useless if they take more than a second.
  • real-time service, important
• Consistency - Does this problem require tight consistency? Or is it okay if things are eventually consistent?
• Availability - Does this problem require 100% availability?

how to send msg to user:

• send to server first
• server send to specific user
• not end-to-end

service:

• message service: manage messages
• real-time service: real-time chatting
  • how to send a msg:
    • client -> server
    • server create threads for both user (if thread not exists)
    • create a message (with thread_id)
  • how to receive a message?
    • pull. every 10 sec (stupid but workable)

Storage:

• Message Table (NoSQL): a lot of data, no need to modify, one message is like one line of log data
  • Cassandra: append only
• Thread Table (SQL):
  • index by:
    • owner user id
    • thread id
    • participants hash
    • updated time
  • nosql is not good for multi indexes.

Scale:

performance, functionality

• message stores in NoSQL, easily scale
• thread sharding by user_id

how to speed up? 10/s checking is too slow.

• Socket!
• new service - push service: push service provide socket connection, keep TCP connection.
  • difference between socket connection & HTTP connection:
    • http connection, the only way to get data is pull.
    • socket connection allows push.
• when user open app, connect to a socket of Push Service, Message Service receive message, than push to Push Service
• if a user is not active, close the socket

How to support group chat:

• a group of 500, Web Server need to send 500 push request to Push Service.
• • • 

Online Status:

• server need to know who is online/offline (push/pull)
• user need to know who is online/offline (push/pull)
• 
• 

FLow: