Challenge 5: 100 Likes/second - Deepstash
Challenge 5: 100 Likes/second

Challenge 5: 100 Likes/second

  • Scale horizontally again to handle more events -> Add multiple Dispatchers and move the subscription table into a key-value store so it's accessible to all Dispatchers.
  • Dispatchers are independent from Frontend nodes and don't have persistent connections between them.
  • Any Frontend node can subscribe to any Dispatcher.
  • Any Dispatcher can publish events to any Frontend node.

9

76 reads

CURATED FROM

IDEAS CURATED BY

ocpodariu

Alt account of @ocp. I use it to stash ideas about software engineering

The idea is part of this collection:

Machine Learning With Google

Learn more about computerscience with this collection

Understanding machine learning models

Improving data analysis and decision-making

How Google uses logic in machine learning

Related collections

Similar ideas to Challenge 5: 100 Likes/second

Challenge 4: 10K Concurrent Viewers

Challenge 4: 10K Concurrent Viewers

  • Scale horizontally to handle more concurrent viewers -> Add multiple Frontend nodes and coordinate them using a Dispatcher node.
  • In a similar fashion to the Frontend node, the Dispatcher has a subscriptions table to know which frontend nodes should r...

Performance and scale

Performance and scale

  • Each Frontend node handles 100k persistent connections. It handles only this many connections because the server is doing a lot of work processing multiple types of data (likes, comments, instant messaging etc.).
  • Each Dispatcher can publish 5k events per seco...

Final architecture

Final architecture

  1. A viewer likes a video and sends an HTTP request to the Likes backend, which stores it in a database.
  2. The backend forwards the like to any random Dispatcher node using an HTTP request.
  3. Dispatcher looks up in the key-value store to find out which Frontend nodes are subscribed ...

Read & Learn

20x Faster

without
deepstash

with
deepstash

with

deepstash

Personalized microlearning

100+ Learning Journeys

Access to 200,000+ ideas

Access to the mobile app

Unlimited idea saving

Unlimited history

Unlimited listening to ideas

Downloading & offline access

Supercharge your mind with one idea per day

Enter your email and spend 1 minute every day to learn something new.

Email

I agree to receive email updates