Key Concepts for System Design

Sharding means splitting your data across multiple nodes. This can be done in different ways:

1. Vertical sharding

• Store tables on different nodes.
• Drawback: If one of these tables gets very large, then you need to re-shard that database.

2. Key-based sharding

• Use part of your data (usually the row ID) to shard it.
• Drawback: The number of nodes is fixed. If you want to add more nodes, then you have to reallocate the data.

3. Directory-based sharding

• Maintain a lookup table for where each data is found.
• Drawback: The lookup table is a single point of failure, that's constantly under load.

Caching systems are usually in-memory key-value databases that provide very fast access to data. They sit between your application and the database.

Depending on your use case, you can either cache only the results of the database queries, or cache the entire response objects (such as fully rendered web pages).

Slow operations should be done asynchronously to avoid making users wait for long periods of time until their requests are processed.

Sometimes this can be done in advance, by pre-processing data that you know users will need. For example, a forum might periodically re-render its page of most popular posts and the number of comments.

Asynchronous processing usually requires 2 components:

1. A queue that stores tasks, ordered by time and priority
2. A system that processes tasks

• Bandwidth - Maximum amount of data that can be transferred in a unit of time. Usually expressed in bits per second.
• Throughput - Amount of data that is actually transferred in a unit of time.
• Latency - How long it takes data to go from one end to the other.