Hello Leetcoders!
A big thanks to everyone for the awesome response on my previous post https://leetcode.com/discuss/general-discussion/1105898/system-design-introduction-to-distributed-systems-designing-a-highly-available-system
This is part 2 and continues the design used in the first post. I request new readers to please go throught the previous post linked above first.
So, we were at the following architecture:
As I mentioned previously, some problems with this architecture is lack of asynchronous processing and caching of data to process requests faster. As in real life, frequent network calls to the database can be expensive, caching is an important aspect.
But first, lets bring in Asynchronous processing to our system. Currently, the web server is processing the requests in a synchronous manner. Any computation required, like image processing (very heavy), is done synchronously on the web server itself. What problems does this cause ?
- Say your users are uploading HD images to your server (like for any e-commerce website). The server needs to make lower resolution copies of this as serving up HD images all the time to all the users just browsing a catalogue can be expensive. This processing of images, in the current architecture is done synchronously by the web server.
- This means that the users have to wait till the web server does all this to be redirected to a success page. BAD DESIGN!!
- The idea is to add a worker/computational server to do this. But, still this is synchronous. To get the needed async mechanism, we need a async message queue. (Check out worker queues in azure for example)
So what is happening here?
- The web servers are delegating the intensive work to computational server via the async queue. So, web server can process the next upcoming requests while the computational server does its thing. But, how will the user know when the processing is done? For that, we have a notification service listening to the queue for "Done" message from the computational server.
- After receiving the done message, it will notify the web servers to send a success message to users , again via the queue.
Our design is evolving. We have a solid design to cater to async processing. I have left out the details for replicating the computational server as it is a minor detail here as ,if you have gone through both articles, you'll know we can always just add a duplicate instance for load balancing and avoid SPOF(Single point of failure).
But, our design is still making a lot of heavy network calls. Suppose Elon Musk tweeted something (with an image/video) and you know a lot of people are gonna be seeing this tweet from across the globe. It doesn't make sense to have every request go through the whole process of calling an API , accessing the DB, and returning the result.
Caching
- Why don't we cache the things that we know is gonna be requested a lot. We can have a cache policy like LRU to support the needs. But, where do we place this cache?
- I just said we need to avoid network calls from web server to DB right? So, I should just have a cache on the web servers ? But, what if that server goes down? Cache goes down with it!!
- Alternative? Have a global cache with both servers would access. But wait, didn't we want to mitigate network calls? YES! But, network calls to cache won't be much expensive as data transferred would be less (as cache stores less data) and access time would be a lot faster than DB (cache is closer to web server and search time is faster as data is stored in main memory). So we can trade this off for accurate info as it is a shared global cache.
- Now, the web server would first query the cache for some data, if its not found, only then it would go to the DB. Saves us a lot of resources and time.
- Obviously , one cache can fail and so, we would have multiple global caches in a distributed system (known as distributed caching). But, this brings in its own problems like how do we keep the data in multiple caches consistent. If you are interested then you can read up on distributed caching for this as its a big topic in itself. I'm just giving you a high level view here. Distributed caching also helps for faster responses on geographically distributed client requests.
So, is this design good finally ? YES, it is actually a good final design for entry level interviews. For experienced people, low level design is also asked and just the high level designs won't help.
Disaster Recovery servers
Just an ending note on DR servers. Ideally you want to have a copy, of all your databases in one datacenter, replicated in another datacenter. In case of a natural disasters, fire, etc. , your data isn't lost. I haven't shown this in a diagram but I guess you can visualize what I mean. If not please comment down below and I'll add it.
More Improvements ?
- Even I have run out of ideas now, so comments are appreciated :P.
- We can actually use SSL termination in the reverse proxy so, the server doesn't have to take load of decrypting the requests.
Thanks a lot. This brings me to the end of my System Design Introduction Part 2. This will be the last part on this. I'll be happy to write more on something you guys suggest.
Any and all feedback is appreciated.