SDE-1 | Gemscap (Trading Firm) | YOE-1 | Rejected
Anonymous User
508

A bit of my history - Worked at Quant and Algo Developer at Futures Frist and after that switched over to tcs as a software dev.

So, the interview was based on how have you managed or will manage millions of incoming data stream like websockets data to which many traders have subscribed to it in office. One more thing is that trading firms have always servers on-premises meaning no cloud like AWS.

Then he asked me to draw the complete architecture for it like in a system design way.

My approach -

  1. Service to manage the connections.
  2. then comens service to manage the stream of data without precessing because that would take time which we can't afford.
  3. Service to put all the data into message queue, for FIFO for traders. [He caught me here first as he said will you make as many number of queues as traders has subscribed? Not optimistic.]
  4. Now comes the layer for processing the messages/data like news for eg, change to some format or timestamp.
  5. Finally we will deliver the data via broadcasting to many traders.
  • First service to consume the websockets. [A gateway]
  • we can categorize one type of data to one mq.
    I was under the assumption that multiple group of traders would need a separate group data which we could have managed through mq.

For broadcasting we would push events to that service and traders can have common dashboard for it. [But in this case server side rendering will come into play increasing cost.]
A subscription manager service acts as a traffic controller between raw data and traders.
So via websockets, a sub service maintains a persistent connections to push real-time updates to subscribed users without repeated client requests.
Interviewer : So basically you are trying to design a fan-out design here?
Me: Yes, (then I go into explaining it.)

Next question came was to how will you store data for live news and old news? His answer was implying towards Array buffers to send/recieve data. And my answer was to use cold/hot storage but it will consume ram.
Source - WebSockets can send and receive binary messages, which are raw byte data, in addition to text messages, allowing for efficient data transfer of formats like images, audio, or game state. Clients can send ArrayBuffer or Blob objects, and receive data as either a Blob (default) or ArrayBuffer by setting the binaryType property on the WebSocket object. This differs from text messages, which are always received as strings, enabling high-performance, low-overhead communication

I would appreciate any feedback from community.
In no regard I think this was SDE-1 interview for 1yr of exp. :)

Comments (3)