Design a bidding system where:

• User's can post any item for auction/bidding at any time
• Bidders can bid on any existing item any number of times
• 10M new auction items are added everyday & 100M new bids come in every day
• User wins an item if there are no higher bids in the next 1hour
• User has to pay for the item within 10 mins of winning the item
• User's can only bid for 1 count of an item at a time.
  • Example: If there are 10 cars in the system, user's can only bid on 1 car at a time.

Functional Requirements:

• Users should be able to put items up for auction
• Bidder's should be able to search for items they are interested in.
• Bidders should be able to bid on items they are interestes in.

Non Functional Requirements

• Low Latency
• High Availability
• Highly consistent
  • The bidding part should be highly consistent. Bidders shouldn't be able to bid on unavailable items
  • New items can take some time before they become available for bidders to bid upon

** Estimation: **

• Read heavy traffic
• Traffic:
  • 10M new items added every day
  • 100M requests to bid on these items
• Storage:
  • 10M new items/objects each day. If each objects needs ~1KB of space to be stored then it would take 10M * 1KB = 10GB of data each day

API:

• Users who want to list new items for auction:
  • putItem(string userId, string itemCategory, int totalQuantity, string description)
  • POST /api/v1/bid?
    • JSON: {userId: "", itemCategory: "", totalQuantity: "", description:""}
• Users who want to bid for an item:
  • bidItem(string userId, string itemId, int quantity, int maxBid}
  • POST /api/v1/bid?
    • JSON {userId: "", itemId: "", totalQuantity: "", maxBid: ""}

** High Level Design: **

• Item Ingestion Arch

  • U-1, U-2, U-n are the users who are posting items for auction
    *

• Search Arch

  • S-1, S-2, S-n are the users who are searching for items in the system to bid upon
  • 

• Bidding Arch

  • B-1, B-2, B-n are the users who are bidding for items in the system
  • 

** Deep Dive: **

• User Item Ingest Service: This is the entry point in the system for people who want to list an item for auction. It would get incoming request from users & put those details in a Kafka topic called "item_ingest_topic". This topic will have multiple partition as many consumers are listening/consuming from this topic
• Kafka Cluster: This is an abstraction of all the queues/topic in the system
  • item_ingest_topic: Would contain all the items which needs to be added into the system
  • item_update_topic: Would contain all the items who have had successfull/un-successful bids in the system
• Search Ingestor & Auction Item Ingestor: These are the 2 services which consume from item_ingest_topic and puts them into corresponding data-store
  • Search Ingestor puts item info into ElasticSearch cluster & makes them available to search queries
  • Auction Item Ingestor puts item into a Mongo DB cluster.After getting the Mongo object id it would put this item into Auction Bids table of PG SQL cluster with status active indicating this item is available for auction.
• Auction Item Mongo Cluster:
  • All new incoming item info will be stored in Mongo. Why Mongo & why not SQL/Relational ???. Since the incoming items are diverse & do not have a common structure or attribute I feel Mongo is the correct data store. For Example:
    • Car may have a different set of attributes than a flower pot but both are items available for auction
  • We can use Mongo Object ID as unique id for each item
• Search Service:
  • This service sits on top of ElasticSearch cluster & enables what kind of items are availble searching. It can filter out some items based on user's age by talking to User Service.
  • For an incoming search query, it would query ElasticSearch(ES) to get the details & return to user. ES will not have item info. Once user clicks particular item the service would then query Mongo DB to get item details
    • Example: Lets say user queries for "Toyota Camry", Search service queries ES & returns:
      * || 4 door Camry|| 2000 ||
      * || 2 door Camry|| 1992 ||
    • When user clicks one of the above two rows Search Service would query Mongo to get rest of the details like, color, total mileage etc
• PG SQL Cluster:
  • All auction/bid related data are stored in Auction Bids table in a Postgres Cluster. All reads go thru slave's & write's to master. We can use pessimistic locking to protect data integrity
  • Table Auction Bids:
    * itemObjectid, maxBid, lastBidTime, status
    • itemObjectid is the unique object id generated by Mongo
    • maxBid is the last max bid received
    • lastBidTime is the max bid received time
    • status is active or inactive
      • active if item is available for auction
      • inactive if user has successfully won the item in auction or if winning bidder didnt pay within 10 mins of winning the bid. Inactive entries can be periodically purged by a different service.
• Auction Service:
  • Whenever a bid comes in this service would query slave to see if this is max bid:
    • If this is not max bid then it would inform user to bid higher
    • If this is max bid then it would update Auction Bids table with this bid & current time and also put an entry in Redis with an expiry timer of 1hour.
  • If user wins the bid(after Redis timer expires) then Auction Service would inform Payment service to look for this payment for the item and put the entry in Redis with a 10 mins expiration timer.
  • if the bid is successfull or not, would put this detail into item_update_topic
• Payment Service & Redis: Are used to track if a user won the auction & if user has paid up for the item within 10 mins of winning.
  • Everytime we have a max bid for an item, it is inserted into Redis with an expiry timer of 1hour. If there are no new bids in the next hours, Redis expires & we declare the current bidder as the winner & the winner has to pay up in the next 10 mins
  • Once we have a winner the info is again added to Redis with a 10 min expiry timer & sent to Payments Service for payment.
    • Successful Tx: Winner pays up within 10 mins. Payment Service would delete the entry from Redis so that we do not get an timeout exception from Redis once its timer expires
    • Unsuccessful Tx: Redis expires and this exception will be handled by Auction Service which will add this item to the item_update_topic
• Inventory Update Service: Would consume from item_update_topic & update the availability of items:
  • If auction was successfull & user paid for it, then this service would decrement the count by one in Mongo. If there are zero instances of item remaining it would then delete the item from Mongo & mark the item inactive in Auction Bids table
  • if bid was unsuccessfull, then this service would update the count in Mongo & reset to original price in Auction Bids table & set the item status as active for re-bidding

Looking forward for suggestions & feedbacks about the design