Design a efficient client side rate limit handler

There is a RESTful API exists for a resource having API rate limits of N req./m for each HTTP method.

User is going to use the RESTful API for CRUDs of the resource. Suppose the resource is highly dynamic in nature at client side. for sake of an example: assume pods in kubernetes and user is reflecting those pods somewhere else using the said RESTful API.

Assertions/Expectations:

  1. Client should backoff for the rate limit window when API reaches rate limiter threshold.
  2. Sequence of firing REST APIs for a particular resource should be as that of sequent of events receieved to listener.
  3. Sequence of events is important for the specific resource means client should make API requests for different resources in parallel to improve performance.
  4. Suppose, if POST reached rate limiter threshold, all the threads firing request should halt for rate limit window. Each worker should not send request and halt after gettting throttled response. The first one who receives throttled response should inform others not to fire further requests.
  5. If POST for a resource is in queue because of throttled POST, next PUT/PATCH, DELETE should halt.
Comments (4)