Design Distributed Job Scheduler.
Anonymous User
2039
LinkedInSystem DesignInterview

Distributed Job scheduler:

YOE: 3.5

Functional Requirements:

  • Create and schedule a job.
  	i) one time job.
  	ii) recurring job.
  	iii) Priority based job can be scheduled.
  • Can check the status of the Job.

Non Functional Requirements:

* Job should be executed at-least one time.
*  No job should be lost.
*  Scheduler should be highly available.
*  Job should start executing within 5 seconds of it's starting time.
*  1M jobs in a day can be scheduled and executed.
*  Retry job execution within 15-20 mins if failed due to system crash or some retry-able error.

Core Entities:

  • Job 
    * job_id
* creation_timestamp.
* created_by
* schedule_cron_syntax
* status: [not_enqueued, enqueued, running, failed]
* isRecurring
* priority ? [Critical ]
* blob_url
* exectuion_start_time
APIs:
a) Job Creation:
    POST: /schedule/job -> Job_id
    payload: <file for execution>

b) Check Status:
    GET: /check/job_status/{job_id} -> Status of job

Please help in evaluating design.

image

Comments (6)