You need to build scp from machine 1 to machine 2  -> For huge File like of 1TB size.

Design a scalable and fault-tolerant data collection system that can collect logs and data from various sources (eg, application servers, tailing a file etc.) and route them to different destinations (e.g., Elasticsearch, Kafka, S3) for further processing and analysis.

Build a  Pipeline Manager where many job can be submitted and many jobs will have dependency on previous jobs one or more.

 A --> B --> C ------
  |                            |  
  |                             --> G
   ---> E --> F-------------^
 K--> L --> M

Now What we are given:
Job { // here A,B ,C are each a job
  jobId
  doWork() throws EnvironmentError
}

execute() -> need to implement this function in the class which I will create.

Now the pipeline can end under 2 conditions:
 1. Success -> when all jobs have executed.
 2. Failure --> If atleast one job fails, then it fails. No job is in a running state.

Given:
// open a file descriptor for a named file
open(name string) (fd int, err error)

// pread will read len(buf) bytes, returning the number of bytes read
// if a read would exceed the end of a file, pread will return EOF
pread(int fd, buf []byte, offset int64) (int, error)  //int -> how many actual bytes were read.
                                                  	// any error occured.
                                                  	//in our case -> only int works (-ve -> error)

// pwrite will write len(buf) bytes to a file, returning an error if not all
// bytes could be successfully written
pwrite(int fd, buf []byte, offset int64) error // filled buffer of data

// close a file, flushing any outstanding writes
close(fd int) error


Please Implement:

// copy the contents of file src to file dst.
copy(dst string, src string) error