Question: We are getting data in different formats such as JSON, plain text, CSV files, XML etc. The file size can vary from 1 MB coming in every 15 mins to 1 GB files coming in every month. Design a system that can process this data coming in from various sources in various formats, parse it and store in in JSON format only. You are free to use any type of database. \n\nFollow up: When the data files are very huge like 1 GB each, how will you process them fast. \n\nPlease suggest solutions.

Question: We are getting data in different formats such as JSON, plain text, CSV files, XML etc. The file size can vary from 1 MB coming in every 15 mins to 1 G