You've given a large log file (csv, txt, json) ( size: 100GB) containing application API logs. All events in file are from similar datasources (Compute Engine, App Engine, vertex AI) etc. Your task is to find all unique logs from a file and store it in another file.

Example: 2021-07-22 03:54:21,999 +0000, level=INFO, pid=111311, tid=140063482943296, comp=remote_configs_sync, loc=remote_configs_sync:52, action=parse_instance_path, status=success, result=('gmi-objects-default-vpc-802bd3fb', 'watchful-walrus-c17/_bundle')