Given a 1000GB file containing URLs, we need to remove duplicate URLs.

I proposed external sorting based approach but seemed interviewer was looking for something else. Can anyone tell me if it's possible to solve it without sorting?

We have 8 GB of RAM and it has to be done in our machine itself

It is clarified that no database should be used