URL Shortener | MD5 | How to deal with collisions | FinTech startup

I was asked to design a URL shortener in a fintech startup's interview.

For the basic part, I proposed using a MD5 kind of algorithm to generate a hash of the url, encode it and then take first 7 bytes of the string.

Next the interviewer asked me the following questions:

  1. What is the probability of collisions in MD5. Based on the discussion at: https://stackoverflow.com/questions/201705/how-many-random-elements-before-md5-produces-collisions, I proposed that if the scale of the application is large, then we should use some other hashing function.

However, the interviewer mentioned that any hashing function would always produce collisions, so we can not use any hashing function while designing a URL shortener.

Somehow I could not agree with the statement. I proposed that we can append a pre-defined sequence number to the URL string and then regenerate the hash and check if it collides or not. In case it collides, then increase the sequence number and go on.

Question:

  1. Does any commercial URL shortening service actually use MD5 or SHA1 or any hashing function?
  2. In case they use a hash function what it is and how do they handle collisions
  3. In case none of the commerical URL shortening API use a hash function, then why does material like g_r_o_k_k_i_n_g or d_o_n_n_e_m_a_r_t_i_n describe using a hash function to generate shortened URLs?
Comments (16)