Interview Experience
I recently had my first online interview at Amazon’s Bengaluru office recently in May 2025. I have 4 years of industry experience as a Data Scientist in a mid sized company. The round was focused on theoretical understanding of large language models (LLMs), transformer internals, and general ML/statistics concepts.
Here are some of the questions I was asked:
🔹 Q1: QKV in BERT vs GPT
Description:
I was asked to explain the concept of QKV (Query, Key, Value) in the attention mechanism, and specifically how it’s implemented and used differently in BERT vs GPT.
Follow-up:
Why does GPT only use causal masking? What architectural changes exist in decoder-only vs encoder-decoder models?
⸻
🔹 Q2: Point Estimators, Bias-Variance Tradeoff
Description:
They wanted to check my statistical intuition. I was asked to define a point estimator, and explain the bias-variance tradeoff with relevant examples.
Follow-up:
What would you prefer in a low-data regime: high bias or high variance model?
⸻
🔹 Q3: Source of Stochasticity in LLMs
Description:
What are the key sources of stochastic behavior in LLMs during inference?
Follow-up:
How does temperature sampling and top-k/top-p affect generation?
⸻
🔹 Q4: How do LLMs perform so well with little data?
Description:
This was more of a conceptual discussion. I was asked:
“How do LLMs generalize well with very little task-specific data during fine-tuning or prompting?”
Follow-up:
Why doesn’t the classical bias-variance limitation seem to apply here?
Update
Further round scheduled which would focus on the coding aspects alongwith deep dive into resume projects.
Other Interviews:
- First Round | SWE III ML | Bengaluru
- Googleyness Round | SWE III ML
- Hiring Manager Round | Gemini Applications Team
Other Tech Firms
- Amazon | Interview (1st Round) | Applied Data Scientist | Bengaluru
- Freshworks | OA (1st Round) | Data Scientist | Bengaluru
- Razorpay | OA (1st Round) | Senior MLE | Bengaluru
- Avaamo | Interview | Senior MLE | Bengaluru
- Visa | Coding Round | Senior Data Role | Bengaluru