Amazon Applied Scientist | L6 Luxembourg | Passed, freeze | 2022
Anonymous User
1082
Interview

Company: Amazon
Location: Luxembourg
Position: Senior Applied Scientist (TEN Search)
Time period: July-August 2022
Outcome: Passed but hit the hiring freeze

Phone Interview

Behavioral (~10 min)

  • Describe a project where you took a decision. What was it like?
  • Describe a case when you had a dilemma about whether to do smth yourself or delegate

ML breadth (~20 min)

  • Describe types of ML – supervised, unsupervised, and reinforcement learning. Provide examples
  • Describe your favorite ML algorithm
  • What are the cons of this algorithm?
  • Can sigmoid activation be used in neural networks? What a the cons of such a choice?
  • What’s an embedding? How are embeddings used?
  • How do we get embeddings with the BERT model?
  • How is BERT trained? What tasks is it solving?

ML depth (~30 min)

Your manager tells you to solve a product categorization task for the Amazon web shop. You are provided with 10k product descriptions classified into 100 categories. Each description contains a title, a textual description of the product, and a photo. It’s also fairly easy to get 1m more product descriptions, not classified into any categories. What are your next steps?

Coding (~25 min)

  1. A very easy one, akin to word count
  2. Validation of records in a log file, parsing those with valid ISBNs. Examples of valid ISBNs: 1-23456-789-0, 1-23456-789-x, 1234567890. Invalid ones: 1-23456-7890, 1-2345w6-789-x, 12345-6789x

2nd round

ML breadth + behavioral

ML/statistics questions:

  1. Pearson correlation
  2. Covariance vs. correlation
  3. Covariance and PCA – how are they related?
  4. The complexity of the SVD decomposition?
  5. You are given two medicines: paracetamol and aspirin. You need to conclude which one is better at fighting a fever. How would you approach this problem (Randomized Controlled Trials, stat tests)
  6. What’s the difference between t-test and z-test?
  7. What’s p-value?
  8. You are given two time series: BTC prices and news sentiments. How would you measure if there’s a price prediction signal in sentiments?
  9. Suppose BTC prices and news sentiments are correlated. How to conclude which one has a causal effect on the other? (didn’t properly answer)
  10. Suppose you build lags for both time series and measure partial correlations. How can this help in identifying the causal relation between BTC prices and news sentiments? (Came to an answer with a hint: you can build lagged series comparing yesterday’s sentiments with today’s prices and vice versa. If there’s a correlation between yesterday’s sentiment and today’s prices but no correlation between yesterday’s price and today’s sentiment, then sentiments drive prices, not vice versa)
  11. What NER metrics do you know?
  12. Why accuracy can be bad for NER? (class imbalance)
  13. The difference between micro-and macro-averaged F1 scores
  14. How to handle class imbalance with specific losses (Focal loss, answered but didn’t properly describe)
  15. What statistical tests for measuring inter-annotator agreement do you know?
  16. LSTMs vs. transformers – describe trade-offs
  17. What’s the complexity of the attention mechanism?
  18. How do you apply transformers for a task with long documents? (I mentioned longformer, chunking, and block-wise attention)
  19. Information retrieval: what’s pairwise loss? (described at a high level, mentioned Lambda MART)
  20. What’s the difference between DCG and nDCG? (described a bit clumsily)
  21. You’re given a large annotated collection of question-document pairs labeled with relevance: excellent, medium, and poor. You are also given a large click dataset. What’s the relation between click data and human-annotated data? Is having click data only sufficient? How would you train an IR model with the data at hand?

Behavioral questions:

  • You a finishing a paper but some experiments are not yet done. A deadline for an A-grade conference is approaching. You feel safer submitting to a B-grade conference with a deadline in 2 months. What would you do and why?
  • Describe your largest ever deliverable.

ML depth + behavioral

Behavioral questions:

  • What’s your most innovative idea?
  • Tell me about a time when you made some user-facing simplification. What drove the need for such a simplification?
  • Tell me about a time when you had to dig deep into the root-cause analysis. How did you know you needed to dig deeper?
  • What would you’ve done differently in any of your projects?

ML questions:

  1. You are given a large collection of user click data for a large set of search queries and documents (product descriptions). How would you come up with a formula to rank documents given a new search query? (described pairwise loss, SVM, and boosting, mentioned Lambda MART)
  2. We might have some frequent queries like “iphone X” and a long tail of infrequent queries. How would that influence your model training process? (group instances – (query, doc) tuples by queries, and only compare 2 docs for the same query)
  3. How do you adapt your search system to a new language? (multilingual models or translating descriptions to a new language, assess quality with a small set created by human translators)
  4. How would you deal with Chinese queries? (no idea, mentioned that I’ll reuse existing Chinese tokenizers)
  5. Suppose you a training an NMT model with data from 15 different sources. Now you need to extend the solution to a new source - news headlines. How would you approach this? (domain adaptation. I mentioned GPL – generative pseudo-labels for domain adaptation. However, wasn’t able to properly describe the model. So instead came up with adversarial validation to select those records in 15 sources that most resemble the new source. Turns out, the interviewer did the same in his real project)

ML breadth (one more) + behavioral

This was supposed to be an ML use case (kinda ML system design) interview but for some reason, I had another ML breadth round with some questions overlapping. Okay, that’s easier.

Behavioral questions:

  • Tell me about a time when you did something significant that was beyond your job responsibility.
  • What could you have done better in the previous example?
  • Tell me about a time when you were not satisfied with some process or status quo in your company and decided to change this.

ML questions:

  1. How does boosting work?
  2. Tell me about bias-variance error decomposition. How does it apply to boosting? Bagging? (I only struggled with bias reduction in case of boosting, pretty dumb: an ensemble of week learners decreases the bias of week learners)
  3. Rf and gradient boosting: practical trade-offs?
  4. What’s the advantage of BERT over RNNs?
  5. Why do vanilla RNNs experience vanishing gradients? (backprop in time)
  6. Can we use ReLU to fix vanishing gradients in a vanilla RNN? (struggled, never heard of ReLU being used in RNNs)
  7. How does LSTM fix the same problem?
  8. How does the attention mechanism work? (queries, keys, values, etc.)
  9. What’s the BERT’s biggest innovation? (mentioned the MLM task)

Coding I. Algorithms & DS + behavioral

Behavioral questions:

  • Tell me about a time when you had to go along with a group decision that you disagreed with. What did you do?
  • Tell me about a time when you took a big risk. How did you decide that you are taking this risk anyway?

Coding:

# We are going to be putting fruits into baskets.
# We are given as input some collection of various types of fruits
# a b c a b c g o o o o o o o o....
# We want to build baskets and put each of the input fruits into those baskets.
# A "basket" is simply some collection of fruits.
# We want to build the FEWEST number of baskets, such that each basket has no
#  more than 1 of the same fruit. 
# We want to return the collection of baskets that we built at the end.
# a b c a b c d -> [ [abc], [abc] ]

Coding II. Problem-solving + behavioral

Behavioral questions:

  • Tell me about a time when you took a calculated risk
  • Tell me about a time when you had several options and had to make a decision. How did you pick one of the options?

Coding:

# Each presentation contains a request ID and a list of impressions
# (individual widget views), each of which contains a widget index and a widget category.

#Sample item:

impessions_history = [
    { 'requestId': 'XYZ', 'impressions': [ {'widgetIndex': 'A', 'category': 'Electronics'},
    {'widgetIndex': 'B', 'category': 'Books'}, {'widgetIndex': 'C', 'category': 'Books'} ] },
    { 'requestId': 'ABC', 'impressions': [ {'widgetIndex': 'A', 'category': 'Toys'}] },
    { 'requestId': 'DEF', 'impressions': [ {'widgetIndex': 'A', 'category': 'Books'}] }
]

# 2. Set of JSON part files describing server-side clicks.
# Each click contains a request ID and widget index.

click_history = [
    { 'requestId': 'XYZ', 'widgetIndex': 'B' },
    { 'requestId': 'ABC', 'widgetIndex': 'A' }
]


# Output: File listing, for each widget category, the #clicks, #impressions,
# and CTR i.e. click-through rate (#clicks/#impressions).

out_dict = {
    'Books': {'clicks': 1, 'impressions': 3, 'ctr': 0.333333 },
    'Toys': {'clicks': 1, 'impressions': 1, 'ctr': 1.0 }, 
    'Electronics': {'clicks': 0, 'impressions': 1, 'ctr': 0.0 }
}

Additional behavioral

  • Tell me a time when you piece when you received a piece of critical feedback from your colleague. How did you handle it? What did you take out from this experience?
Comments (0)