Gligorijevic, J.*, Gligorijevic, Dj.*, Stojkovic, I., Bai, X., Goyal, A., Obradovic, Z. (2019) “ Deeply Supervised Model for Click-Through Rate Prediction in Sponsored Search,” Data Mining and Knowledge Discovery, Springer, 2019, April 3, doi 10.1007/s10618-019-00625-3.
*Authors contributed equally.
(Impact Factor: 2.481)


In sponsored search it is critical to match ads that are relevant to a query and to accurately predict their likelihood of being clicked. Commercial search engines typically use machine learning models for both query-ad relevance matching and click-through-rate (CTR) prediction. However, matching models are based on the similarity between a query and an ad, ignoring the fact that a retrieved ad may not attract clicks, while click models rely on click history, being of limited use for new queries and ads. We propose a deeply supervised architecture that jointly learns the semantic embeddings of a query and an ad as well as their corresponding CTR. We also propose a novel cohort negative sampling technique for learning implicit negative signals. We trained the proposed architecture using one billion query-ad pairs from a major commercial web search engine. This architecture improves the best-performing baseline deep neural architectures by 2% of AUC for CTR prediction and by statistically significant 0.5% of NDCG for query-ad matching.