Content clustering using word2vec

Honeyscreen, a key product of Buzzvil, serves not only ads which give reward point to users, but also various content. Users can see breaking news and hot content in their lock screen without opening social or portal apps. Likewise, serving high quality content to users is a very important task to solve to provide an improved Honeyscreen UX. To serve content efficiently, we focused on clustering content by category through machine learning. There are two approaches to cluster content: Clustering based on the semantics of content (i.e. body/title). Clustering based on the collaborative filtering. We are trying to solve the problem using various machine learning techniques. To develop a more efficient clustering method, we used many Natural Language Processing (NLP) techniques such as Latent Dirichlet allocation (LDA) and Term frequency Inverse Document frequency (TF-IDF). Id like to introduce word2vec which is an easy, powerful, and efficient method, and explain how we used word2vec to cluster content. word2vec Mikolov and Google engineers created word2vec, which is a neural network-based natural language processing algorithm that models words as vectors in high dimensional space. Over the years, word2vec has proven to bring a tremendous improvement in accuracy. word2vec assumes co-appearing words must be semantically similar, and defines the similarities of different words as the distances between corresponding word vectors. Additionally, word2vec is highly optimized so that a standard laptop can train such a model in reasonably short time. More concrete and mathematical details can be found in the paper mentioned