Pretrained word embeddings for image captioning

Abstract

All modern methods in one way or another related to natural language processing problems use vector representations of words. These can be either representations learned for a specific task or pretrained vector representations learned from a huge corpus of texts. Image captioning is not an exception. Mostly pretrained vector representations are not used, but they are trained along with the rest of the model during the training models that generate a textual description of an image. In this work, we decided to investigate whether the use of pretrained vector representations for words will improve the quality of the model as it did for other tasks. Our research shows that the use of such representations as Word2vec and GloVe improves the quality of the model, while GloVe embeddings are even more suitable for this task. Moreover, even greater gain is obtained if they are used as an initial approximation and fine-tuned in the process of training the entire model.