In this article, the author discusses the concept of embeddings and their use in representing text such as words, sentences, or paragraphs. These embeddings are mathematical vectors that can be manipulated in various ways. The author highlights Lilian Weng’s project, which ranks the closest emojis to a search query based on their meaning-space. The author then introduces the idea of using arithmetic on language vectors and presents a simple app that allows users to add two emojis and see the closest known emoji to the result. The author also mentions the potential of embeddings in other media types such as audio and video. However, it is noted that the models used in these processes may reflect stereotypes and flaws present in the training data. The author concludes by mentioning the possibility of a future where machines can reason about meaning in a text without the need for traditional file organization.
https://montyanderson.net/writing/embeddings