Virtual participation: Zoom details available here
Speaker: Professor John Kelleher, Maynooth University
Title: "Probing Neural Language Model Embeddings"
Abstract: Neural language models have become a fundamental building block in modern natural language processing systems such as ChatGPT and machine translation. A challenge in using a neural network to process language is that neural models use a vector-based representation of data and so we need to develop methods to translate words and sentences into vectors for a neural network to process them. These vector-based representations of language are known as embeddings. These embeddings, however, are not directly interpretable by humans. Consequently, once a text has been translated into an embedding it is natural to wonder whether the embedding has accurately captured the meaning of the original text. This seminar will introduce neural language models and embeddings and present a set of experiments focused on understanding what information is encoded in neural embeddings and where in an embedding is information encoded.
Biography: John Kelleher is a Hamilton Institute Professor of Computer Science at Maynooth University. He is a PI in the SFI ADAPT Research centre where he leads several research projects on natural language processing. He is also a PI on a number of European projects focused on developing clinical decision support systems for stroke, and a supervisor on the SFI CRT on Foundations of Data Science. John has published a number of books on machine learning and data science, including Fundamentals of Machine Learning for Predictive Data Analytics (https://mitpress.mit.edu/9780262044691/fundamentals-of-machine-learning-for-predictive-data-analytics/), Deep Learning (https://mitpress.mit.edu/9780262537551/deep-learning/), and Data Science (https://mitpress.mit.edu/9780262535434/data-science/).