They however have many issues with doing so, the most important here being the need for prior knowledge to understand context. Semantic meaning in text based on the principles of the distributional hypothesis that underlies embeddings has a flaw in that it doesn’t consider context not directly in the text to ascribe meaning.

For example if I wrote everyday on reddit “God I love pizza” you wouldn’t be able to tell without prior context if I’m an Italian food fiend or I have a cat named Pizza.

While LLMs are good patern recognition machines they don’t inherently create valid correlations and can’t understand causation.