I see LLMs as "language simulators". They just execute language in - language out, but they have no space to memorise all the facts in the world. Each token "touches" the whole network, so you could say it visits the whole culture before being created.