OpenAI Co-Founder Ilya Sutskever on the mystical powers of artificial neural nets
Ilya on LLMs understanding the world:
— Lior⚡ (@AlphaSignalAI) December 13, 2023
"predicting the next token well, means that you understand the underlying reality that let to the creation of that token"
Seem like the opposite view of Yann. pic.twitter.com/32MLEeDQnx
Transcription (which I found here):
Ilya Sutskever: I challenge the claim that next-token prediction cannot surpass human performance. On the surface, it looks like it cannot. It looks like if you just learn to imitate, to predict what people do, it means that you can only copy people. ...