LLMs FAIL at 2K context length - Yours Too?
Context length is a limiting factor for Large Language Models, especially for RAG systems. But 2/3 of current LLMs fail at 2K token length, in Jan 2024!
My test prompt is with 5272 characters too long to upload to Youtube.
I use a prompt of 741 words, corresponding to 1254 tokens (of my adapted tokenizer).
Every tested models applies its specific tokenization.
It is amazing, that open source LLM outperform even huge commercial LLMs.
By the way: A year ago I explored the "Lost in the Middle" phenomenon already in detail in my video: • LLM Performance Secrets | Jailbroken ...
Including benchmark data for LLM systems at the time.
#aieducation
#performancetest
#best
