A group of researchers working with MIT have come up with a solution to a baffling problem with ChatGPT and other large language models. As these models talk to users, they gradually start to collapse, eventually leading the bot’s performance to drop rapidly. With this solution, though, that could be a thing of the past.
The issue, the researchers note, stems from the key-value cache, which is essentially the bot’s conversation memory. When this cache becomes full and needs to…
Read the full article here