New study explores how much LLMs memorize! Researchers found memorization happens, but isn't as widespread as some feared. Interesting insights from Meta, Google, Nvidia & Cornell.
- LLMs *do* memorize data, but less than previously thought.
- Memorization scales with model size, but isn't linear.
- Researchers developed methods to measure memorization effectively.