In this research paper, the authors address the limitations of large language models (LLMs) in tasks that require extended context, such as document analysis and extended conversations. They propose a technique called virtual context management, inspired by hierarchical memory systems in traditional operating systems, to overcome these limitations. They introduce MemGPT (Memory-GPT), a system that intelligently manages different memory tiers to provide extended context within the LLM’s limited context window. MemGPT performs well in document analysis, allowing the analysis of large documents, and in multi-session chat, enabling conversational agents that can remember and evolve through long-term interactions. The authors provide the MemGPT code and data for further experimentation.
https://arxiv.org/abs/2310.08560