In this article, the authors introduce Optimization by PROmpting (OPRO), a novel approach to optimization using large language models (LLMs). They explain that while derivative-based algorithms have been successful in solving many problems, they are limited in cases where the gradient is unavailable. OPRO leverages the power of LLMs by using natural language descriptions to guide the optimization process. The authors provide examples of using OPRO for linear regression and traveling salesman problems, and also highlight its effectiveness in prompt optimization tasks. They demonstrate that prompts optimized by OPRO outperform human-designed prompts by significant margins on various benchmark tasks. This research presents an innovative solution for optimization challenges and showcases the capabilities of language models.
https://arxiv.org/abs/2309.03409