macOSpilot is a personal AI assistant for macOS that can answer any questions you have within any application. By using a keyboard shortcut, you can trigger the assistant and speak your question, and it will provide the answer in context and in audio format within seconds. The assistant takes a screenshot of your active window and sends it along with the transcript of your question to OpenAI GPT Vision. The answer is then displayed in text and converted into audio using OpenAI TTS. This application works with any macOS application and runs on NodeJS/Electron. You can install it by following the provided instructions and configure it to your preferences. The author of this project is a self-taught developer who enjoys creating fun projects and sharing them with others.
https://github.com/elfvingralf/macOSpilot-ai-assistant