Jetstream: Shrinking the AT Protocol Firehose by >99%

Bluesky has experienced a significant increase in activity following Brazil’s ban on Twitter, leading to a surge in traffic volume on Bluesky’s AT Proto firehose. The firehose, designed to maintain synced copies of user repos, has seen a 1,300% increase in event rate, resulting in a jump from 24 GB/day to over 232 GB/day of traffic. To address the burden of operating tools like bots and feed generators, Jetstream was introduced as a filterable JSON firehose alternative for AT Proto. By implementing zstd compression to reduce bandwidth, Jetstream offers a lightweight solution for consuming events, making it more efficient and cost-effective.

https://jazco.dev/2024/09/24/jetstream/

To top