Running large language model (LLM) workloads in-house is one of several patterns teams adopt alongside managed API services. Managed API services are convenient and well suited to many workloads. Self-hosting is a complementary option that some teams choose for reasons such as cost predictability at high request volumes, more control over latency, and controlling data-residency to meet contractual...