LLM Management 2

Portkey: AI control panel for observing, governing, and optimizing AI apps with AI Gateway and Observability Suite. Portkey empowers AI teams to observe, govern, and optimize their apps across the entire organization with just 3 lines of code. It offers an AI Gateway, Prompts, Guardrails, & Observability Suite, helping teams ship reliable, cost-efficient, and fast apps. Portkey integrates with Langchain, CrewAI, Autogen, and other major agent frameworks, making agent workflows production-ready. It also provides an MCP client to build AI agents with access to real-world tools.

LiteLLM: LLM Gateway for managing and accessing 100+ LLMs in OpenAI format. LiteLLM is an LLM Gateway (OpenAI Proxy) designed to manage authentication, load balancing, and spend tracking across 100+ LLMs, all while maintaining the OpenAI format. It simplifies the process of using LLM APIs from various providers like OpenAI, Azure, Cohere, Anthropic, Replicate, and Google. LiteLLM offers consistent outputs and exceptions for all LLM APIs, along with logging and error tracking for all models. It provides features like cost tracking, batches API, guardrails, model access, budgets, LLM observability, rate limiting, prompt management, S3 logging, and pass-through endpoints.