Observability 14

Full-stack cloud observability platform for monitoring infrastructure, logs, and application performance. Middleware is a full-stack cloud observability platform that boosts operational efficiency by monitoring infrastructure, logs, APM, and more. It offers AI-driven issue detection, real-time data access, and strong security.

Openlayer is an AI testing and observability platform for ML models and data. Openlayer is a powerful testing and observability platform for ML, designed for enterprises. It enables collaboration on finding and debugging issues in models and data, and committing new versions. Openlayer provides unified AI evaluation, observability, and governance across the AI lifecycle, from ML to LLMs. It supports testing, monitoring, and governing AI systems, ensuring a smooth transition from prototype to production through ongoing testing. Openlayer integrates with Git, offers SDKs, and works with various LLM providers, customizable via CLI and REST API.

Portkey: AI control panel for observing, governing, and optimizing AI apps with AI Gateway and Observability Suite. Portkey empowers AI teams to observe, govern, and optimize their apps across the entire organization with just 3 lines of code. It offers an AI Gateway, Prompts, Guardrails, & Observability Suite, helping teams ship reliable, cost-efficient, and fast apps. Portkey integrates with Langchain, CrewAI, Autogen, and other major agent frameworks, making agent workflows production-ready. It also provides an MCP client to build AI agents with access to real-world tools.

Unified cloud monitoring, observability, security, and incident response platform Datadog is a cloud-based observability, monitoring, security, and incident-response platform for infrastructure, applications, logs, networks, user experiences, software delivery, cloud costs, and AI systems. It unifies metrics, traces, logs, security signals, dashboards, alerts, integrations, and automated workflows so engineering, operations, security, and business teams can detect issues, investigate root causes, improve performance, and automate remediation across any stack.

Open-source AI platform for LLM chatbot management, observability, and evaluation. Lunary is an open-source observability and evaluation toolkit for AI developers, providing log queries and tools to improve app quality. It's an AI developer platform designed to manage and improve LLM chatbots, offering chatbot analytics, internal knowledge management, customer support solutions, and autonomous agent deployment.

AI-native, open-source observability for automated bug fixing and debugging. Traceroot is an AI-native, open-source observability tool that connects logs, traces, metrics, code, and team discussions. It doesn’t just summarize issues — it helps fix them by creating GitHub issues and PRs in a developer-friendly workflow. TraceRoot.AI is an AI-enhanced production debugging platform that visualizes logs, traces, and function calls in an interactive tree structure with contextual insights. It utilizes AI agents that automatically fix production bugs by analyzing structured logs and traces.

LiteLLM: LLM Gateway for managing and accessing 100+ LLMs in OpenAI format. LiteLLM is an LLM Gateway (OpenAI Proxy) designed to manage authentication, load balancing, and spend tracking across 100+ LLMs, all while maintaining the OpenAI format. It simplifies the process of using LLM APIs from various providers like OpenAI, Azure, Cohere, Anthropic, Replicate, and Google. LiteLLM offers consistent outputs and exceptions for all LLM APIs, along with logging and error tracking for all models. It provides features like cost tracking, batches API, guardrails, model access, budgets, LLM observability, rate limiting, prompt management, S3 logging, and pass-through endpoints.

Managed ClickHouse for AI-native developers to build real-time analytics APIs. Tinybird is an infrastructure and tooling platform designed for AI-Native Developers to build and ship real-time analytics APIs on ClickHouse®. It enables users to manage and query billions of rows of data with ease, skipping backend boilerplate and accelerating software development. The platform offers a hosted OLAP database, scalable and secure REST APIs for queries, an Events API for high-volume JSON streaming, and a comprehensive suite of developer tools including a CLI, local development environment, and integrations with various data sources and dev tools.

QA and observability platform for reliable Voice AI agents. Roark is a QA + Observability Layer for Voice AI, designed to help teams ship reliable voice agents. It provides comprehensive tools for monitoring live calls, running simulations at scale, and transforming call failures into repeatable tests. Roark tracks over 40 built-in call metrics, offers multi-speaker analysis, and enables on-demand or automated evaluations. For pre-deployment, it allows stress-testing agents with simulated callers across various accents, languages, and speaking styles, using graph-based scenarios and configurable personas. Roark also features one-click native integrations with popular voice platforms like VAPI, Retell, LiveKit, and Pipecat Cloud.

Open-source LLMOps platform for reliable AI apps. Agenta is an open-source LLMOps platform designed for building reliable and robust AI applications. It provides a comprehensive suite of tools for prompt management, prompt engineering, LLM evaluation, debugging, and monitoring of complex LLM applications. The platform aims to facilitate collaboration among developers and domain experts, enabling them to ship LLM applications faster and with confidence by moving from scattered workflows to structured processes.

AI-powered practice test generator for various subjects and certification exams at a reasonable price. This website uses AI to generate practice tests on various subjects and topics. It offers practice tests and certification exams at a reasonable price, leveraging AI and automation. Exams are available for subjects like SRE, DevOps, Verilog, Distributed Computing, ML, AI, Data Analytics, Observability and Monitoring, Database Administration, and MDM.

Digma AI provides continuous feedback to developers, identifying code issues early in the SDLC. Digma Continuous Feedback (CF) enables developers to uncover code runtime regressions, anomalies and code smells, right from their IDE. Digma resolves performance issues earlier in the SDLC, preventing incidents in production and engineering disruptions. Digma’s MCP Server makes AI Coding smarter. Digma highlights the affected areas and impacted components for each code change and Pull Request, eliminating the risk of breaking changes and consequent loss of engineering time. Digma goes beyond traditional monitoring and alerting, focusing on preempting issues.

A directory for discovering AI infrastructure tools and services. Infrabase.ai is a directory for discovering AI infrastructure tools and services. It provides a resource focused on helping users find the components for their AI projects, whether they're seeking alternatives to existing solutions or exploring new AI infrastructure products. It aims to be the destination for understanding the AI infrastructure landscape.

Trainkore is a prompting and RAG platform for automating prompts and saving costs. Trainkore is a prompting and RAG (Retrieval-Augmented Generation) platform designed to automate prompts and save costs. It offers features like auto prompt generation, model switching, evaluation, observability, and a prompt playground. It integrates with various AI providers and frameworks like Langchain and LlamaIndex.