Evaluation 9
Open-source platform for tracing and evaluating AI applications.
Laminar is an open-source platform designed for tracing and evaluating AI applications. It allows developers to build, deploy, and monitor next-generation AI apps with features like complex async pipelines, a fast Rust backend, version control for production releases, and zero-latency-overhead logs for monitoring each pipeline node. Laminar provides observability, evals, and browser agent capabilities to help developers build reliable AI products.
Terracotta simplifies LLM experimentation with a unified platform for training and evaluation.
Terracotta is a platform designed to simplify and accelerate the process of experimenting with Large Language Models (LLMs). It streamlines the training and evaluation of LLMs by providing a unified interface for managing models, data, and evaluations. Terracotta aims to supercharge the LLM iteration process, making it more intuitive and efficient.
Coval: AI agent simulation and evaluation platform for faster, reliable development.
Coval is a simulation and evaluation platform designed to help developers build reliable voice and chat agents faster. It offers tools to create custom metrics, run thousands of scenarios, trace workflows, and integrate with CI/CD pipelines for actionable insights and peak agent performance. Coval allows users to test, monitor, and optimize their AI agents at scale, simulating conversations with customizable voices and environments. It also provides features for monitoring production calls, defining alerts, and analyzing performance to optimize AI agents.
Create reusable scorecards to assess candidates, skills, risks, and vendors.
GetScorecard allows you to create reusable scorecards and use them to assess candidates, employee skills, risks, vendors and more. Share reports with anyone via a secure link or export them as pdf or text.
Appen provides data and services to improve AI model performance and accelerate AI development.
Appen provides high-quality, scalable data for AI models and applications. They offer an end-to-end platform, flexible services, and deep expertise to ensure the delivery of diverse data crucial for building foundation models and enterprise-ready AI applications. Appen supports the AI lifecycle by providing software to collect, curate, fine-tune, and monitor traditionally human-driven tasks, creating efficiencies through a trustworthy process. They offer AI training data, data annotation, data collection, LLM training data & services, multilingual AI, evaluation & benchmarking, supervised fine tuning, and off-the-shelf datasets.
Open-source AI platform for LLM chatbot management, observability, and evaluation.
Lunary is an open-source observability and evaluation toolkit for AI developers, providing log queries and tools to improve app quality. It's an AI developer platform designed to manage and improve LLM chatbots, offering chatbot analytics, internal knowledge management, customer support solutions, and autonomous agent deployment.
Parea AI: Experimentation and human annotation platform for AI teams to ship LLM apps.
Parea AI is an experimentation and human annotation platform designed for AI teams. It provides tools for experiment tracking, observability, and human annotation, helping teams confidently ship LLM applications to production. Parea AI offers features such as auto-creating domain-specific evals, performance testing and tracking, debugging failures, human review, prompt playground, deployment tools, observability, and dataset management.
AI-powered platform for evaluating and improving startup pitch decks.
PitchLeague is a platform that uses AI to evaluate and score pitch decks submitted by startup founders. It analyzes the structure, design, messaging, and overall impact of pitch decks, providing actionable feedback to improve them. It also allows founders to see how they compare to other startups and increase visibility with investors.
Open-source playground for testing and evaluating LLMs.
OpenPlayground is an open-source playground designed for testing Large Language Models (LLMs). It provides a platform to experiment with and evaluate different GPT models.
