EvalCore

5.0 0 reviews
9 Views 2026-09-21
Visit site

About This Site

Offline snapshot testing for AI behavior EvalCore is a single-binary test runner for LLM apps and agents. It lets you define test cases and scorers in YAML, record live behavior, and replay it offline in CI to catch regressions.

Alternatives

Key Features AI

Core Features
Single-binary runner for LLM apps and agents
YAML-based eval definitions and scorers
Record and replay model calls with offline deterministic CI runs
Support for OpenAI-compatible, REST, shell, and trace-based targets
Baseline checks and regression gating
Model comparisons, trials, and reporting
Local SQLite cassette storage
OpenTelemetry and OpenInference trace support
Advantages
Deterministic offline replay makes CI fast and cost-free
Works with any language through REST or shell targets
No SDK or custom test harness required
Supports baselines, comparisons, and trace recording
Open source and dependency-free as a single binary
缺点:Requires an initial live run to record cassettes
缺点:Best suited for regression testing rather than interactive experimentation
缺点:YAML and dataset setup may take some initial configuration
缺点:Recorded outputs can become stale if the underlying behavior changes significantly

EvalCore Reviews (0)

5.0 0 reviews
  • No reviews yet. Be the first to write one!

30-Day Click Trend

2026-10-10: 2 clicks 2026-10-13: 1 clicks 2026-10-14: 1 clicks 2026-10-15: 2 clicks 2026-10-20: 1 clicks
10-07 10-21 11-05

Related Sites

Play strategy games against AI and evaluate multi-agent model behavior Olam Labs evaluates AI models in multi-agent simulations focused on capability, behavior, and social intelligence. Its Multi-Agent Arena lets people play social strategy games such as Diplomacy, Poker, and Risk against frontier language models, which negotiate and strategize alongside human players. Olam Labs also works with AI labs and researchers on evaluations and training in simulated environments where agents can cooperate, deceive, and negotiate.
AvantaGPT is an unrestricted AI platform built for developers, cybersecurity researchers, Linux users, and technical power users. AvantaGPT provides flexible AI assistance for coding, cybersecurity research, Linux development, debugging, automation, and complex technical workflows. Designed for users who want fewer restrictions and more control, AvantaGPT supports an OpenAI-compatible API and can integrate with tools such as OpenCode. Use it to analyze code, develop scripts, research vulnerabilities in authorized environments, troubleshoot systems, and work through complex technical problems. AvantaGPT is built around the idea that AI should fit into the tools and environments technical users already rely on.
Receive SMS verification codes online with virtual phone numbers from 180+ countries. DogeSMS lets you receive one-time verification codes (OTP) online using temporary virtual phone numbers. Pick a country and a service, get a number in seconds, and the SMS code shows up in your dashboard. It covers 180+ countries and 5,000+ services such as Telegram, WhatsApp, Google and more, with pay-per-use pricing from $0.05 and number rental for longer-term use. A REST API is available for developers and automation workflows, so verification steps can be handled programmatically.
Platform for comparing AI agents on real computer-use tasks Coarena by Coasty is a computer-use evaluation arena where users watch multiple AI agents complete real computer tasks side by side. It enables users to compare agents based on speed, accuracy, reliability, and task completion quality, then vote for the better-performing agent. The platform focuses on everyday workflows across browsers, applications, and enterprise software rather than synthetic benchmarks.
AI codebase auditing and verified fixes delivered as human-reviewed GitHub pull requests. Hyrax is an AI-powered codebase analysis and improvement platform for engineering teams. It connects to GitHub, maps repository structure, and uses specialized agents to identify issues across security, correctness, performance, maintainability, architecture, and operations. Before reporting a problem, Hyrax writes a failing test to validate it. It then prepares a fix and checks it against the repository's tests, build, lint, and additional verification steps before proposing it as a pull request. Engineers review and merge every change; Hyrax does not merge autonomously.
Platform for building, evaluating, and improving AI agents for business automation. Foundry is a platform to build, evaluate, and improve AI agents that can automate key parts of your business—customer support, hiring, sales, and more. Foundry is a platform for building and evaluating browser agents. Users can configure tasks, define evaluation criteria, and collect high-quality data for reinforcement learning (RL) and agent improvement. Our platform provides a deterministic web simulator and an annotation framework so you can collect labels, benchmark agents, and debug performance without worrying about web drift, IP bans, or rate limits.