AI testing 33

All-in-one LLM evaluation platform for testing, benchmarking, and improving LLM application performance. Confident AI is an all-in-one LLM evaluation platform built by the creators of DeepEval. It offers 14+ metrics to run LLM experiments, manage datasets, monitor performance, and integrate human feedback to automatically improve LLM applications. It works with DeepEval, an open-source framework, and supports any use case. Engineering teams use Confident AI to benchmark, safeguard, and improve LLM applications with best-in-class metrics and tracing. It provides an opinionated solution to curate datasets, align metrics, and automate LLM testing with tracing, helping teams save time, cut inference costs, and convince stakeholders of AI system improvements.

AI-powered test automation platform for web, mobile, API, and desktop applications. Testnut is an AI-powered test automation platform that simplifies testing for web, mobile, API, and desktop applications. It seamlessly integrates with CI/CD pipelines, reduces flaky tests, and accelerates release cycles. Testnut provides testing solutions and support, helping users with software, mobile, and API testing tools.

End-to-end testing platform for generative AI models. Contentable.ai is an end-to-end testing platform for generative AI. It allows users to A/B test multiple AI models, compare speed, costs, and outputs, and build and deploy low-code AI models. It supports major AI vendors and facilitates rapid prototyping and collaboration on AI models.

A unified platform for accessing, testing, and comparing multiple AI models. GiGOS brings all AI models together in one platform, making it easy to use without the hassle. GiGOS simplifies access to multiple AI models through a unified platform, addressing fragmentation and complexity in the AI landscape. GiGOS lets you test, battle and compare the best AI models, all in one place.

Platform to evaluate and test LLM-powered applications for faster and reliable product releases. Ottic is a platform designed to evaluate LLM-powered applications, enabling both tech and non-technical teams to test their LLM apps and ship reliable products faster. It offers features like visual prompt management, end-to-end test management, comprehensive LLM evaluation, and user behavior monitoring. Ottic aims to provide a 360º view of the QA process for LLM applications, ensuring quality and reducing time to market.

AI-powered QA agent for mobile app end-to-end testing. Autosana is an AI-powered QA agent designed for mobile applications. It integrates with CI/CD pipelines to create and run end-to-end tests using natural language, eliminating the need for coding. Its self-healing tests adapt to UI changes, ensuring test stability. Autosana supports all major mobile frameworks (iOS, Android, Flutter, React Native, Swift, Kotlin) and provides features like session replay and nightly runs. It aims to accelerate deployment times and proactively identify bugs, reporting failures directly to teams via email or Slack.

AI copilot for automated software unit testing, especially for React components. Supertest is an AI copilot for software testing, designed to automate QA engineering tasks, particularly writing unit tests. It offers a VS Code extension that allows users to generate unit tests in seconds with a single click. Supertest currently supports React Testing Library and plans to expand to support multiple testing libraries.

A platform for evaluating and optimizing generative AI applications. EvalsOne is a platform designed to streamline the process of prompt evaluation for generative AI applications. It provides a comprehensive suite of tools for iteratively developing and perfecting these applications, offering functionalities for evaluating LLM prompts, RAG flows, and AI agents. EvalsOne supports both rule-based and large language model-based evaluation methods, seamless integration of human evaluation, and various sample data preparation methods. It also offers extensive model and channel integration, along with customizable evaluation metrics.

Automated AI answer verification service using C3-score for accuracy and efficiency. EvalMy.AI is a service that automates AI answer verification using the C3-score metric (correctness, completeness, contradiction). It helps users quickly identify where their AI might fall short, reducing friction and speeding up testing. It offers automated RAG assessment, accuracy prioritization, configurability with customizable Sem-Score parameters, scalability as a cloud-based SaaS, and pluggability with a user-friendly API that integrates into CI/CD pipelines and supports popular ML tools like LangChain.

Cloud-based testing platform for conversational AI and LLM-powered chatbots. LLM Tester is a cloud-based testing platform designed for teams developing conversational AI and LLM-powered chatbots. It's a SaaS solution that allows users to create, manage, and automate conversation flow tests for their bots. The platform helps evaluate, compare, and improve language models through automated conversation flows, offering comprehensive tools to assess and enhance AI conversational models.

AI-powered testing framework for writing end-to-end tests in plain English. Shortest is an open-source AI-powered testing framework that allows developers to write end-to-end tests in plain English. It's built on Playwright and offers seamless GitHub integration. Shortest aims to run CI suites faster than ever by enabling QA via natural language and letting AI handle the execution of tests.

AI-powered testing automation platform for efficient, low-code end-to-end software testing. BotGauge is a Gen AI-powered testing automation platform designed to enhance testing efficiency, reduce costs, and accelerate time-to-market. It uses AI test agents to automate UI, functional, API, and database testing. The platform enables anyone to automate end-to-end tests, even without coding experience, by generating test cases from PRDs, UX screens, and documents.

KushoAI is an AI agent for API testing, generating test suites in minutes. KushoAI is an AI Agent for API testing. It generates an exhaustive test suite for your API in 2 minutes by inputting an API spec and provides functional tests covering real-world scenarios. These tests can be run with AI-generated assertions in a single click. KushoAI transforms inputs into a comprehensive ready-to-run test suite, testing both web interfaces and backend APIs. It helps automate workflows, write AI-driven tests, and serves numerous companies.

Open-source tool for automated head-to-head evaluation of GenAI systems using LLM judges. AutoArena is an open-source tool designed to automate head-to-head evaluations of GenAI systems using LLM judges. It allows users to quickly and accurately generate leaderboards comparing different LLMs, RAG setups, or prompt variations. Users can fine-tune custom judges to fit their specific needs. AutoArena facilitates trustworthy evaluation of LLMs, RAG systems, and generative AI applications through automated head-to-head judgement.

AI platform for battle-testing and improving AI agents. Janus is an advanced AI platform designed to battle-test and improve AI agents. It conducts thousands of AI simulations against chat and voice agents to surface critical failures such as hallucinations (fabricated content), rule violations (policy breaches), and tool-call/performance failures. Janus offers custom evaluations, personalized datasets, and actionable insights to help users detect and mitigate risky agent behavior, ensuring model reliability and performance.

AI simulation environment for testing LLM apps at scale. Snowglobe is a simulation environment for LLM teams designed to test how their AI applications respond to real-world user behavior. It enables users to run full workflows through realistic scenarios, catch edge cases early, and confidently improve model performance before deploying to production. Snowglobe helps AI teams test LLM apps at scale by simulating real-world conversations, uncovering risks, and improving overall model performance.

AI-powered QA platform for automated software testing and high-velocity QA services. QASolve is an AI Augmented Software testing & Automation Platform that generates and maintains test cases with new software releases leveraging Gen AI, CV and NLP. It offers cost-effective, high-velocity QA services, generating 80%+ test automation in 1 week. Expert engineers curate a list of issues with every sprint within 1 business day. QASolve provides high-coverage automated regression tests that can be exported to Playwright or Selenium.

End-to-end AI evaluation and observability platform for testing and deploying AI applications. Maxim is an end-to-end AI evaluation & observability platform that helps you test and deploy AI apps with greater speed & confidence. Its developer stack includes tools for the full AI lifecycle: experimentation, pre-release testing, & post-release monitoring. It offers features like agent simulation and evaluation, prompt engineering tools, observability, and continuous quality monitoring. Maxim supports various AI frameworks and provides SDKs, CLI, and webhook support.

AI-driven E2E testing tool for B2B SaaS, identifying bugs and scaling testing efficiently. QA.tech is an AI-driven E2E testing tool designed for B2B SaaS web applications. It helps identify bugs quickly, scale end-to-end testing without increasing headcount, and allows teams to focus on building and coding. It uses AI agents to write and run tests, providing detailed bug reports and integrating with CI/CD pipelines.

AI-powered testing platform for faster, smarter, and more affordable software testing. Webo.AI offers a cloud-based, AI testing service solution that provides a faster, smarter & more affordable way to test for startups. It creates test strategies, test cases, and automation. It also provides unlimited test automation executions and AI healing. Webomates provides a faster, smarter, and more affordable way to test for startups, helping them fast-track their product to market. It automates testing for web applications and helps overcome software testing challenges.

AI-powered platform for code review, testing, and generation to improve code quality. Qodo (formerly Codium) is an agentic code integrity platform designed for reviewing, testing, and writing code. It integrates AI across development workflows to strengthen code quality at every stage. Qodo provides AI-powered tools for code review, testing, and code generation, aiming to boost productivity and code quality. It supports various programming languages and integrates with IDEs like VSCode and JetBrains, as well as Git providers like GitHub and GitLab.

BotLab helps understand and test bots in simulated game environments. BotLab helps you understand bots and test their reliability, performance, and safety. It allows you to simulate game clients, enabling you to test bots in any scenario within seconds without risk. You can observe a bot's behavior in challenging situations without needing to read code.

AI-powered, no-code platform for automated UI testing of web and mobile apps. Autonoma AI is an AI-powered platform designed for building and running end-to-end tests for web and mobile applications. It offers a no-code, self-healing solution that enables developers to understand legacy, spaghetti, or throwaway code. It generates a wiki with API Docs, feature descriptions, and implementation details, maintaining the original style, language, and dependencies.

AI-powered platform for fully automated end-to-end software testing solutions. TestSprite is an AI-powered platform that provides fully automated end-to-end testing solutions for software development managers. It offers end-to-end software testing by building test plans, writing code, executing tests, debugging, and reporting—all with minimal input. It is ideal for dev teams needing self-serve testing across multiple scenarios.