AI safety 14

Align API provides guardrails for AI applications, detecting sensitive content and aligning with brand values. The Align API offers plug-and-play guardrails for your AI application. It proactively identifies NSFW prompts before it gets sent for inference. This saves unnecessary cost, protects your brand and prevents you from getting banned by OpenAI, Apple or Google. It helps detect sensitive topics, align with your brand, and reduce costs. It also offers prompt cleaning and aligning with your values.

A visual deep learning framework for creating AI solutions with drag-and-drop or natural language. Graphbook is a deep learning framework with a visual interface, transforming cumbersome model development into a user-friendly experience. With Graphbook, you can edit and debug deep learning models as a drag-and-drop data orchestrator inside your models. Cerbrec Graphbook is the AI development platform for everyone to create bespoke AI solutions, by natural language queries or drag-and-drop.

Industry body focused on safe and responsible development of frontier AI models. The Frontier Model Forum is an industry body focused on ensuring the safe and responsible development of frontier AI models. It draws on the technical and operational expertise of its member companies (Anthropic, Google, Microsoft, and OpenAI) to ensure that the most advanced AI systems remain safe and secure, so that they can meet society’s most pressing needs. The Forum is an industry-supported non-profit focused on addressing significant risks to public safety and national security.

Real-time AI security platform protecting against prompt injections and jailbreaks. grimly.ai is a real-time AI security platform that protects LLMs and AI agents from prompt injections, jailbreaks, and abuse. It offers multi-layer defense mechanisms, including semantic threat detection, prompt injection firewall, flexible rule engine, visibility & logging, system prompt guard, and agent safety features. It is designed for developers, security teams, and enterprise AI adopters, providing easy deployment and integration with existing AI solutions.

Free, open-source course for learning prompt engineering and AI communication. Learn Prompting is a free, open-source course on communicating with artificial intelligence. It is the largest and most comprehensive course in prompt engineering available on the internet, with over 60 content modules, translated into 9 languages, and a thriving community. It offers AI courses, live courses, and certification exams to help learners master AI and prompt engineering.

Scale AI provides high-quality training data and platforms for AI development and evaluation. Scale AI delivers high-quality training data for AI applications such as self-driving cars, mapping, AR/VR, robotics, and more. They offer a Scale Data Engine for training models, supervised fine-tuning and RLHF, and high-quality data for the public sector and automotive industries. Scale AI also provides solutions like Scale Donovan for mission-critical Agentic AI and the Scale GenAI Platform for full-stack Generative AI. They offer evaluation of AI models and applications for model developers, the public sector, and enterprises.

AI platform for battle-testing and improving AI agents. Janus is an advanced AI platform designed to battle-test and improve AI agents. It conducts thousands of AI simulations against chat and voice agents to surface critical failures such as hallucinations (fabricated content), rule violations (policy breaches), and tool-call/performance failures. Janus offers custom evaluations, personalized datasets, and actionable insights to help users detect and mitigate risky agent behavior, ensuring model reliability and performance.

An AI model focused on ethical principles and safety, avoiding controversial topics. GOODY-2 is an AI model designed with a strong emphasis on ethical principles and safety. It avoids answering questions that could be considered controversial, offensive, or dangerous, prioritizing responsible discourse and the prevention of harm. It is marketed as an AI model suitable for enterprise use, particularly in customer service, paralegal assistance, and back-office tasks.

Imandra provides a Reasoning as a Service® platform for logical reasoning in AI systems. Imandra Inc. is the home of Reasoning as a Service®. Their platform, built on advances in automated reasoning, brings rigorous logical reasoning to modern AI systems. It is used by researchers, corporations, and government agencies worldwide. Imandra offers tools for integrating automated logical reasoning into LLMs, applications, and agentic workflows, fostering the creation of trustworthy Neurosymbolic AI systems. Their products include ICL CodeLogician, Imandra Universe, Imandra Core, Imandra Markets, and Imandra Connectivity. They provide solutions for financial services, formal verification, trustworthy AI, system integration, autonomous systems, MBSD, and government & defense.

Zeus: Fastest way to build and deploy AI agents with enterprise-grade features. Zeus is the fastest way to build AI agents. It allows users to build powerful text & voice agents, integrate them with 40+ apps, and deploy them in seconds. It offers enterprise-grade security, privacy, reliability, and scalability. Zeus supports everything large enterprises need, including SOC2 readiness, GDPR/CCPA compliance, SAML SSO, uptime guarantee, comprehensive analytics, and AI safety features.

AI evaluation and optimization platform for automated quality assessment and performance enhancement. Future AGI develops advanced AI evaluation and optimization products, enabling automated quality assessment and performance enhancement for AI models. It offers a comprehensive platform to help enterprises achieve high accuracy in AI applications across software and hardware, replacing manual QA with Critique Agents and custom metrics.

AI learning platform for kids focusing on safety and interactive games. SlowMo Club is an AI learning platform designed for children aged 6-12. It focuses on teaching kids how to use AI safely through interactive games, AI literacy lessons, and prompt engineering challenges. The platform aims to spark curiosity, encourage discovery, and promote smart choices regarding AI and digital safety.

Research organization focused on understanding and managing the risks and opportunities of advanced AI. The Centre for the Governance of AI (GovAI) is a research and field-building organization dedicated to understanding and managing the risks and opportunities posed by advanced artificial intelligence. They build a global research community to help humanity navigate the transition to a world with advanced AI. GovAI conducts research, publishes analysis, and hosts events related to AI governance and policy.

Play strategy games against AI and evaluate multi-agent model behavior Olam Labs evaluates AI models in multi-agent simulations focused on capability, behavior, and social intelligence. Its Multi-Agent Arena lets people play social strategy games such as Diplomacy, Poker, and Risk against frontier language models, which negotiate and strategize alongside human players. Olam Labs also works with AI labs and researchers on evaluations and training in simulated environments where agents can cooperate, deceive, and negotiate.