LLM inference 5

Low-cost, scalable inference for high-volume AI workloads Doubleword is an AI inference platform for high-volume workloads, offering OpenAI-compatible access to open-weight language, vision, OCR, and embedding models. It supports real-time, async, and approximately 24-hour batch inference, allowing users to trade latency for substantially lower token costs. Doubleword is designed for agentic reasoning, evaluations, synthetic data generation, annotation, extraction, coding workflows, and large-scale data processing. Its inference stack includes optimization technologies such as FlashOffload, Speculative KV Coding, and Cloudburst to improve throughput, cache efficiency, and startup times.

AwanLLM: Cost-effective, unlimited token LLM inference via monthly subscription. AwanLLM is a cloud provider for LLM inference focusing on cost and reliability. Unlike other providers, it charges monthly instead of per token by hosting its data center in strategic cities, offering unlimited tokens, unrestricted use, and cost-effective LLM inference API platform for power users and developers.

Serverless decentralized AI compute for deploying and scaling models Chutes is a decentralized serverless AI compute platform for deploying, running, and scaling open-source AI models through a web platform or API. It supports production inference, private confidential GPU workloads, and multiple modalities including image, video, speech, and music.

World's fastest AI inference provider powered by purpose-built ASICs. General Compute is a high-performance AI inference infrastructure provider built on purpose-built ASICs instead of traditional GPUs. Designed specifically for running large language models and machine learning workloads, it offers ultra-fast inference speeds reaching up to 1,000 tokens per second, sub-millisecond Time to First Token (TTFT), and high throughput. It provides an OpenAI-compatible REST API, allowing developers to seamlessly swap their inference provider by simply changing the base URL and API key. Additionally, the platform operates on highly energy-efficient, air-cooled hardware that significantly reduces infrastructure power consumption and operational costs compared to standard GPU cloud environments.

Native macOS server for fast local AI model inference oMLX is a native macOS inference server for running local AI models on Apple Silicon using Apple's MLX framework. It provides paged SSD KV caching to reduce time-to-first-token for coding agents, continuous batching for higher throughput, and OpenAI- and Anthropic-compatible APIs. oMLX supports LLM, vision-language, embedding, and reranker models, along with tool calling, MCP integration, multi-model serving, and a native menu bar application with a web dashboard.