Last updated: 2026-08-19
Together AI is an AI-native cloud platform for serving, fine-tuning, and deploying open-source models. It hosts more than 200 models, including Llama, DeepSeek, Qwen, and Mistral, layering in custom inference kernels like FlashAttention and speculative decoding to cut latency versus stock serving stacks.
About Together AI
Together AI is a research-driven cloud platform founded in 2022 that lets developers train, fine-tune, and deploy open-source AI models at scale. The company has raised $800M in a Series C round (July 2026, $8.3B valuation), following a $305M Series B in February 2025 ($3.3B valuation) and earlier rounds, from investors including NVIDIA, Salesforce Ventures, Kleiner Perkins, and General Catalyst.
Production users of the platform include Cursor, Pika Labs, and NexusFlow.
Pricing
50/1M output tokens (largest frontier models). Batch API at up to 50% discount. 00/1M (70B-100B).
09/hr (H100 181+ days). 99/hr. 0015/min.
0006/image. 14/video. Free tier with starter API credits for new accounts.
| Tier | Monthly price | What it includes |
|---|---|---|
| Serverless Inference - Llama 3.1 8B | Free | Pay-per-token via API |
| Serverless Inference - Llama 3.1 70B | Free | Pay-per-token via API |
| Serverless Inference - Llama 4 Maverick | Free | Pay-per-token via API |
| Serverless Inference - Mistral Small 3 | Free | Pay-per-token via API |
| Dedicated Inference Endpoint | Free | Hourly reserved capacity: H100 HGX $5.49/hr, B200 $8.99/hr; contact for higher-tier hardware |
| GPU Cluster - On-Demand | Free | Per GPU-hour on-demand: H100 $3.99, H200 $5.99, B200 $8.19; reserved discounts from $3.09/hr (H100, 181+ days) |
| Fine-Tuning | Free | Per million training tokens: $0.48-$1.35 (up to 16B), $1.50-$4.12 (17B-69B), $2.90-$8.00 (70B-100B); $4 minimum per job |
Key Features
- Serverless Inference: Call any hosted open-source model on demand with pay-per-token pricing and no infrastructure to manage. Custom kernels for FlashAttention, Medusa, and speculative decoding cut latency versus stock serving stacks.
- Dedicated Model Inference: Deploy a model on single-tenant reserved GPU compute with guaranteed throughput, full isolation, and a production SLA, built for latency-critical workloads.
- Fine-Tuning at Scale: Fine-tune open-source models from 8B to 405B parameters using SFT, DPO, or LoRA, with up to 6x higher throughput than comparable alternatives per company benchmarks.
- GPU Clusters and Accelerated Compute: Rent on-demand GPU clusters that scale from a handful of GPUs to large multi-node deployments, with external OIDC authentication for team access and automatic node repair.
- Batch Processing API: Process large asynchronous inference workloads, up to 30 billion tokens per model, at a discount versus real-time serverless pricing, with flexible scheduling and error recovery.
- Audio AI (TTS and STT): Real-time text-to-speech via WebSocket with Orpheus 3B and Kokoro 82M models, plus real-time speech-to-text transcription via Whisper streaming. Both support REST and streaming endpoints.
- Image and Video Generation: Serverless image generation via FLUX.1 and Google Imagen 4.0 Ultra. Video generation via Seedance 2.0 (text-to-video, image-to-video, 4K up to 3840x2160) and Google Veo 3.0 with audio.
- Managed Storage and Code Sandboxes: High-performance shared filesystem storage colocated with compute at $0.16/GiB/month with zero egress fees. Secure code sandboxes for AI agent development at $0.0446/vCPU-hour.
Pros
- Serves inference at roughly 60% lower cost than comparable closed-source APIs, per company benchmarks.
- Largest open-source model catalog among GPU-cloud competitors, with native support for the Llama, Mistral, Qwen, and DeepSeek families.
- True vendor independence: every hosted model is open-source, fine-tuned weights are user-owned, and workloads can be redeployed on other infrastructure without lock-in.
- Enterprise-grade compliance: SOC 2 Type II and HIPAA-eligible dedicated endpoints, with reserved capacity for regulated workloads.
- Ships published inference-research optimizations to production faster than most GPU-cloud competitors, translating into consistently faster response times.
Cons
- Token-based pricing complexity: variable rates per model make budget prediction difficult, with no fixed-price tier for unpredictable workloads.
- Smaller ecosystem of integrations vs. OpenAI/Claude; LangChain/LlamaIndex support requires additional setup.
- Developer overhead: users must select, benchmark, and integrate models themselves, with no opinionated defaults for non-technical teams.
- Learning curve for fine-tuning and GPU cluster management compared to fully managed services.
Data Handling
- Compliance
- SOC 2 Type II · HIPAA · ISO 27001:2022
Frequently Asked Questions
What does Together AI actually cost?
Together AI uses pay-as-you-go pricing with no monthly subscription. Serverless inference starts around $0.03 per million input tokens for small models and scales up for larger ones, fine-tuning starts at $0.48 per million training tokens with a $4 minimum per job, and on-demand GPU clusters start at $3.99 per GPU-hour. Batch requests get up to 50% off standard serverless rates.
Is Together AI free to use?
Together AI gives new accounts starter API credits to test the platform, but there is no recurring monthly free allowance once those credits run out. After the credits are used, everything bills at the standard pay-per-token or pay-per-GPU-hour rates, with no minimum monthly spend required.
What should you use instead of Together AI?
Fireworks AI, Groq, Modal, and Replicate are the closest competitors. Groq and Fireworks AI both push hard on raw inference speed, Modal leans toward general-purpose cloud compute rather than a model catalog, and Replicate keeps its API the simplest of the four. Together AI differentiates by bundling fine-tuning, GPU cluster rental, and multi-modal generation on one platform.
How does Together AI compare to Fireworks AI in 2026?
Fireworks AI's own listing highlights a larger open-model catalog and fast DeepSeek throughput. Together AI counters with a wider range of fine-tuning methods (SFT, DPO, and LoRA), on-demand GPU cluster rental, and HIPAA-eligible dedicated endpoints, none of which Fireworks AI's page mentions.
How do you set up Together AI?
Create an account on together.ai to receive starter API credits, then call any hosted model through the OpenAI-compatible chat completions endpoint with an API key. From there, most developers benchmark a few candidate models for their use case before moving to fine-tuning or a dedicated endpoint for production.
Top Alternatives
- Fireworks AI: Pick Fireworks AI if you want the largest open-model catalog and fastest DeepSeek throughput; pick Together AI if you need broader fine-tuning methods and dedicated HIPAA-eligible endpoints.
- Groq: Pick Groq if you need the fastest raw inference latency on custom LPU hardware; pick Together AI if you need fine-tuning, GPU cluster rental, or multi-modal generation that Groq doesn't offer.
- Modal: Pick Modal if you want general-purpose serverless compute for custom Python workloads; pick Together AI if you want a ready-made catalog of open-source models with no infrastructure setup.
- Replicate: Pick Replicate if you want the simplest pay-per-second API for community models; pick Together AI if you need fine-tuning, dedicated GPU clusters, and HIPAA-eligible deployment options.
HokAI guides covering Together AI
- What Together AI Is Actually For, Now IBM Is Betting $240 Million on It: IBM is spending $240M on Together AI's inference cluster and Together raised $800M in July. Here's what changed for buyers this week, and what did not.
- Cast.ai vs Together AI: One Rents You GPUs, the Other Makes Yours Cheaper: Cast.ai and Together AI both sit under AI infrastructure, but one rents GPU inference by the token and the other cuts costs on GPUs a team already owns.
- Fireworks AI vs Together AI: Which Should You Use in 2026?: Fireworks and Together AI now charge the identical per-token rate on DeepSeek V4 Pro, so price no longer separates these two open-model inference platforms.
- Stripe Didn't Just Buy OpenRouter. It Already Owned the Money Underneath It.: Stripe's reported $7B OpenRouter buy isn't new; Stripe became its billing backbone in January 2026. What that means for neutral AI routing, and what to watch.
- Best Generative AI Infrastructure in 2026: 7 Platforms, Three Separate Decisions: Fireworks AI, Together AI, Groq, Modal, Replicate, Archil and Cast.ai compared on price and speed, plus the storage and cost layers most 2026 roundups skip.
- CometAPI vs Orthogonal: Same Gateway Label, Two Different Jobs: CometAPI and Orthogonal both call themselves a unified AI gateway, but one routes 500+ models and the other gives agents pay-per-call access to 40+ tools.