Last updated: 2026-10-05
What changed
Together AI is a cloud platform for serving and customising open-weight AI models through an OpenAI-compatible API. It offers serverless per-token inference, dedicated single-tenant endpoints, provisioned throughput and self-serve NVIDIA GPU clusters, and in August 2026 signed a $240 million IBM agreement for a B300 inference cluster due in early 2027.
About Together AI
Together AI is a San Francisco cloud platform, founded in 2022, for running, fine-tuning and training open-weight AI models without owning GPUs. It raised an $800M Series C at an $8.3B valuation on 1 July 2026, led by Aramco Ventures, and on 11 August 2026 signed a $240M multi-year agreement with IBM for a dedicated NVIDIA Blackwell inference cluster on IBM Cloud, due in the first quarter of 2027. The company itself is profiled at Together AI.
The serverless catalog is built around current open-weight families: DeepSeek V4 Flash from DeepSeek, Kimi K3 from Moonshot AI, GLM-5.3, several Qwen releases, OpenAI's open-weight GPT-OSS and Meta's Llama 3.3. Beyond serverless calls, Together sells single-tenant dedicated endpoints, provisioned throughput units, self-serve GPU clusters on current NVIDIA Hopper and Blackwell hardware, per-model fine-tuning, code sandboxes and shared storage. In its August 2026 announcement the company said the platform serves about 400 trillion tokens a month. Production users it names include Cursor, Pika Labs and NexusFlow.
Who should look elsewhere: a team that needs one popular model at the lowest latency may do better on Groq, a team that wants one key across many hosts may prefer OpenRouter, and a team running custom Python jobs rather than choosing from a model list fits Modal better. Together is also a poor place to try models without paying, because the account is prepaid from the first call. The most direct competitor is Fireworks AI, and the Fireworks AI vs Together AI comparison covers where each one wins. Other platforms in this category sit under generative AI infrastructure.
Pricing
Together AI is fully prepaid: there is no free trial, and an account needs a minimum $5 credit purchase before the API accepts a call. Credits do not expire, and access is suspended when the balance reaches zero (Together billing docs, checked 5 Oct 2026). Serverless chat bills per million tokens: Ternary Bonsai 27B is $0, Tev1 4B Experimental is $0.042 input with free output, GPT-OSS 120B is $0.15 input and $0.60 output, Llama 3.3 70B Instruct Turbo is $1.04 each way, and Kimi K3 tops the list at $3.00 input and $15.00 output.
Dedicated endpoints cost $5.49 per GPU-hour on H100 and $8.99 on B200. GPU clusters run $3.99 per GPU-hour on-demand for H100 ($1.99 preemptible, $3.19 reserved for 91-180 days), $5.99 for H200, $8.19 for B200 and $9.99 for B300. Supervised fine-tuning is priced per model, from $0.34 per million training tokens on small Qwen3.5 models ($4 minimum per job) to $5.60 on Qwen3.5 397B ($22 minimum).
Provisioned throughput is $0.05 per PTU-minute, transcription starts at $0.0015 per audio minute, video generation at $0.115 per clip (Seedance 2.5), code sandboxes at $0.0446 per vCPU-hour and shared storage at $0.16 per GiB-month (Together pricing page, checked 5 Oct 2026).
| Tier | Monthly price | What it includes |
|---|---|---|
| Prepaid credits (account access) | $5 minimum purchase | No free trial; credits never expire; API access pauses at a zero balance |
| Serverless Inference - Ternary Bonsai 27B | $0 per 1M tokens | Prism ML model, free input and output once the account holds credits |
| Serverless Inference - GPT-OSS 120B | $0.15 in / $0.60 out per 1M tokens | OpenAI open-weight model, MXFP4, function calling |
| Serverless Inference - Llama 3.3 70B Instruct Turbo | $1.04 in / $1.04 out per 1M tokens | The only Llama model on the serverless list, FP8 |
| Serverless Inference - Kimi K3 | $3.00 in / $15.00 out per 1M tokens | Moonshot AI model, $0.30 cached input |
| Dedicated Inference Endpoint | $5.49 per GPU-hour (H100) | Single-tenant GPUs with autoscaling; B200 $8.99 per GPU-hour; H200, B300 and GB200/GB300 by quote |
| GPU Cluster - On-Demand | $3.99 per GPU-hour (H100) | H200 $5.99, B200 $8.19, B300 $9.99; preemptible from $1.99; reserved H100 from $3.19 for 91-180 days |
| Fine-Tuning | From $0.34 per 1M training tokens | Per-model SFT and DPO pricing; $4 minimum per job on small models, $22 on Qwen3.5 397B |
Key Features
- Serverless Inference: Call any hosted open-source model on demand with pay-per-token pricing and no infrastructure to manage. Custom kernels for FlashAttention, Medusa, and speculative decoding cut latency versus stock serving stacks.
- Dedicated Model Inference: Deploy a model on single-tenant reserved GPU compute with guaranteed throughput, full isolation, and a production SLA, built for latency-critical workloads.
- Provisioned Throughput: Reserve fixed capacity in throughput units for models such as MiniMax M3, Kimi K3 and GLM-5.2, billed per PTU-minute, for steady high-volume traffic.
- Fine-Tuning at Scale: Fine-tune open-weight models from 0.8B to 397B parameters with SFT or DPO, as LoRA or full fine-tuning, priced per model; the company claims up to 6x higher throughput than comparable services.
- GPU Clusters and Accelerated Compute: Rent on-demand GPU clusters that scale from a handful of GPUs to large multi-node deployments, with external OIDC authentication for team access and automatic node repair.
- Batch Processing API: Process large asynchronous inference workloads, up to 30 billion tokens per model, at a discount versus real-time serverless pricing, with flexible scheduling and error recovery.
- Audio AI (TTS and STT): Real-time text-to-speech via WebSocket with Orpheus 3B and Kokoro 82M models, plus real-time speech-to-text transcription via Whisper streaming. Both support REST and streaming endpoints.
- Image and Video Generation: Serverless image generation through FLUX.2, GPT Image 2 and Gemini 3 Pro Image, and video through Seedance 2.5, Kling 2.1 and Sora 2, all billed per image or per clip.
- Managed Storage and Code Sandboxes: High-performance shared filesystem storage colocated with compute at $0.16/GiB/month with zero egress fees. Secure code sandboxes for AI agent development at $0.0446/vCPU-hour.
Pros
- Serves inference at roughly 60% lower cost than comparable closed-source APIs, per company benchmarks.
- Broad open-weight catalog spanning DeepSeek, Qwen, Kimi, GLM, GPT-OSS and Llama, all behind one OpenAI-compatible API.
- Low lock-in: hosted models are open-weight and fine-tuned weights stay with the customer, so workloads can move to other infrastructure.
- Enterprise compliance: SOC 2 Type II and HIPAA-eligible dedicated endpoints, with reserved capacity for regulated workloads.
- Runs its own published inference research, such as FlashAttention and speculative decoding, in the production stack.
Cons
- No free trial: the API needs a prepaid credit balance before the first call and stops when the balance runs out.
- Token-based pricing complexity: rates vary per model, so budgets are hard to predict for spiky workloads.
- Older models rotate out of the serverless list (Llama 3.1 and Mistral Small 3 are no longer offered there), so pinned model names can need updating.
- Developer overhead: teams select, benchmark and integrate models themselves, with no opinionated defaults for non-technical users.
- Learning curve for fine-tuning and GPU cluster management compared to fully managed services.
Data Handling
- Compliance
- SOC 2 Type II · HIPAA · ISO 27001:2022
Frequently Asked Questions
Is Together AI free to use?
No. Together AI has no free trial or free tier: you must buy at least $5 of prepaid credits before the API accepts a call (billing docs, 5 Oct 2026). Once funded, Ternary Bonsai 27B costs nothing per token, and Tev1 4B Experimental charges only for input.
What does Together AI actually cost?
Billing is pay-as-you-go from that prepaid balance, with no subscription. Per million tokens, chat runs from GPT-OSS 120B at $0.15 in and $0.60 out up to Kimi K3 at $3.00 in and $15.00 out; an on-demand H100 is $3.99 an hour, and supervised fine-tuning starts at $0.34 per million training tokens (pricing page, 5 Oct 2026).
What should you use instead of Together AI?
Fireworks AI is the closest match, selling a similar open-weight model API with fine-tuning. Groq targets raw speed on its own LPU chips, OpenRouter routes one key across many hosts, Modal suits custom Python jobs rather than a model list, and Replicate keeps the simplest API for community models.
How does Together AI compare to Fireworks AI in 2026?
Fireworks AI's own listing highlights a larger open-model catalog and fast DeepSeek throughput. Together AI counters with a wider range of fine-tuning methods (SFT, DPO, and LoRA), on-demand GPU cluster rental, and HIPAA-eligible dedicated endpoints, none of which Fireworks AI's page mentions.
How do you set up Together AI?
Create an account at together.ai, add credits under billing settings (card top-ups land within minutes, US bank transfers take one to three business days), then create an API key and call the OpenAI-compatible chat completions endpoint. Auto-recharge can refill the balance, but only with a card as the default payment method.
Top Alternatives
- Fireworks AI: Pick Fireworks AI if you want the largest open-model catalog and fastest DeepSeek throughput; pick Together AI if you need broader fine-tuning methods and dedicated HIPAA-eligible endpoints.
- Groq: Pick Groq if you need the fastest raw inference latency on custom LPU hardware; pick Together AI if you need fine-tuning, GPU cluster rental, or multi-modal generation that Groq doesn't offer.
- Modal: Pick Modal if you want general-purpose serverless compute for custom Python workloads; pick Together AI if you want a ready-made catalog of open-source models with no infrastructure setup.
- Replicate: Pick Replicate if you want the simplest pay-per-second API for community models; choose Together AI when dedicated GPU clusters, model customisation and HIPAA-eligible deployment matter.
Hokai guides covering Together AI
- What Together AI Is Actually For, Now IBM Is Betting $240 Million on It: IBM is spending $240M on Together AI's inference cluster and Together raised $800M in July. Here's what changed for buyers this week, and what did not.
- Cast.ai vs Together AI: One Rents You GPUs, the Other Makes Yours Cheaper: Cast.ai and Together AI both sit under AI infrastructure, but one rents GPU inference by the token and the other cuts costs on GPUs a team already owns.
- Fireworks AI vs Together AI (2026): Managed Platform or Neocloud?: On 7 November 2026 Fireworks AI and Together AI charge the same per token on nine shared models. The split: Together rents GPU clusters, Fireworks does not.
Also mentioned in
- Hugging Face vs OpenPipe: Which Should You Use in 2026?: OpenPipe stopped training on 30 July 2026. We compare Hugging Face TRL, CoreWeave Forge, Together AI, Fireworks and Overmind on price, RL support and data.
- Stripe Didn't Just Buy OpenRouter. It Already Owned the Money Underneath It.: Stripe's reported $7B OpenRouter buy isn't new; Stripe became its billing backbone in January 2026. What that means for neutral AI routing, and what to watch.
- Best Generative AI Infrastructure in 2026: 7 Platforms, Three Separate Decisions: Fireworks, Together, Groq, Modal, Replicate, Archil and Cast.ai re-checked on 7 Nov 2026: token prices, GPU hourly rates, plan changes and the best fit.