Inkling 2 Small Ties Gemini 3.8 Flash at One-Sixth the Cost

You can now download a mid-tier agent model that scores level with Gemini 3.8 Flash and run it on two B300s under an Apache 2.0 licence. At most enterprise volumes, renting it by the token is still cheaper. Thinking Machines put the Inkling 2 Small weights on Hugging Face on November 12, a week after announcing the model on November 5 with the weights held back. Artificial Analysis scores it 31 on its Intelligence Index at $1.18 a task, the same score as Gemini 3.8 Flash (high) at $7.77 a task.

Three things decide whether that changes your build-versus-buy math: what the hardware costs per month, whether Thinking Machines' own API is ready for production, and what the separate Acceptable Use Policy adds on top of Apache 2.0.

What Thinking Machines Actually Released

Inkling 2 Small is a 594-billion-parameter mixture-of-experts model with 17 billion parameters active per token, per the Hugging Face model card. It takes text and images (no audio), outputs text, and supports a context window of up to 1M tokens. The card describes a 48-layer decoder with 8 of 384 experts routed per token.

The model was trained natively in NVFP4, a 4-bit floating-point format, and that is the main checkpoint. A second MXFP8 checkpoint is published alongside it. The card says "We recommend the NVFP4 checkpoint for all use-cases" while also stating MXFP8 scores 1 to 2 percentage points higher on evals. If you have read our test of FP8, NVFP4 and INT4-AWQ, you know a two-point gap is the kind that disappears or doubles depending on the task, so measure it on yours.

The hardware floor, from the same card:

Checkpoint	Minimum aggregate VRAM	Example configuration
NVFP4	500GB	2x B300 or 4x H200
MXFP8	800GB	4x B300 or 8x H200

The card lists SGLang, vLLM, TokenSpeed, Unsloth and Transformers as supported serving stacks, with one-line launch commands but no pinned framework versions. The announcement also names hosted options: Fireworks, Together, Baseten, Nebius, Modal, NVIDIA and Cloudflare, plus fine-tuning through Snowflake Cortex Training.

---

How Good Is It Against Gemini 3.8 Flash?

On the one independent benchmark that matters most for a procurement shortlist, Inkling 2 Small ties Gemini 3.8 Flash and costs about a sixth as much per task. Artificial Analysis lists Inkling 2 Small at 31 on the Intelligence Index, $0.30 input and $1.00 output per million tokens, and $1.18 average cost per task. It lists Gemini 3.8 Flash (high) at 31, $0.75 and $3.75 per million tokens, and $7.77 per task.

The cost gap comes from price, since token use runs the other way. Inkling 2 Small used 154M output tokens to run the index, which Artificial Analysis calls "very verbose" against a 104M median; Gemini 3.8 Flash used 126M. Thinking Machines concedes the point in its own limitations section: the model "is not the most succinct communicator" and sometimes picks the wrong tool in codebases with many skills. A verbose model is cheap per token and slow per answer. Artificial Analysis measures 78 output tokens a second against 114 for Gemini 3.8 Flash.

Eden AI's roundup of Artificial Analysis figures shows where it is weak. On Terminal-Bench 4.0, Inkling 2 Small scored 13% to Claude Sonnet 5.5's 30%. Its hallucination rate fell from 61% on the previous Inkling Small to 27%, while accuracy rose only from 32% to 34%. Eden AI puts Sonnet 5.5 (medium) at $10.02 a task.

The vendor's own numbers need a discount. Thinking Machines ran its benchmark table at effort 0.99 (xhigh), temperature 1.0 and a 1M-token trajectory limit, the most generous settings available. Its headline figures (86.3% on SWEBench Pro v2 Hard, 90.2% on BrowseComp with compaction, 81.7% on ARC-AGI-2) are self-reported. Even its Artificial Analysis number shows up as 31 in the text and 30.6 in the table. We made the case for replicating vendor benchmarks on your own tasks a month ago, and it applies here too. Treat the table as a starting point for your eval, run at the effort and temperature you would actually deploy.

For comparison inside Google's own lineup, Gemini 4 Hydrogen now runs $1.87 a task, so Gemini 3.8 Flash at $7.77 is no longer the closed-model price you would actually pay at this tier.

---

When Does Self-Hosting Beat the $0.30/$1.00 API?

Self-hosting the NVFP4 checkpoint pays off only if you can keep roughly 3,900 output tokens a second flowing through two B300s around the clock. That figure is our arithmetic from published rates.

The cheapest on-demand B300 in Thunder Compute's October 9 survey is Modal at $7.10 per GPU-hour, with AWS at $17.80. Two B300s at the Modal rate cost $14.20 an hour, or about $10,366 for a 730-hour month. The H200 route, at Thunder Compute's November 13 low of $3.99 per GPU-hour from Hyperbolic, costs $15.96 an hour for four cards, or about $11,651 a month. At AWS's B300 rate the two-GPU floor is roughly $25,988 a month.

Now price the same $10,366 on Tinker, which charges $0.30 per million input tokens, $0.03 cached and $1.00 output. That buys about 10.4 billion output tokens a month before you count input, or about 8,800 tasks at Artificial Analysis's $1.18 average. Spread over 730 hours, 10.4 billion tokens is about 3,900 output tokens every second. Artificial Analysis clocks one stream at 78 tokens a second, so break-even means about 50 streams decoding at full speed all month. Below that load the API is cheaper. Our break-even analysis on gpt-oss-120b reached the same conclusion for a smaller model: utilisation decides it, and most enterprise agent workloads are bursty.

Three reasons can still justify the hardware below break-even: data that cannot leave your network, a fine-tuned variant you need to serve, or GPUs you already own and underuse. The first two are the reasons the Apache 2.0 licence matters at all.

The API has a catch of its own. Tinker's pricing page lists the $0.30/$1.00 rate under "Serverless inference (Beta)" and says production use requires joining a waitlist. Those rates are dated October 19. If you need an SLA this quarter, your realistic API options are the third-party hosts, and none of their per-token prices for this model were in the sources we checked. Ask for them in writing.

---

What the Acceptable Use Policy Adds to Apache 2.0

The weights carry an Apache 2.0 licence, and Thinking Machines also binds anyone who downloads them to a separate Model Acceptable Use Policy, last updated July 15, 2026. Its text: "By accessing, downloading, or using any Model Materials, you agree to be bound by this Model AUP." Its definition of Model Materials includes "any modified version," which likely reaches your fine-tunes, though the policy never mentions fine-tunes or derivatives by name.

Four clauses matter for enterprise deployments:

1. Automated decisions. "Do not use the Model Materials to make automated (non-human) decisions that affect an individual's rights." Credit, hiring, claims and benefits workflows need a human in the decision.
2. Professional advice. "Do not use the Model Materials to engage in unauthorized or unlicensed professional practice or advising." The model card separately warns against medical, legal or safety-critical use "without additional fine-tuning, domain-specific validation, and human oversight."
3. Your users are your problem. "If you permit others to use the Model Materials through your products or services, you are responsible for their compliance," and you must disclose "any known material limitations or risks of the AI system" to those users.
4. The terms can move. "We may update this Model AUP at any time." Continued use counts as acceptance.

The policy has no termination clause, no named enforcement mechanism, no ban on training competing models and no revenue threshold. That makes it lighter than two recent open-weight releases we covered: Moonshot's Kimi K3.1 Pro licence carries a $20 million model-as-a-service clause, and Meta's Muse Spark Open bans critical infrastructure use outright. Thinking Machines' AUP prohibits facilitating attacks on critical infrastructure, which is a much narrower restriction.

Your legal team should still read it as a contract. An AUP that the licensor can change unilaterally sits awkwardly next to a licence that cannot be revoked, and nobody has tested in court which one governs a weight file you downloaded in November.

Was It Safety-Tested Before Release?

Thinking Machines says outside testers worked on Inkling 2 Small, though the announcement does not name all of them. The announcement refers to "external testers" and reports an Apollo Research covert-action evaluation (0 with a goal, 8 without) alongside internal scores such as 97.1 on StrongREJECT. For the first Inkling in July, the lab named Scale AI, Handshake AI, FAR.AI and Apollo Research and published their scopes. That level of detail is not yet public for this model. The announcement also says the model still trails closed models such as GPT-6.1 Sol on alignment evaluations.

None of that tells you whether the file you pull is the file they tested. Microsoft's Lazarus research showed how a tampered open-weight model can carry a dormant backdoor, so mirror the checkpoint into a private registry and record its hash. The serving stack needs the same discipline. SGLang and vLLM have both shipped serious flaws this autumn (the NEAR AI SGLang breach, vLLM's October advisories), and the model card's pip install vllm with no version pin is not a production install.

What to Do Before You Commit GPUs

This Week:

1. Pull your last month of agent traffic and compute sustained output tokens per second. If the peak hour is far below 3,900 tokens a second, skip the hardware decision for now.
2. Send the AUP to legal with the automated-decisions and professional-advice clauses marked, and list which of your planned workflows touch either.
3. Ask Fireworks, Together and Baseten for their Inkling 2 Small per-token price, SLA and data-retention terms, since Tinker's serverless tier is still a beta with a waitlist.

This Month:

1. Run 50 of your own tasks through Inkling 2 Small and Gemini 3.8 Flash at the effort settings you would deploy, and record cost, tokens and pass rate per task. Include tool-heavy tasks, where the vendor admits weakness.
2. If the data must stay on-prem, rent two B300s for a week, serve the NVFP4 checkpoint on a pinned vLLM or SGLang release, and measure real throughput under your concurrency.
3. Mirror the weights from Hugging Face into an internal registry with a recorded hash before anyone builds on them.

Before Renewal:

1. Bring the $1.18 versus $7.77 per-task figures to your Gemini or other closed-model negotiation. An open model at the same index score gives you a credible alternative to point to, whether or not you deploy it.

The Bottom Line

Apache 2.0 weights at Gemini 3.8 Flash's benchmark level move the comparison from "which closed API" to "API or our own GPUs." The last US Apache 2.0 contender, Reflection's Beam, lost to Kimi K3 on 13 of 14 benchmarks in October. The economics still favour renting unless your traffic is heavy and steady or your data cannot leave the building. Price your actual tokens per second against $10,366 a month before anyone orders hardware.

Continue Reading

• Together AI and Bedrock beat self-hosted gpt-oss-120b on cost
• FP8 vs NVFP4 vs INT4-AWQ: INT4 Beat NVFP4 on 3 of 4 Models
• Moonshot Makes Kimi K3.1 Pro Weights Free for Internal Use
• Meta Releases Muse Spark Open With Critical Infrastructure Ban
• Gemini 4 Hydrogen Costs $1.87 a Task vs $7.77 for 3.8 Flash
• Vendor AI Benchmarks Need Independent Runs With Task Logs