CFContextFair Runs in your browser
Plan multilingual AI conversations

What will your AI
conversation forget?

Plan which messages fit before you send them. Protect your system instruction, reserve a reply, and see exactly which older exchanges get dropped—then check the decision with a real tokenizer.

Plan a conversation Explore the token lab
01

Count transparently. Estimates are labelled as estimates. On-demand exact verification is labelled separately.

Conversation budget planner

See what stays. See what goes.

Local processing · no inference API

Start with an illustrative JEE tutoring conversation, edit the messages, or paste your own JSON. The system instruction and newest exchange are protected. Older user-led exchanges are removed together, oldest first.

Your conversation

Illustrative messages, not real student data. Reset replaces the editor contents. Nothing is saved between visits.

Import conversation JSON

Paste an array of { "role": "user" or "assistant", "content": "text" }. An optional first system message is supported. Import replaces the editor. Limit: 100 messages / 100,000 characters.

Choose your budget

Small demo budget to make trimming visible; not a model's advertised limit. Allowances approximate role markers, chat formatting and other prompt overhead. Defaults (8/message + 16 fixed) are adjustable placeholders, not calibrated Qwen template costs.

Estimates work immediately. Qwen measurement downloads ~19 MB of tokenizer files plus library code on first use.

“Measured” means exact Qwen content counts per message, plus your estimated formatting allowances. It is not an exact serialized chat prompt or a guarantee a provider will accept it. No messages are sent to a model or deleted from your editor.

Interactive lab

One message. Five tokenizer profiles.

  1. Choose your textTry हिन्दी or paste a message. A token is a piece of text a model counts.
  2. Set your budgetThe context window is the total space. Reserve part of it for the reply.
  3. Check the estimateMeasure Qwen to see an actual count and how much space your text uses.
01 / Input0 characters
Try:

Examples are illustrative translations, not evaluation data.

A shared hypothetical budget, not each model's advertised limit. Counts cover pasted text only; chat formatting and system instructions use extra space.

Script compositionUnicode code points
    02 / ComparisonEstimates
    shorter bar = fewer estimated tokens
    Exact Qwen verificationQwen 3.5 · pinned tokenizer revision

    First use downloads about 19 MB of tokenizer files plus library code. Your browser may cache them for later.

    Why it matters

    A token difference becomes a product decision.

    01

    History retained

    If a chat app trims history using an inflated estimate, it can drop earlier messages that would still fit. Check the estimate before deciding what to remove.

    02

    Overflow risk

    Underestimation can admit too much text and trigger truncation at the real tokenizer boundary.

    03

    Multilingual design

    Test messages in the languages your users write. Different token counts can leave different amounts of room for context.

    Token count is not model quality.

    ContextFair measures budget behavior—not intelligence, cultural understanding, or answer accuracy.
    Method, honestly

    What you are seeing

    The current build uses fitted script-aware profiles to estimate content tokens. Each Unicode character is assigned to a script class, multiplied by a model-family weight, summed, and rounded up.

    Estimate ≠ exact tokenization.Five profiles remain fast estimates. Exact Qwen verification is an explicit on-demand action using a pinned tokenizer revision with special tokens disabled.

    The estimator is adapted from MIT-licensed LocalMind work investigated in the independent Tokenizer Estimation Study. Exact verification uses the browser-supported Transformers.js tokenizer API. Product scenarios and interface code belong to ContextFair.

    What the count covers. Exact Qwen counts use the selected tokenizer with special tokens disabled. The token lab counts only its input text. The conversation planner also counts the system instruction and each message separately, then adds your adjustable formatting allowances. It does not tokenize a complete chat template or tool definitions and cannot guarantee provider acceptance. The five comparison rows always remain estimates.

    Retention policy. The planner removes the oldest complete user-led exchanges until a recent suffix fits, protecting the system instruction and newest exchange. If those alone exceed the budget, it shows a blocked plan and disables export. It does not summarize dropped text or rank it by importance.

    Privacy. ContextFair does not send your entered text to a server or save it between visits. The page loads fonts from Google; exact measurement downloads library code from jsDelivr and tokenizer files from Hugging Face. Those services receive normal network requests. Browser caching depends on your settings.

    Read the method, evidence, and limitations