LLM Token Counter Online

Count tokens with the exact OpenAI encodings (o200k_base, cl100k_base), estimate other models, cut text to N tokens and split it into RAG chunks.

Result

Press “Run” to see the result.

Counts tokens with the real OpenAI tokenizers: o200k_base (GPT-4o, GPT-4.1, GPT-5, o1 / o3 / o4) and cl100k_base (GPT-4, GPT-3.5, text-embedding-3), using the tiktoken rank tables through gpt-tokenizer. Pass model (for example gpt-4o) to pick the encoding automatically. The count also returns characters, words, tokens per word, an optional cost (price_per_million) and a limit check (max_tokens). For Claude, Gemini, Llama, Mistral, Qwen and DeepSeek, whose tokenizers are not public or not bundled, the response gives clearly labelled estimates with a range, never presented as exact.

action=truncate cuts the text to at most max_tokens tokens (from the start, or keep the end with keep=end) without breaking a character. action=chunk splits long text for retrieval (RAG): it prefers Markdown headings, then paragraphs, lines, sentences (Vietnamese sentence rules such as "TP." or "ThS." are handled), then words, packs pieces up to max_tokens and adds an overlap made of whole sentences. Each chunk has exact start / end character offsets in your text. action=encode / decode converts between text and token ids.

For Vietnamese text the response shows how much more it costs than English (vietnamese_vs_english, measured on the same paragraph in both languages). Special tokens such as <|endoftext|> are counted as plain text unless special=true. Limits: 1 million characters (500,000 for chunking). The text is processed in memory and not stored.

Use this in your code

curl -X POST "https://apibox.io.vn/v1/llm-tokens" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"text":"Xin chào! Hôm nay trời đẹp, chúng ta đi dạo nhé."}'

const res = await fetch('https://apibox.io.vn/v1/llm-tokens', {
  method: 'POST',
  headers: {
    Authorization: 'Bearer YOUR_API_KEY',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    "text": "Xin chào! Hôm nay trời đẹp, chúng ta đi dạo nhé."
  }),
});
const data = await res.json();
console.log(data);

import requests

res = requests.post(
    "https://apibox.io.vn/v1/llm-tokens",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={"text": "Xin chào! Hôm nay trời đẹp, chúng ta đi dạo nhé."},
    timeout=30,
)
print(res.json())

<?php
$ch = curl_init('https://apibox.io.vn/v1/llm-tokens');
curl_setopt_array($ch, [
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_CUSTOMREQUEST => 'POST',
    CURLOPT_HTTPHEADER => ['Authorization: Bearer YOUR_API_KEY', 'Content-Type: application/json'],
    CURLOPT_POSTFIELDS => '{"text":"Xin chào! Hôm nay trời đẹp, chúng ta đi dạo nhé."}',
]);
$response = curl_exec($ch);
print_r(json_decode($response, true));

API documentation → Get a free API key

FAQ

Are the counts for Claude and Gemini exact?

No. Their tokenizers are not public, so those numbers are estimates derived from an exact OpenAI count and a factor per script, with a range. Use the vendors’ own count-tokens endpoints when you need exact numbers.

Why does Vietnamese use more tokens than English?

Tokenizers are trained mostly on English, so Vietnamese syllables with diacritics are split into more pieces. With o200k_base the gap is small (about 1.15× on our reference paragraph); older cl100k_base needs far more tokens.

Do chunks cut sentences in half?

Only when a single sentence is longer than max_tokens. Otherwise chunks end at a heading, paragraph, line or sentence boundary, and offsets let you map each chunk back to the source.