Inspectors

LLM Token Counter

Paste text and see how many tokens it contains for LLM models like GPT-4, GPT-3.5, and Claude. The tool uses tiktoken to count tokens accurately — the same tokenizer used by OpenAI models. Shows a per-token breakdown so you can see exactly how the text is split. Essential for managing context window limits and estimating API costs.

LLM Token Counter — Count GPT tokens for prompts, with per-token breakdown

What it does

  • ▸Counts tokens using the tiktoken tokenizer (GPT-4/GPT-3.5 cl100k_base encoding)
  • ▸Shows a per-token breakdown with token IDs
  • ▸Displays the token count alongside character and word counts for comparison
  • ▸Handles code, JSON, markdown, and multi-language text
  • ▸All processing is local — your text never leaves your machine

Use cases

  • ▸Checking if a prompt fits within a model's context window
  • ▸Estimating API costs for a batch of prompts
  • ▸Understanding how a specific text is tokenized for debugging
  • ▸Optimizing prompt length to reduce token usage

How it works

1

Paste your text into the input pane

2

The tool runs the tiktoken tokenizer locally

3

The total token count is displayed with a per-token breakdown

4

Review how your text is split to understand tokenization

Frequently asked questions

1Which tokenizer does it use?

The tool uses tiktoken with the cl100k_base encoding, which is the tokenizer used by GPT-4, GPT-4 Turbo, and GPT-3.5 Turbo. For GPT-4o models, the o200k_base encoding is used. The token count may differ slightly for other models like Claude or Llama.

2Why does my code use more tokens than expected?

Code typically uses more tokens than natural language because syntax characters (brackets, semicolons, operators) each count as separate tokens. A JSON object with many keys will use significantly more tokens than the same data in prose.

3How accurate is the token count?

The count is exact for the cl100k_base encoding. If you're using a different model (Claude, Llama, Gemini), the tokenizer may produce a slightly different count. The count is typically within 5% across modern tokenizers for English text.

Related tools

String Inspector — Inspect length, words, lines, bytes and unicode code points

String Inspector

Inspect length, words, lines, bytes and unicode code points

JSON Formatter & Validator — Format, validate, repair, and minify JSON

JSON Formatter & Validator

Format, validate, repair, and minify JSON

Mock Data Faker — Generate realistic mock data from JSON templates with {{faker}} placeholders

Mock Data Faker

Generate realistic mock data from JSON templates with {{faker}} placeholders

Lorem Ipsum Generator — Generate placeholder text: paragraphs, sentences, words, names, emails, URLs, tweets

Lorem Ipsum Generator

Generate placeholder text: paragraphs, sentences, words, names, emails, URLs, tweets

Use LLM Token Counter offline in Devinity

All 95 tools run locally on your Mac. No internet required, no data sent to servers.

Download Devinity →