Five-Gram Frequency Counter | Repeated Phrase Checker

I think most writers imagine that they have some kind of ear for when something is repeated. You read what you’ve written; your brain goes through the rhythm; if you like it, you guess that it’s original phrasing. That’s why there are tools called “five-gram frequency counters.” They wipe out your sense that you just wrote this thing. Now you’re looking at the exact spots where your writing repeats itself.

How does it do this? Well, it cuts your text up into successive five-word phrases. And then it counts each unique one. So yeah: you think that repeating a phrase once isn’t a problem. The math shows you a story about density.

How Five-Gram Tools Help You Edit

Reading the output is only half of the story. Knowing the inputs is equally important. For example, do you want to lowercase everything? That’s fine if you’re analyzing word choice, but not so good if you care about title capitalization style. Do you want to strip punctuation? That’ll help find those echoes between sentences, but it can also lump different thoughts together under an artificial phrase.

What exactly do you want to be looking at? Are you searching for intentional refrains, or lazy filler words? The calculator categorizes these patterns according to their density vs. Count: Is your repetition a dense loop dominating the page, or a scattered echo across it all?

The tool comes with a set of reference tables that give you some sense of where “normal” falls, which is helpful. Obviously shorter texts (such as query letters or book blurbs) are going to have lower numbers of unique combinations. It’s harder to see repeated text in a two-hundred-word blurb than it is in a fifty-thousand-word novel. From this, we can tell that six-or-more should of been considered potentially formulaic (unless you intentionally wrote it that way). For most non-rhyming prose, that number is shockingly low. You might think dozens of repeats would be an issue before you worry. However, five-gram analysis shows that as few as three can slow down reading if they all group together in one chapter.

Stopword removal adds further complication. By removing common words such as “is,” “the” and “and,” you create a situation in which the counter examine only those parts of the sentence that is full of meaning. What this does is make your sentences appear to be far more repetitive than they actualy are. The skeleton of what you have written is exposed. This can help check whether there’s a consistent theme running through your work. But it’s misleading when you’re evaluating flow. Keeping all the words allows for a more realistic sense of what a reader experiences. They do indeed hear the connectors as well. To remove them artificially both deflates word count, and inflates the apparent density of your key phrases.

There are also boundary settings for the start and end point of the five-word window. If you want to allow phrases to span boundaries between paragraphs and lines, you may well find that they pick up echoes that persist despite scene-breaks. If instead you want to make your analysis tighter by restarting with each sentence, you could potentially overlook a rhythm which recurs over larger stretches. By being able to toggle the boundary setting, you have the opportunity to check various forms of rhythm.

If your narrative contains short, punchy sentences, you may fail to spot them with a paragraph-wide scan. Similarly, long, winding prose may hide its loops when viewed as sentence fragments alone. Altering the boundary mode alters focus. It’s the same subject, but what shifts is the detail you’re looking at.

What you’re really seeing is what you hadn’t seen: patterns. Writers have particular structural tics they don’t realize they repeat. You might begin every description with “The old house sat…” or you might finish all your dialogue tags with some variation on “he said quietly.” These become invisible while you write because they seem like part of the character’s natural voice. But once you see them listed in rank order, you’ll recognize them as mechanical crutches.

These tics will typically be at the top of your report. And yes, sometimes repetition is fine, it creates atmosphere; it reinforces a motif. It becomes an issue when you use this tool simply because you couldn’t think of anything better. This isn’t about eliminating all repetition; this is about making sure each phrase deserves its position on the page. Keep the five-word phrase if there is no other way to show the emotion. Get rid of it if you ran out of things to say during that section. There’s a fine line here, but an important one.

It makes editing less like a squishy “I don’t know what feels wrong” feeling and more like a surgical operation with clear areas of weakness. One idea is to listen to yourself repeating things; the alternative is to trust your ear and then back it up with data. You’re trained to skip over what sounds familiar. You do this to fill in the blanks. A frequency report closes these blind spots for you.

Where the rhythm soars. Where it stumbles. After you’ve removed the glaring loops, your prose will sound lighter and more intentional. The echoes retreat. Only the voice you wanted to hear remains.