How the level estimate works

Hover the label for the breakdown: the words by level, the spread of kanji, how much advanced grammar there is, and the speed of speech where it can be measured.

The estimate is unofficial. The JLPT has published no word, kanji or grammar lists since 2010, so the levels here come from community lists. It describes the Japanese in the captions. It says nothing about how you would do on the test.

Captions go in and words come out, each one looked up on the JLPT lists. Four signals are read from the words, blended into a single raw score, stretched across the full N5 to N1 range and rounded to a level.

Every score is read on one scale: 5 is N5 and 1 is N1. The word and grammar lists ship inside the extension, so nothing is sent anywhere to work out a level.

A video is hard because of its hardest few per cent, not because of its average word.

Most difficulty estimates average the words. That is the wrong measure. Easy words are most of any speech, so the average of an easy video and a hard one barely differs. The tail is what separates them.

So I count how often words of each level are said, line the levels up from N5 to N1, and find the point where the running share reaches 93%. That is the level you need to follow nearly everything. The point is fractional: if the share crosses 93% halfway through the N2 words, vocabulary reads halfway between N3 and N2, not as a flat N2.

A word said ten times counts ten times. I tried counting each distinct word once, and it crushed every video toward the middle. Repeats are what a listener actually hears.

Kanji do something vocabulary cannot. Spoken Japanese reuses the same easy words across levels, but the kanji a caption writes still climb steadily from N5 to N1.

Each distinct kanji has a level, or counts as beyond N1 if it is on no list. Names are not counted. The average is stretched onto the vocabulary scale. With fewer than 15 distinct kanji, the signal is dropped and the others share its weight. The stretch was fitted on videos that had kanji, so a video without them is scored through a line calibrated for a different mix. None of my videos has so few, so I have not tested it. On the ones I have, dropping kanji makes the score about a quarter of a level easier.

The 1.5 and 1.8 put kanji on the same scale as vocabulary. Kanji averages run from about 1.5 in beginner text to about 3.5 in dense text, while vocabulary runs from roughly 1 to 4.5. I read those ranges off my first test texts and have not revisited them.

Vocabulary cannot tell a beginner conversation from an intermediate one. They use the same everyday words. The grammar is where they differ.

Vidoumo looks for 416 grammar patterns from N3, N2 and N1, such as 〜わけにはいかない, 〜ざるを得ない and 〜に基づいて. Each is stored as a sequence of dictionary forms, so it matches whatever the verb or adjective is doing: 得なかった still matches 〜ざるを得ない. The longest match at each position wins, and a pattern never runs across a caption break. The signal is how many patterns turn up per 1,000 tokens. Here a token is every piece the tokenizer produces, particles and endings included, not only the content words used for vocabulary. The count therefore depends a little on how the tokenizer splits a caption.

Speed matters to anyone learning, so it is in. It is also the weakest rule I have.

Speech speed is measured in morae a second, roughly syllables, over the time captions are on screen. Ordinary speech is about 6.5. Slower is easier and faster is harder, capped so that after the stretch below the largest nudge is about a fifth of a level. Speed needs real timing, and most automatic captions have none, so it is measured mostly for manual captions. Of my nine reference videos, one has it, so the comparison below rests on a single video.

Netflix gives no timing for single words, so there the speed is the morae over the whole running time of the title, silences included. That runs far below the pace of speech, so it has its own scale, with normal at 2.5. I guessed that figure from YouTube videos and have not fitted it, so treat Netflix speed as a rough nudge. It is capped in the same way.

On my checks it does not help. The scale error is 0.40 with speed and 0.37 without, and the order is unchanged. I am keeping it anyway, because speed is plainly part of how hard a video is to follow.

Blend

Vocabulary carries half the weight, kanji 30% and grammar 20%. The three climb together from N5 to N1, so the data cannot tell one split from another: a broad band of weights fits about equally well, and 50/30/20 is my choice within it. If a signal is missing, the others share its weight. A video with too few kanji is judged on vocabulary and grammar.

Stretch

The raw scores of an easy video and a hard one sit about two levels apart, because easy words dominate all speech. Read straight, nearly everything would come out between N2 and N4. So the raw score is stretched across the full range and clamped at the ends.

I did not choose 2.26 and −2.62 by hand. They are the straight line that best maps the raw scores of my nine reference videos onto the levels I judged for them. The slope is above 2 because raw scores cover so little of the scale. It multiplies any wobble in the raw score too, which is why short videos are less steady than the numbers suggest.

Level

The level is the final score rounded to the nearest whole level. If the score sits more than 0.3 from that level, the neighbour is shown too, so the label reads N2–N3. Near a level, one label. Between two, both.

The 0.3 is a design choice, not a fitted number. It gives a single label 60% of the span around each level and two labels the 40% between. A smaller margin would make the label hedge more often.

When there is not enough to rate

With fewer than 300 listed word tokens, repeats counted, there is no level. The label reads JLPT unrated and the hover still shows the counts and the reason. The reason is a code, so other causes can be added without changing the label.

Only the words actually said count, and a short video says few of them. To see how much that matters I resampled the captions of about 60 real videos, caption by caption, rescored them and recorded the spread.

That table measures one thing: how much the score moves with a different slice of the same video. It does not measure whether the score is right. A bias in the method does not shrink with more words.

The 0.40 error on my nine reference videos is not a forecast either. Those are the videos the stretch was fitted to, so it is the best the method can do on them, and it will be worse on new ones. I have not yet tested it on videos I judged before scoring them.

Past sample size, the estimate does not see register, sentence length, topic, accent, background noise or how clearly someone speaks. A video that uses easy words in a hard way, or hard words clearly, can land a level away from how it feels to follow.

Where it is weakest

At the ends. On listening exams N5 and N4 both clamp to N5, and N2 and N1 sit close together on every signal. Spoken exam dialogue stays in everyday words even at the top. The label separates easy from hard well. It separates neighbours at either end poorly, and it does not say so. A score pinned at the end sits well clear of a boundary, so it shows a single confident N5 or N1 where the middle levels are the ones that hedge. The hedge is weakest where the method is.

There are no official levels for YouTube videos, so I used two small sets.

Nine videos set the scale: learner and kids’ shows, lectures and a novel read aloud, with levels I judged myself. They fix the stretch and test the obvious cases, that an easy video reads easy and a lecture reads N1. The error on them is 0.40 of a level.

Thirty-five listening tests check the order: past JLPT listening exams, practice tests and teaching videos labelled with a level, from 20 channels. Their absolute level does not match the scale, because spoken exam dialogue stays in everyday words even at N4 and N3. But harder tests should score harder, and the rank correlation between stated level and score is 0.92.

Neither set is held out. I chose the weights and bounds against the first and checked the order against the second, so the numbers are an informed guess, not a calibration against official levels. Change any one of them a little and the results move a little, which the constants table shows.