Some of the words chosen are rather absurd/inappropriate: breviary (which I got wrong but felt like a vaguely religious word) was characterized as intermediate but I think it's much more obscure and less obvious than that; Hippopotomonstrosesquippedaliophobia was used as a word (I got that wrong as well) - any type of 'phobia' word is really the sort of thing a fourth grader opens up a page in the dictionary and points out, not a word that is used... ever; metamorphosis and kinetic were labeled expert, which I don't agree with (what elementary schooler doesn't learn about the metamorphosis of a caterpillar into a butterfly? what high schooler doesn't learn about kinetic energy?).

Most words were reasonably well defined in a way that most people would understand or recognize. A few words had poor definitions: lethargy ("the state of being lethargic" - obvious); complacent ("smug satisfaction with oneself" - I disagree that complacency is intrinsically smug); magnanimous ("generous toward a rival" - I disagree that a rival must be involved); gauche ("socially awkward" - this is sort of close but the given definition completely misses the idea of being tactless).

They call it scientific and give a hand-wavey formula, but they don't explain how words are stratified in the first place. If stratified sampling is a formally recognized method of doing this, it would be nice to have a link to a real reference. I think I know a lot of words, but I am skeptical of the estimate this app provided (north of 75k).

I've seen other systems like this calibrate far more quickly by assigning a sort of score and confidence behind the scenes. Confidence starts out low and increases over time - correct/incorrect answers rapidly adjust score at the beginning, then things settle down.

In practice this means you get a sequence of increasingly uncommon words initially, until you get one wrong, then you drop back to something easier until you start getting things right again, and eventually circle around words at your level.

Also - too many clicks per word. It's low stakes, just let me click the definition once and I'll live if I misclick (or add an undo button).

(context: native English speaker, big reader, huge nerd, perfect SAT score)

I got all 100 correct on the first try without looking anything up! Confusingly, that only resulted in a "SCIENTIFIC ESTIMATE" that I know 85,000/~170,000 words?

Their "How is this calculated" page that appears at the end explains their error:

According to the Oxford English Dictionary (Second Edition), there are approximately 171,476 words in current use.

We use Stratified Sampling. Instead of testing random words, we divide the language into 5 distinct difficulty bands based on frequency of use:

1. Core Basics ~3,000 words > 2. Intermediate ~7,000 words > 3. Advanced ~10,000 words > 4. Expert ~25,000 words > 5. The Obscure ~40,000+ words

If you answer 2 out of 3 'Intermediate' questions correctly, we estimate you know roughly 66% of the 7,000 words in that band.

Total Score = Σ (Accuracy in Band × Band Size)

Their strata add up to 85000, not ~170k, making a perfect score still give a 50%.

They're also using a pretty limited and perhaps non-difficulty-representative subset of the language.

Cute, but wrong on many counts.

Most of the corpuses I've found heavily over-represent newspaper articles and books, obviously. So the frequency ranking is biased towards academic/crime/geopolitics, not spoken english. But even then, it depends what you most commonly speak about!

There's no better way to do it, though. I'm just providing context.

I got credit for a few that I would have happily just missed.

I suggest skipping the submit button and just showing it's correct when pressing and moving on after a sec or so. Having to click on submit twice really breaks the flow.

Also in all the words I tried I noticed out of the 4 options one is the correct one, another is the opposite of the correct one, and the other 2 are random stuff. You can basically skip any option whose antonym isn't present as well.

The sample of words is also heavily biased towards concepts relating to words, speech, speakers, and/or persuation. They are likely generated by an LLM which is primed on the task of choosing words, and end up choosing words related to "words".

For context, I'm an L2 speaker, linguistic nerd, and I use English mostly in academic/professional settings. I got 75,400 by a combination of the tactics above; in reality it might be closer to 10-15k.

The design is also painfully similar to Duolingo if anyone can spot that.

Good news for the project is that I think you can easily tweak the LLM to generate better alternatives.

I got 89/100, which extrapolates to 72,700. As a non-native speaker, I'm quite happy with that.

Also many highest difficulty words are actually combinations of multiple smaller words which makes it easier to guess, I got more right in expert/grandmaster than in advanced.

There were many words I couldn't have explain the meaning of at all, if I wouldn't have had the options, but having the options made it easy. I wouldn't count those correct answers as a part of my vocabulary (even passive), even if I could answer with relative confidence.

My shorter OED contains 163,000 words (compared to the 600,000 words of the longer).

According to this site I know 71,000 words... Let's test that against the OED. I should have about 43% chance if knowing a word picked at random.

In my totally scientific test (ha) I chose 50 words at random from the OED and discovered I knew 29 of them for a score of 58% which is more than two sigma from 43%, this disproving the hypothesis.

I forgot what that was now, but it was a fun experiment.

Interesting how literally everyone here's performing better than I do. Perhaps that's because I just clicked on the first option whenever I don't know about a word.

Core Basics 19/20

Intermediate 17/20

Advanced 19/20

Expert 14/20

Grandmaster 12/20

I guess, it's not too bad for a non-native speaker.

Minor feedback:

1. The correct answer for "Lethargic" is "Affected by lethargy". I think, definitions should not use words that share common root with the defined word, because:

a. it makes guessing too easy

b. it basically becomes a circular definition which is meaningless

2. Options almost always include 1 correct answer, 1 direct opposite and 2 completely random. Once you learn to recognise it, you can easily rule out 2 random options and have a 50/50 guess.

Like if author used LLM to generate wrong definitions per word instead of actually mixing definitions of words.

Like for me most of more complex words been adjectives with few nouns. And in many cases you can just see 2/4 or 3/4 definitions are not for adjective.

At least I learned a bunch of «faux-amis» in the process.

At least I can step away from the laptop now I've got RSI.

If you force me to guess, then I'm going to guess. Not only does that give me a 25% chance of getting it right at random, but as others have pointed out, it is very hard to make a multiple choice question that isn't guessable by an astute enough test taker. I think I knew 80 - 85 of those words, but I scored 97, because those questions were very guessable.

Also, reiterating everyone else's comments with respect to the UX needing fewer clicks, and also the definitions not being exact or precise in many cases.

But then below it said "you are a man of few words".

I take it the latter is just because I've only done the test once? But it's mixed messaging on first attempt I think.

So it's not uncommon to see a native English speaker totaling 90 as 20,20,19,17,14, and a foreigner reaching the same total as 18,18,18,18,18. Strangely enough, the algorithm favors the latter, because it assigns more weight to the higher-end bands.

Is this of any use? I doubt so, but it was fun.

P.S. of course a more reliable clue of nativeness is the use of "its" and "it's" interchangeably, a mistake EFL learners wouldn't do.

A word's "difficulty" would be some function of how rare it is. Once you have a reasonable estimate of the user's "skill" you can infer that a user won't know more difficult words. The benefit of this is you're not spending time asking the user about words they probably know.

Of course it's possible at an individual level, difficulty does not monotonically increase as a function of how rare the word is. A person might be very familiar with a domain-specific subset of English. But the "stratified sampling" approach will also have this problem.

There is a similar problem in chess, where players have ratings which really only change on one dimension. So there can theoretically be a mismatch when puzzles are also scored on a single axis, since a "harder" puzzle that contains a motif a player is familiar with will actually be easier for the player.

Aside from that, I didn't like that most of the words only had one or at most two definitions that sounded viable.

A lot of these words have either Latin or Greek origins, for most questions you can deduce the correct answer by asking the question: "Which of these would make sense to develop into a separate word through the mostly non-modern history of the language?".

I would enjoy it way more if all four options sounded equally viable, and I couldn't deduce the correct answer without actually being sure about the meaning of the word. I understand that coming up with choices like that for each question is way harder if you actually validate all of them manually.

I got a score of 76000 best estimate with 85 being correct, even though English is not my native language and I'm not that good at it.

Gemini 3 Flash AI enough to ingore the results

The very first one was "Unique". I wondered if "the only one of its kind" was still the correct answer, having seen "very unique" used all too often recently. They accept "only one of its kind".

Missed "hegemony" (wasn't sure a hegemony had a leader), "quotidian" (should have known that, seen it before), "ultracrepedarian" (new word to me), "absquatulate" (19th century slang), and "fartlek" (Swedish interval training).

Regardless, this was fun.

Required Reading

Read the dictionary from A to Z. It's a gripping tale with a terrible plot.

I actually have! I was very bored with the barely-above-"see spot run" books in the classroom at around 8, and we didn't yet have open access to the school library. The dictionary was a better option than all the others I had access to (in class).

Any other dictionary-completionists in here? Regardless of size - I'm fairly sure mine was rather small, though not a pocket-sized one.

At least that was my experience as a native Italian speaker. My English vocabulary is good, but not great by any means and by reading books in English I know that there are plenty of words that are not derived from Latin

This approach could also work for getting more accurate results:

1. Show word without any definitions

2. User clicks "I know" or "I don't know"

3. If user clicked "I know", show actual definition of word

4. User selects "I was correct" or "I was not correct"

(My first language is Russian.)

I would guess this causes an up shift in results even if not consciously noticed.

From what I can tell they actually have a bit more robust science behind their algorithm (and a lot less questions to answer)

TV vocabulary is targeted at 6th grade reading level.

Conversational English is about 2,000 words.

High school vocabulary is about 10,000 words.

College degree vocabulary is about 30,000 words

English has over a million words.

Which heartens me, because it means I can be "fluent" in another language by learning just 2,000 words.

It needs some kind of auto adjusting difficulty...

Same strategies apply for guessing the unknown especially with a modicum(it was on the test!) of Latin knowledge..

Strange that pretty every one here is getting 70k estimates (93/100 for me).

Feels a bit high at least for me as a non-native speaker.

I got 2 words I knew wrong, and guessed about 5 unknown words correctly. Those were bizarre repetitive words I've never seen before.

I remember doing a similar test from a reputable university about 10-15 years ago also in an app format and only got about 30k estimate.

As a non-native English speaker, I found that result pretty good! Though being a native Portuguese speaker certainly helped me as many difficult words in English borrow from Latin, and in Portuguese the Latin influence is more pronounced.

I'm curious how the difficult is chosen because "obfuscate" was included in the hardest difficulty but I would not consider that to me a difficult word.

Also I found that some of the definitions were not completely correct.

However I have some other ideas and my quiz isn't "science based"

- in my quiz there are only "yes / no answers" This way you don't spend eternity reading descriptions of the word "apple". It also means I can estimate separately my passive and active vocabulary.

The OP missing "I don't know button" which will overestimate any result by 25% percent.

- I'm adjusting dynamically how many questions to ask in each bucket.

the goal of my quiz is to estimate a number of German words an English speaking learner has learned.

So I have curated vocabulary to remove "free words" like rare compounds of common words and other rare words which satisfy "any European knows this word without learning".

The final vocabulary used in a quiz is approx 8k words only

https://wortschi.de/quiz

I thought it was going to be tougher because the very first word on my run was "Yield" and none of the options seemed convincing to me. I went with something that was at least fairly adjacent to the "something produced by" (as opposed to "submit to") meaning and this did successfully yield (he he) my first point.

Be fun to start at Master and up, but is kerfuffle really grandmaster?

Gaikwar and Kowtow are English words?

I'm not sure exactly how you did this, but I think you asked an LLM to come up with the wrong options. Two things to consider:

1. While the LLM can go r good options, they won't be always hard to guess. I wonder if instead you can have the LLM generate very close words (or skip using an LLM entirely) and put those as the options. 2. If you will generate options with an LLM, make sure you are mindful of its inability to shuffle things around. The correct answer was overwhelmingly the first or second option in the list. You should ask the model to give the options in a uniform order (say from true meaning then decreasing amount of replayability), then manually shuffle them so that the probability of which option (A, B, C or D) is always 25%.

A lot of words used in Software Engineering as metaphors helped.

Also one weird tip. If I didn't know the answer went for the negative description of human behaviour answer and I guess 50% chance rather than 1 in 4.

I do concur that a refined collection of incorrect proposed responses which includes selections among terms with semantic proximity, conflated synonyms and plausible morphology could refine the accuracy of evaluations; and if the test was intended to bestow authentic assessments of lexicographical capability this would in all probability become an efficacious approach, but as a simply presentable quiz for folks with sesquipedalian proclivities I was not unduly discomfited by anything moreso than the extraneous clicks leading to and following the display of dichotomous determinations.

1. Frame each option with one key (1,2,3,4). User press 2, select the second option

2. Let the user change options if they want until they press Enter. Enter submits the answer.

3. Once submitted, another Enter brings the next one

So it’s not a test of how many words you know but how good you are at guessing what words mean.

I wonder if the test is calibrated to the fact that some answers are just well guessed? I am not a native English speaker, but I speak 3 languages overall and have basic notions in Latin, and I have to admit it helped a lot in "deciphering" a few words that I didn't know at all. And in at least 2 cases I just guessed correctly.

It's annoying that you need to click 3 times per question, and the buttons are in 2 different places.

Maybe would be better to just let me click the answer I want and then instantly show me the next question?

Also who is Sandi?

Scientific Estimate: 69 100 word

It began very simple, so that I took it not very serious for a moment, but I never heard many of the later words. But thanks to knowing some latin and other languages, I could understand many of them.

A fun idea!

I scored 71,000.

But to be honest many that might catch out a native speaker are just the Spanish/French/Latin word, so it was too easy in a way.

It would have paired well with an exposition of vanilla Monte Carlo and the benefits of stratified sampling.

Although stratified sampling is good, one can do better in this case by using adaptive sampling, where one uses a runtime (Bayesian) estimate of vocabulary to maximize information gain per question -- preferrentially sample from those strata where the current strata specific estimate has higher variance.

I do wonder how much of these were “what AI thinks are hard words to know” vs. actually hard to know.

I used to do this in school tests too.

Got 64,650: 20/19/17/18/12 (the intermediate one was a dumb mistake)

This is true of any LLM-generated quiz.

Some definitions were not great and alternatives a little silly at times but on the whole seemed pretty accurate.

Also probably needs calibrated as 96/100 was projected to 77k words, what would the estimate be for 100/100?

You are a person of few words, or perhaps just a mysterious one. Quite intriguing.”

—- This sounds more like a cute assessment of only getting two words right. And what do you mean “new words”? It wasn’t until eighty-odd words in that I actually got a word I didn’t know and had to guess by ruling out multiple-choice options.

I got 83/100 suggesting 60,000.

My SAT reading was 760/800.

One suggestion would be more convincing decoy choices, some were pretty silly. But I have no idea how they come up with them.

Also, as others have said, mixing easy and difficult words would make the process less boring.

Might I suggest adaptive difficulty? After getting 10, 15, 20 correct in a row it should scale up the difficulty immediately, rather than waiting for 100 in the basic level 1...

I suppose the words must be weighed, because other people in the thread with more correct words got a not much higher estimate.

My score: 78,000 words, 20/20/19/18/18.

It told me to read the dictionary.

Also add a keyboard focus state on the continue button.

But many of the hard words were quite similar to more common words we have here.

Are accoutrement and ziggurat really English words? Accoutrement is even pronounced as French!

Each word is a double-click.

also, some of these words are actually not good ‘obscure vocabulary’ but trivia crap. overall a bit AI slop and too easy.

And it didn't even tell me at the end how many words I know!

There is a similar variant of such a test where you just go down a list of words of increasing obscurity, ticking the ones you are familiar with. If you do this once or twice, you can get a fairly good estimate of the actual number of words you know.

I’m not sure how you’d gauge what knowing each word would indicate.

Also adequate options, that sound plausible.

Test could be completed in 1/5 of the time if the user could use numeral keys [1, 2, 3, 4] plus "enter" to input selections instead of the cursor.

[0] https://slate.com/human-interest/2011/11/bumbershoot-it-mean... "the digital archive of the Times of London, comprising 7,696,959 articles published between 1785 and 1985, yields precisely zero hits for bumbershoot"

But no - other people pointed out the same things I noticed, such as many of the wrong answers being very weird.

This could have been a neat game, but it is ruined by being unrefined AI slop.

"Verbose," for instance, is defined as "Using more words than are needed."

That's not exactly wrong, but it's kind of misleading. "Verbose" explicitly means using a large pile of words, drowning the reader in far more words than are strictly necessary.

"More words than are needed" could be as limited as "used a three-word construction in a sentence where it could have been one."

There are many more like this.

Please, I beg all of you - don't use LLMs to generate linguistic slop that claims to be linguistic education.

I weep for the world that is to come.

Fun!

Probably not too bad for a person whose native language is not English.

https://smackernews.com/item/2806377 HN

Level 0: Core Basics Abundant, Baffle, Candid, Dwell, Emerge, Frugal, Generic, Hinder, Impartial, Jovial, Knack, Lucid, Meager, Naive, Obsolete, Peculiar, Quench, Refute, Seldom, Tedious, Unique, Valid, Wary, Yearn, Zeal, Adequate, Barren, Coarse, Diligent, Esteem, Fickle, Gloom, Hoax, Ignite, Jolt, Keen, Linger, Mend, Numb, Omit, Pledge, Quota, Rural, Soothe, Toxic, Urge, Vow, Witty, Yield.

Level 1: Intermediate Acumen, Benevolent, Complacent, Dilapidated, Eloquent, Fabricate, Gregarious, Hypothetical, Imminent, Juxtapose, Lethargic, Meticulous, Nostalgia, Oblivious, Pragmatic, Reiterate, Scrutinize, Tentative, Ubiquitous, Verbose, Wane, Aesthetic, Bolster, Candor, Defer, Elicit, Furtive, Glut, Heed, Impeccable, Lament, Modicum, Notorious, Opulent, Plausible, Resilient, Stagnant, Trivial, Viable, Zenith.

Level 2: Advanced Alleviate, Breviary, Cacophony, Deferential, Ephemeral, Fastidious, Garrulous, Harangue, Iconoclast, Juggernaut, Laconic, Magnanimous, Nefarious, Obsequious, Paradigm, Recalcitrant, Sanguine, Taciturn, Ubiquity, Vacillate, Winsome, Zephyr, Abase, Banal, Capricious, Debilitate, Ebullient, Facetious, Gaikwar, Hackneyed, Idiosyncrasy, Jargon, Kindle, Labyrinth, Maverick, Narcissism, Ostracize, Palliate, Quagmire, Rancorous, Sagacity, Tantamount.

Level 3: Expert Abstemious, Bellicose, Chicanery, Deleterious, Enervate, Fatuous, Gauche, Hegemony, Inculcate, Jejune, Kowtow, Lugubrious, Mawkish, Nonsectarian, Obdurate, Pernicious, Quotidian, Recapitulate, Supercilious, Tempestuous, Unctuous, Vehement, Winnow, Xenophobe, Ziggurat, Acquiesce, Bombastic, Circumlocution, Desultory, Equinox, Fiduciary, Gerrymandering, Hubris, Incognito, Kinetic, Loquacious, Metamorphosis, Nihilism, Orthography, Precipitous, Quasar, Reparation, Soliloquy.

Level 4: Grandmaster (The Obscure) Accoutrement, Brobdingnagian, Crepuscular, Defenestrate, Equanimity, Flibbertigibbet, Grandiloquent, Hippopotomonstrosesquippedaliophobia, Ineffable, Jingoism, Kerfuffle, Logorrhea, Mellifluous, Obfuscate, Panacea, Quixotic, Rococo, Sesquipedalian, Tergiversate, Ultracrepidarian, Vicissitude, Weltschmerz, Xeric, Yclept, Zeitgeist, Absquatulate, Bumbershoot, Callipygian, Dord, Ergophobia, Fartlek, Gobbledygook, Houghmagandy, Interrobang, Kakistocracy, Lollygag, Mumpsimus, Nudiustertian, Omphaloskepsis, Pogonotrophy, Quire, Ratoon, Snollygoster, Tittynope, Ucalegon, Vagitus, Widdershins, Xylopolist, Yarborough, Zenzizenzizenzic.

Improve the wrong answers to be closer to the correct answer, to test the subject’s mastery.

Anyone who has practiced standardized tests would do well on this, even with poor vocab.

Also, too many Britishisms

SCIENTIFIC ESTIMATE 74,000 words "Unbelievable. Are you actually Stephen Fry in disguise?"

You mastered 93 new words! THE VERDICT

You are a person of few words, or perhaps just a mysterious one. Quite intriguing. REQUIRED READING

Read the dictionary from A to Z. It's a gripping tale with a terrible plot.

* Correct word * Opposite definition * Another word's definition * Opposite of that word's definition

Which massively reduces the difficulty

You mastered 88 new words!

You know 60000 words, that’s not a lot, go back to reading the dictionary

Goes to the about section: an average native speaker knows 35000 words.

Ah yes, the classic British insult, should have known it.

Impartial does not mean "treating all parties equally". It means "uninterested in the results". Fair would be "treating all equally". That's why there's a phrase "fair and impartial". "Partial" of course, doesn't mean "unfair", so negating it can't turn it into "fair". Partial means to favor one side or the other.

This is why when people tell me I'm wrong, so often I feel smarter than they are. HN quizzes are conditioning me for some antisocial attitudes, I think.

I got 93 words (not a native speaker), but the expert/grandmaster words were kinda easy?

You mastered 100 new words!

No, I read about 97 words I already knew and guessed at a couple of made-up ones like "snollygoster".

Is this what passes for an advanced vocabulary in the US?

Also, it took far too many clicks per word, pretty tedious stuff.

The green button (which should not exist) was also hidden under Firefox for Android's address bar until I tried to "scroll* to hide it.

That was fun! tho a lot were cuz the longer the answer, the more likely it was to be right (for words I had utterly no clue)

Was really hard to stop once started lol

I mean, select the word, then press check, then press continue.

It could be one single click and move to the next, show me my last result at the same time you ask me for the next one.

Then I was doing poorly in grandmaster, until I realize you can ace grandmaster by just picking the longest explanation every time.

Vibe coders need to be forced to spend one day learning basic CSS before they're allowed to use an LLM to make a website and the internet would be a lot more pleasant as we move forward with slopification.. It doesn't have to be sloppy, and doesn't take all that much studying to at least be able to steer an llm in the right direction to make something look nice. At this point everything is just the same 3 colors and a centered flex column with weird spacing.

3 clicks per is what gives it away. and the little compliments. and that it's 100 questions

English is not my native language. I get my vocabulary from browsing the Internet. There is no way I know that many words.

... got 95. Can't believe there's a word for a neighbor whose house is on fire.

"Yield: Produce or provide a natural product"

Eh?

I use the language to understand not get an effect

But Candid can certainly mean secretive, as in “Candid camera”.