2
$\begingroup$

Number of lemmas can be used as a rough measure for the number of words in a language. A lemma can have multiple word-form types. It can be understood from the following paragraph taken from p12 of Regular Expressions,Text Normalization, Edit Distance

Another measure of the number of words in the language is the number of lemmas instead of wordform types. Dictionaries can help in giving lemma counts; dictionary entries or boldface forms are a very rough upper bound on the number of lemmas (since some lemmas have multiple boldface forms). The 1989 edition of the Oxford English Dictionary had 615,000 entries.

It is also given that a lemma can have multiple boldface forms, what are the boldface forms referred here? Are they different from wordforms?

If possible, provide an example for lemma having multiple boldface forms.

$\endgroup$
1
$\begingroup$

It is very confusingly worded, and I would think it's incorrect according to linguistic terminology.

A lemma is the canonical form of a word, commonly the infinitive of a verb, the nominative singular of a noun, and the positive of an adjective. The inflected forms belonging to a word would the the forms used for other tenses and persons etc for verbs, case and number for nouns, and comparative/superlative for adjectives.

This raises the question of what a word is, and there is no satisfactory answer to this, even more than 100 years after the foundation of modern linguistics...

Anyway, the 'boldface form' (a term I have not come across in 30 years as a linguist), refers to dictionary headwords, which are lemmas. There are some lemmas that are 'shared' by words which have multiple meanings: the common example in linguistics is bank, which can be a financial institution, the side of a river, a term to describe the process of tilting the wings of an airplane in flight, or it can mean to deposit an amount of money in an account, etc. All these words you would find under bank in a dictionary, but usually under several different entries. So I guess this is what is meant by "multiple boldface forms". However, these are usually completely unrelated words which by accident share the same spelling; in some cases it could also have been the same word that then developed different meanings.

To summarise: the paragraph you quote is plain wrong/sloppy in its use of terminology, as a dictionary headword is a lemma in every dictionary I have seen, but these are not unique, as several different words might have lemmas which are spelled the same way (but they are still different lemmas — no single word would have multiple dictionary entries).

For example:

  • bank (bank, banks), noun: a financial institution
  • bank (bank, banks), noun: the side of a river
  • bank (bank, banks, banked, banking), verb: tipping the wings of an airplane
  • bank (bank, banks, banked, banking), verb: depositing money in an account

We have four lemmas (in bold), two of which have two inflected forms, and the other two have three each. These are also four different words, with a total of four different word forms (bank and banks are common forms of all words)

Often, to avoid confusion, you would refer to them as $bank_1$ for the financial institution, and $bank_2$ for the river bank, etc. to indicate that they are different words.

You can probably see that English has a number of lemmata (which is the proper plural of lemma, since it's of Greek origin) which is by a factor of 3-4 smaller than the number of word types, whereas in other languages this ratio will be a lot smaller, as they have more inflected variants. An English noun has just singular and plural forms, whereas a German noun would have singular and plural across each of four cases (though some of them would share the same word forms).

$\endgroup$
2
  • $\begingroup$ are they different forms though? they seem like the same form repeated 4 times. $\endgroup$
    – user253751
    Sep 3 at 14:13
  • $\begingroup$ @user253751 They look the same, but are different words. Their meaning is unrelated, and they have different origins. The financial bank comes from Italian banca (table), the river bank from Old Danish banke or Old Norse bakki. It is just chance that we spell them all bank. $\endgroup$ Sep 5 at 13:57
0
$\begingroup$

Here are some examples: reducing or reduces or reduced or reduction -> reduce; am or are or is -> be; n't -> not and 've -> have. When using spacy, the token can be referenced to find the lemmatized root.

lemmas =[token.lemma_ for token in doc]
lemmas =[lemma for lemma in lemmas
       if lemma.isalpha() or lemma == '-PRON-'
       ]


I use lemma to find parts of speech

Wiki

In computational linguistics, lemmatisation is the algorithmic process of determining the lemma of a word based on its intended meaning. Unlike stemming, lemmatisation depends on correctly identifying the intended part of speech and meaning of a word in a sentence, as well as within the larger context surrounding that sentence, such as neighboring sentences or even an entire document

$\endgroup$
4
  • $\begingroup$ That's not quite correct: reduction is a base form in itself, not an inflected form of reduce. You're probably thinking of stemming, which is similar to lemmatisation, but also includes derivations rather than just inflections. $\endgroup$ Jun 28 at 15:25
  • $\begingroup$ Lemmatisation is also not grouping words together. It is a mapping from a word form to a canonical form, which in English can be achieved by removing inflectional morphemes (such as a plural marker -s for a noun). $\endgroup$ Jun 28 at 16:17
  • $\begingroup$ explain what a canonical form is $\endgroup$ Jun 28 at 17:53
  • $\begingroup$ It's the base form. Nominative singular for nouns, infinitive for verbs, positive for adjectives. $\endgroup$ Jun 28 at 18:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.