An AI model should use a fixed number of rare words in its output. The rarity of these words should vary across different levels. This approach adds depth to the AI model's vocabulary without increasing the total number of rare words.
By Pawan Nayar · from the LLOS AI Learning record
An AI model should use a fixed number of rare words in its output.
The rarity of these words should vary across different levels. This approach adds depth to the AI model's vocabulary without increasing the total number of rare words.
An AI model can use a set number of rare words in its output. For example, it can use six rare words. These words should come from different levels of rarity. This means some words are a little rare, some are more rare, and some are very rare. This method makes the AI model's language richer. It does not add more rare words overall.
A developer wants an AI model to create slogans. The developer decides each slogan will have six rare words. Instead of picking all six words from one rarity level, the developer tells the AI model to pick two words from a slightly rare group, two from a moderately rare group, and two from a very rare group. This makes the slogans more interesting and varied in their language.
More from the same record, pulled by shared topic — not written new for this page.
An AI model should use rare words from different difficulty levels. This means not just using words that are a little uncommon.
An AI model sometimes uses the same words too often. These words can be boring or try to sound wise without being truly wise.
Certain words are overused in general communication. When these words are used in prompts or code, an AI model tends to produce generic or unoriginal text.
An AI model learns from many texts. It might favor some words, called 'comfort words.' It might also use words that sound important but add little meaning, called 'fake-depth words.' A developer can identify these words.
An AI model should use a fixed number of rare words in its output. The rarity of these words should vary across different levels. This approach adds depth to the AI model's vocabulary without increasing the total number of rare words.