An AI model can mistake instructions for content. When a developer uses special formatting for rules, the AI model might include that formatting and the rules in its output. The AI model does not always know the difference between an instruction and text for the user.
By Pawan Nayar · from the LLOS AI Learning record
An AI model can mistake instructions for content.
When a developer uses special formatting for rules, the AI model might include that formatting and the rules in its output. The AI model does not always know the difference between an instruction and text for the user.
The AI model saw special formatting in the instructions. This formatting made the instructions look like important text. The AI model then included these instructions in its response to the user. The AI model did not understand that these were commands for itself, not part of the answer.
A developer tells an AI model to follow 'GOLDEN RULES' written in bold. The developer uses colons to separate parts of the rules. The AI model then outputs the phrase 'GOLDEN RULES' in bold, followed by the developer's instructions, including the colons. The AI model thought the rules were part of the content to share.
Resolved from this lesson's own record — real definitions, not invented for this page.
More from the same record, pulled by shared topic — not written new for this page.
The AI model needs to treat each instruction as a separate, important guide. If a user gives a phrase, the AI model should use that phrase directly.
An AI model follows negative instructions most of the time. Use phrases like 'ABSOLUTELY DO NOT USE' for these instructions.
An AI model needs clear and complete instructions. The user must state the desired outcome precisely.
A developer includes parameters in a prompt. These parameters help the AI model understand the task.
An AI model can mistake instructions for content. When a developer uses special formatting for rules, the AI model might include that formatting and the rules in its output. The AI model does not always know the difference between an instruction and text for the user.