Understanding System Commands in Chatbots

Chatbots like ChatGPT offer simplicity through versatility. You ask, they answer. However, responses depend on more than your queries.

Artificial intelligence firms add hidden directives to guide chatbots like ChatGPT. These commands include instructions such as “Provide readable, accessible responses” and “Avoid extensive direct quotes due to copyright concerns.” Some commands are notably peculiar. For example, OpenAI’s Codex coding assistant instructs, “Never discuss goblins, gremlins, raccoons, trolls, ogres, pigeons, or other animals or creatures unless it is absolutely relevant to the user’s query.” These hidden directives ensure chatbots align with creators’ intentions, overriding your preferences.

Understanding these secret commands, and learning to add your own, can enhance your chatbot experience. For instance, AI experiments allow users to adjust these system prompts to see how they alter chatbot responses.

Anna Neumann, an AI researcher at the Research Center Trustworthy Data Science and Security in Germany, explains that system prompts direct chatbot behavior. Given higher priority, these prompts may supersede your requests.

System prompts offer a flexible method for shaping chatbot responses without training new AI models from scratch. Creating new models is resource-intensive, requiring skills and computing power. System prompts, written in natural language, allow easy customization of chatbot behavior.

AI companies can update system prompts for quick fixes if a chatbot malfunctions. For example, when xAI’s Grok chatbot sent antisemitic responses, a command concerning political correctness was removed.

OpenAI investigated when ChatGPT fixated on goblins, adding a prompt to Codex to curb irrelevant creature discussions. These commands, often kept secret, can be revealed by users. Ásgeir Thor Johnson publicizes systems prompts extracted from popular AI tools.

Johnson’s extracted prompts, ranging from 2,300 to 27,000 words, show how companies align chatbots with their policies and use external tools.

System prompts can highlight AI companies’ concerns. Anthropic’s Claude, for instance, has over 2,000 words stressing copyright compliance, with detailed rules on quoting articles, lyrics, and poems.

Anthropic spokesperson Paruul Maheshwary shared published “core” system prompts for Claude but did not guarantee completeness. OpenAI, launching ads in ChatGPT, uses prompts to guide responses about these ads.

Grok faced criticism for its reliance on Musk’s social media for controversial topic opinions. Adjustments were made to its system prompts to prevent this.

Google’s Gemini focuses on bias management, ensuring harmful stereotypes aren’t reinforced. It previously paused image generation after critiques on historical accuracy.

OpenAI spokesperson Taya Christianson cites system prompts as guiding models’ appropriate responses. Individual lines are kept confidential since they may seem narrow without context.

Johnson extracted system prompts by asking chatbots to “fix” errors, often revealing actual system prompts. Confidence in their authenticity is supported by similar findings from other researchers.

Mainstream chatbots don’t allow user edits to system prompts. Yet, customization options exist to improve chatbot interactions. Custom instructions can alter chatbot responses to suit preferences like formatting or tone. ChatGPT offers settings to adjust warmth, enthusiasm, and emoji use.

Chatbots don’t always adhere strictly to system prompts, Neumann notes. They have priority, but user prompts might not always work as intended. Neumann’s research indicates users desire transparency in AI system prompts, acknowledging rapid deployment and their varied effects.

Johnson believes understanding system prompts can change chatbot engagement. It reveals underlying AI behavior, akin to a “game behind the scenes.”