Chatbots, now omnipresent, face a crisis of accuracy and security highlighted by recent public blunders at Air Canada and Chevrolet, where bots made unintended promises. Air Canada's attempt to deflect blame onto its bot was rejected by authorities, underscoring a harsh reality: companies are indeed responsible for their bots' actions. Despite the prowess of language models like ChatGPT, their inherent nature to occasionally fabricate with confidence poses unique challenges. Drawing lessons from cybersecurity, this article explores four advanced red team testing strategies aimed at reining in bot misstatements and significantly bolstering chatbot security.