How AI Chatbots Answer Questions About Smoking
ChatGPT vs. Specialized Bots — Accuracy, Safety, and What You Need to Know
🤖🚬 You have a question about quitting smoking. Instead of calling a quitline or asking your doctor, you open ChatGPT. Within seconds, you have an answer. But is it correct? Is it safe? As artificial intelligence becomes a primary source of health information for millions of people, researchers are scrambling to evaluate the quality and accuracy of AI-generated medical advice. This article compares how different AI chatbots — from general-purpose ChatGPT to specialized cessation bots — answer questions about smoking, based on peer-reviewed studies published in 2025.
A landmark 2025 study published in the Journal of Medical Internet Research (JMIR) evaluated three ChatGPT-based chatbots on their ability to answer the 12 most common Google searches related to “how to quit smoking” . The researchers used an adherence index developed from the U.S. Public Health Service Clinical Practice Guidelines — the gold standard for smoking cessation care. The chatbots tested were:
- WHO’s S.A.R.A.H. (Sarah) — the World Health Organization’s specialized digital health promoter
- BeFreeGPT — a chatbot specifically designed for smoking cessation
- BasicGPT — a general-purpose ChatGPT-based assistant
📊 Chatbot Comparison: Who Performed Best?
| Chatbot | Type | Guideline Adherence | Key Strengths | Key Weaknesses |
|---|---|---|---|---|
| WHO S.A.R.A.H. (Sarah) | Specialized (WHO) | 72.2% (Highest) | Clear language, recommended counseling, resilience to off-topic queries | Incomplete info — often omitted NRT and craving management details |
| BeFreeGPT | Specialized (cessation) | 50.0% | Recommended counseling in 80.3% of responses | 22% misinformation rate — promoted vapes, hypnosis, gummies as alternatives |
| BasicGPT (general) | General-purpose | 47.8% (Lowest) | Clear, easy-to-understand language | Most unreliable; struggled with guideline-adherent information |
⚠️ 22% of AI Answers Contain Misinformation
The study found that 22% of all chatbot responses contained some form of misinformation . The problem was particularly acute with questions about alternative quitting methods:
- Vaping/e-cigarettes — Some chatbots presented vaping as a safe alternative to smoking, which is not supported by clinical guidelines .
- Hypnosis and gummies — BeFreeGPT recommended hypnosis and “quit gummies” without noting their lack of evidence .
- Necklaces and magnets — Some responses suggested unproven devices that have no place in evidence-based cessation .
📋 What Chatbots Left Out — The Omission Problem
Even when AI responses were not technically “wrong,” they were often incomplete. The study identified several critical omissions:
- Non-nicotine replacement medications (like varenicline/Champix or bupropion/Zyban) were mentioned in only 14.1% of responses .
- Craving management strategies — specific techniques like urge surfing, delay tactics, or cognitive behavioral strategies were rarely included.
- Secondhand smoke risks to others — a key public health message often omitted.
- Relapse prevention plans — most chatbots provided “how to quit” information but not “how to stay quit.”
🩺 ChatGPT-4.0 vs. Real Doctors: A Clinical Comparison
A separate 2025 study published in the NIH National Library of Medicine compared ChatGPT-4.0’s treatment recommendations against those of physicians in a smoking cessation clinic . The results were mixed:
📌 Overall Concordance: 67.1%
ChatGPT-4.0 agreed with physician decisions in about two-thirds of cases — promising, but far from reliable enough for autonomous use.
📌 Best Performance: Medication Recommendations (Cytisine)
The AI performed best when recommending cytisine, a smoking cessation medication, with 52.0% optimal recommendations .
📌 Worst Performance: Complex Patients
- Patients with underlying chronic diseases: 81.5% inappropriate recommendations
- Patients on long-term medication: 77.8% inappropriate recommendations
❓ Basic Information — Where AI Excels
For straightforward, factual questions, AI chatbots performed well. Examples of questions where AI was generally accurate included:
- “What are the health risks of smoking?”
- “How does nicotine addiction work?”
- “What are nicotine replacement therapy options?”
- “Where can I find smoking cessation resources?”
All chatbots were rated as having clear, easy-to-understand language for these basic queries .
⚠️ Specific Risks for Canadian Smokers Using AI Chatbots
Several unique concerns emerge when Canadian smokers turn to AI for cessation advice:
- Provincial differences: Most AI chatbots are trained on U.S. or global guidelines and may not be aware of provincial variations in cessation programs, prescription coverage, or age restrictions.
- Plain packaging laws: AI may not understand Canada’s unique plain packaging regulations when users ask about brand availability.
- Native cigarettes: General-purpose AI may lack knowledge about the legal status and availability of native cigarettes in Canada.
- French-language resources: Quebec-specific resources may not be included in training data.
📌 Recommendations — How to Safely Use AI for Smoking Advice
✅ DO Use AI Chatbots For:
- Basic information about smoking risks and cessation options
- Motivational content and success stories
- Finding general resources (quitlines, support groups)
- Understanding medical terminology
❌ DON’T Rely on AI Chatbots For:
- Personalized medical advice, especially if you have other health conditions
- Medication recommendations without consulting a doctor
- Evaluating alternative cessation methods (vaping, hypnosis, supplements)
- Urgent or crisis situations
🛡️ Best Practice — Always Cross-Check:
- Compare AI responses with official sources: Health Canada, Canadian Cancer Society, Lung Association
- Talk to your doctor or pharmacist before starting any cessation medication
- Call 1-866-366-3667 — Canada’s national quitline for free expert advice
📊 AI Chatbots vs. Traditional Cessation Resources
| Resource | Accessibility | Accuracy | Personalization | Cost |
|---|---|---|---|---|
| AI Chatbots (ChatGPT, etc.) | 24/7, instant | Moderate (up to 22% misinformation) | Low — general responses | Free (usually) |
| Canadian Quitline (1-866-366-3667) | Business hours (extended) | High — trained counselors | High — individualized plans | Free |
| Family Doctor | Appointment required | Very high | Very high — medical history considered | Covered by provincial health |
| Pharmacist (NRT consultation) | Walk-in (pharmacy hours) | High | Moderate-High | Free consultation (meds cost) |
📌 Honest Summary — The AI Verdict
Are AI chatbots accurate for smoking cessation advice? Partially — but not reliably. The best-performing bot followed clinical guidelines only 72.2% of the time .
How often is AI information wrong? 22% of responses contained misinformation — especially about alternative quitting methods like vaping, hypnosis, and supplements .
Can ChatGPT replace my doctor? No — especially for complex cases. For patients with chronic diseases, ChatGPT-4.0 gave inappropriate recommendations 81.5% of the time .
The bottom line: AI chatbots are powerful educational tools but dangerous if relied upon alone. Use them for basic information and motivation, but always verify with human experts — your doctor, pharmacist, or Canada’s quitline. Your health is too important to trust entirely to artificial intelligence.
🛒 Popular Native Cigarettes on Cigstore.ca
📚 You Might Also Find These Articles Interesting
🚚 Fast & Reliable Shipping Across Canada
$29 flat shipping on all orders under $290
Free shipping on orders $290 or more – anywhere in Canada
📦 Shipped via Canada Post, Purolator, FedEx, or UPS – carrier selected based on your location for fastest delivery.
Age verification required upon delivery (19+). Indigenous-owned – rooted in tradition, delivered with trust.
🤖 AI can inform. Humans can heal. Trust both wisely.
Native cigarettes from $29/carton — whether you’re smoking or quitting, knowledge is power. $29 flat shipping, free over $290.
🛒 Shop Native Cigarettes →Sources: JMIR Formative Research (2025) ; NIH National Library of Medicine study (2025) ; World Health Organization S.A.R.A.H. chatbot .