Keeping AI Honest: A Worked Example
This is a member-only chapter. Log in with your Signal Over Noise membership email to continue.
Log in to readModule 5 · Section 7 of 10
Keeping AI Honest: A Worked Example
Meet Priya, a freelance marketing consultant who uses AI for client proposal drafts, social media scheduling, and market research summaries.
The challenge: She needs her AI outputs to be accurate, sound like her (not like every other marketing consultant), and never misrepresent her capabilities to clients.
Her Approach:
The Essentials (Level 1):
- Client briefs go into ChatGPT Plus (data not used for training) — never the free tier
- Every proposal gets a fact-check pass before sending, especially any statistics or market claims
- She runs a monthly “mystery shopper” test — sending herself a sample AI-generated email to check if it still sounds like her
The Smart Habits (Level 2):
- She keeps her own templates and frameworks as the backbone, using AI to speed up drafting rather than replacing her thinking
- Her Constraints Document includes: “Never claim ROI figures without citing the source study” and “Never use the phrase ‘in today’s fast-paced digital landscape’”
- Every quarter, she reviews her AI configurations against her current services and client types
Her Content Classification:
- For her eyes only: research summaries, competitor analysis drafts, brainstorming sessions
- Client-facing: proposals, strategy decks, campaign reports — all get her expert review and specific client context added
The result: Her clients consistently comment that her proposals feel personal and well-researched. She’s faster than she was without AI, but the quality and voice remain distinctly hers.
Key Principle: Your review process should grow alongside your AI use. Build guardrails that are clear enough to use now but flexible enough to improve as you learn — don’t try to anticipate every scenario upfront.