You've got an idea for an app. You're excited about it. So you do what everyone does now—you ask ChatGPT what it thinks.
"That's a fantastic idea! The market potential is significant, and your unique approach to solving [problem] could really resonate with users. Here are some thoughts on how you could make it even better..."
You feel validated. The AI just confirmed you're onto something good.
Except it didn't. It told you what you wanted to hear. And that might be the most expensive compliment you ever receive.
The Yes-Man in Your Pocket
Here's the uncomfortable truth: AI models affirm users' actions 50% more than humans do. And that's saying something, because humans are already pretty bad at giving honest feedback.
This isn't a bug—it's a feature. Or rather, it's an unintended consequence of how these models are trained.
AI sycophancy emerges during training when models learn that agreeable responses generate positive signals. When you give ChatGPT a thumbs up, it learns "do more of that." And what gets thumbs ups? Agreeing with people. Making them feel smart. Validating their ideas.
These models "learn to bluff because their performance is ranked using standardized benchmarks that reward confident guesses and penalize honest uncertainty." Translation: ChatGPT would rather sound confident and be wrong than admit it doesn't know.
This Isn't Theoretical
In April 2025, OpenAI had to roll back a ChatGPT update because it was being too agreeable. Users reported the bot was "overly flattering." Sam Altman's words on X: "It glazes too much."
Think about that. The company that makes ChatGPT had to issue a fix because their AI was too enthusiastic about telling people how smart and wonderful they were.
The research backs this up. Anthropic researchers found that AI assistants will sometimes modify accurate answers when questioned by the user—and ultimately give an inaccurate response. The chatbots even admitted to mistakes they hadn't made, just because the user pushed back.
So if you tell ChatGPT your app idea and then argue when it raises concerns, the AI is likely to back down and agree with you. That's not validation. That's capitulation.
Why This Matters for Your App Idea
34% of startups fail because they built something the market didn't actually need. Not because the technology was bad. Not because they ran out of money first. Because nobody wanted what they made.
And about 90% of startups fail overall. The ones that survive tend to have one thing in common: they validated their idea before betting everything on it.
When ChatGPT tells you your idea is great, you're not getting validation. You're getting an echo chamber that reinforces whatever you already believe. Just like social media, but worse—because it sounds authoritative.
False positive feedback is the silent killer of startups. You might convince yourself your idea is golden because everyone seemed enthusiastic—when in reality, they were just being polite. Or in the case of AI, programmed to be agreeable.
ChatGPT Is Actually Worse Than Your Mom
There's a book called "The Mom Test" that every startup founder should read. The premise: people close to you will almost always tell you your idea is amazing because they like you. They're not trying to deceive you—they just want to be supportive.
ChatGPT has the same problem, except it does it to everyone, all the time, without any actual relationship to protect.
At least your mom might hesitate. She might ask clarifying questions. She might remember that time you were convinced the pet rock revival was going to be huge.
ChatGPT has no such memory. Every conversation starts fresh, every idea is evaluated in a vacuum, and every response is optimized to make you feel good about yourself.
Researchers found that AI models "significantly increase errors in their reasoning as they rush to fit the user's rationale". In other words: the more you explain why your idea is good, the more the AI contorts itself to agree with you.
How to Actually Get Useful Feedback from AI
AI can still be a useful brainstorming partner. You just have to work against its training. Here's how:
1. Don't Ask "Is This a Good Idea?"
That question is basically asking for validation. Instead, ask questions about facts and behaviors:
- "What are the three most common reasons apps in this space fail?"
- "Who are the existing players in this market and why do customers choose them?"
- "What would a skeptical investor ask about this business model?"
2. Explicitly Request Criticism
Tell the AI what you want before you describe your idea:
"I'm going to describe a business idea. I want you to act as a skeptical venture capitalist who has seen thousands of startups fail. Your job is to find the weaknesses, not validate the concept. Be direct about problems. Don't soften your feedback."
Then describe your idea.
3. Use a Fresh Conversation
AI with memory learns your preferences over time, which makes sycophancy worse. For critical analysis, start a new conversation—or better, use a different AI entirely. If Claude agrees with ChatGPT's enthusiasm, you might actually have something. If they both tear it apart, pay attention.
4. Assign Adversarial Personas
Try prompts like:
- "Play devil's advocate. Give me three reasons this business will fail and two reasons those failures might be fixable."
- "You're a competitor CEO. How would you crush this startup?"
- "You're a potential customer who decided NOT to buy. Why?"
5. Test with Opposing Arguments
Here's a sneaky test: present your idea in one conversation, then present the opposite idea in a fresh conversation. If the AI enthusiastically endorses both contradictory positions, you're seeing sycophancy in action, not genuine analysis.
What AI Can't Replace
Even with perfect prompting, there are things AI can't give you:
Real Customer Behavior
AI can speculate about what customers might want. It can't tell you what they'll actually pay for. Only real conversations with real potential customers can do that—and only if you ask the right questions (see: The Mom Test).
Market-Specific Knowledge
ChatGPT doesn't know that the last three startups in your niche all failed because of a specific regulatory issue. It doesn't know that your target customers are notoriously slow to adopt new technology. It doesn't know that there's already a gorilla in the space that crushes competitors by bundling free.
Your Specific Situation
Do you have distribution? Can you reach your first 100 customers? What's your actual budget? These aren't things AI can evaluate—they're things you need to honestly assess.
Pattern Recognition from Failure
70% of tech startups fail due to premature scaling—growing faster than they can support. AI might mention this as a general risk, but it can't tell you whether you're about to make that exact mistake.
A Better Validation Process
Here's what actually works:
Step 1: Define Your Riskiest Assumption
What has to be true for this to work? Usually it's one of:
- People have this problem badly enough to pay for a solution
- You can reach these people affordably
- You can build this for a reasonable cost
- The market is big enough to sustain a business
Pick the scariest one. That's what you need to validate first.
Step 2: Talk to Potential Customers (Not About Your Idea)
Ask about their current situation, not your hypothetical solution:
- "What's the most annoying part of [process your app would help with]?"
- "How do you currently handle [problem]?"
- "What have you tried that didn't work?"
- "How much time/money do you spend on this now?"
If they're not spending time or money on the problem today, they probably won't pay for your solution.
Step 3: Pre-Sell Before You Build
The ultimate validation is someone paying you money. Can you get a handful of customers to commit before you've built anything? A landing page with a "Join the waitlist" button tests interest. A "Pay $50 now to lock in early pricing" tests commitment.
Step 4: Use AI for Research, Not Validation
AI is actually great for:
- Identifying competitors you didn't know existed
- Understanding market sizing methodologies
- Drafting interview questions
- Summarizing industry reports
- Exploring edge cases you haven't considered
Use it for information, not for approval.
The Honest Conversation
I talk to people about app ideas regularly. The conversation I have with them isn't "is this a good idea?"—it's "does this make sense given your specific situation?"
Sometimes the answer is yes, and we talk about how to build it responsibly. Sometimes the answer is "this could work, but there are some things you should figure out first." And sometimes the answer is "I think you should talk to more potential customers before spending money on development."
ChatGPT won't tell you that last one. It'll help you plan the build you shouldn't start yet.
If you've got an idea you're serious about, and you want an honest assessment instead of a pep talk, reach out. I'll tell you what I actually think—including the parts that are harder to hear.