Skip to content
AI Tools

How to Evaluate an AI Tool Before You Trust It

When not to use AI tools for important decisions, explained through risk awareness, trust limitations, and practical decision‑support boundaries.

How to Evaluate an AI Tool Before You Trust It

Most AI tools guides focus on what you can do with them. This one focuses on when you shouldn’t use them at all — a conversation that happens far less often but matters just as much. I’ve made the mistake of reaching for AI tools in situations where they made things worse: introducing errors into work that needed to be accurate, creating legal and privacy risks I hadn’t considered, and undermining skills and relationships that would have served me better than efficiency. You’ll find the complete rundown in our Complete Guide to AI Tools.

Knowing when not to use AI tools isn’t about being anti-AI. It’s about using them wisely enough that the situations where they genuinely help aren’t undermined by the situations where they cause problems.

Medical, legal, and financial decisions

The clearest cases. There’s a meaningful distinction worth stating precisely:

Using an AI tool to understand a medical condition in general terms — what it is, what the common treatments are, what questions to ask a doctor — is reasonable. Using it to decide on a specific treatment, interpret your specific test results, or replace a consultation is not. The AI doesn’t have your medical history, doesn’t know about drug interactions with your other medications, and may be confidently describing treatment guidelines superseded by more recent research.

Same distinction applies to legal questions. AI tools can help you understand general legal concepts and prepare questions for a consultation. They cannot reliably tell you whether a specific contract clause is enforceable in your jurisdiction, whether you have a viable legal claim, or how to navigate a specific dispute — and the cost of being wrong on legal questions can be severe and irreversible.

For financial decisions — investment choices, tax strategy, retirement planning — the AI’s knowledge cutoffs, lack of access to your specific financial situation, and the constantly changing nature of financial regulations make AI output an unreliable basis for decisions with significant consequences. Use AI tools to learn about financial concepts; use qualified professionals for decisions that matter.

Situations requiring verified current information

AI tools with training data cutoffs cannot reliably provide current information. This is a fundamental technical limitation, not a gap that will be fixed by the next update. The specific risk isn’t just getting outdated information — it’s getting outdated information presented confidently, without any indication it may no longer be accurate.

I’ve seen AI tools describe regulations that had been amended, products that had been discontinued, and company policies that had changed — all in present tense, with no qualification. For anything time-sensitive, the right tool is one with real-time web search capability (Perplexity, Bing AI) or a primary source you can check directly.

When human relationships are the point

Some tasks derive their value from the human process of producing them, not from the quality of the output itself. The clearest example from my own experience: a handwritten note of condolence to a colleague who had lost a parent. The note doesn’t need to be eloquent. What it needs is to be genuine — to reflect that a human who knows this person took time and emotional energy to acknowledge their loss. An AI-generated condolence message, however well-worded, is the opposite of that.

The same principle applies to personal apologies, letters of recommendation (where the recommender’s genuine knowledge of the candidate is the entire point), wedding speeches, and feedback conversations that need to be honest and human rather than polished and generic. When the relationship and the human effort are what make the communication meaningful, AI tools diminish rather than improve the result.

Confidential or sensitive information

Pasting confidential information into a consumer AI tool is a data privacy risk that many people don’t adequately consider before doing it. When you paste a client’s business data into ChatGPT to save time on analysis, you’re potentially sharing that data with a third party in a way the client hasn’t consented to and that may violate your obligations to them.

The specific contexts where this risk is serious:

  • Client or customer personal data — names, contact details, financial information, purchase history
  • Confidential business information — unreleased product plans, financial projections, personnel matters
  • Medical or health information — patient records, personal health details
  • Legally privileged communications — anything covered by attorney-client privilege
  • Information under NDA — by definition, information covered by a non-disclosure agreement cannot be shared with unauthorised third parties, and an AI tool provider qualifies as one

Our guide on AI tools and data privacy covers the specific data handling policies of major tools and the enterprise options that provide better privacy guarantees for professional use.

Creative work where originality is the actual value

AI tools can produce competent, fluent creative writing. What they can’t produce is genuinely original work that reflects a specific human perspective, lived experience, or distinctive voice. For creative work where those qualities are what make it valuable — literary fiction, personal essays, original journalism, distinctive brand voice — AI tools are the wrong starting point, even if they produce output that superficially resembles the target.

The nuance here matters: AI tools as a brainstorming scaffold (generating ten possible openings when you can’t find the right entry point, then writing the actual opening yourself) is different from AI tools as a replacement for the creative work itself. One is using AI to remove friction. The other is substituting a process that should be yours with one that isn’t.

When developing the skill is the point

If the purpose of a task is to develop your own capability — learning to write clearly, learning to code, learning to analyse data — outsourcing it to an AI tool defeats the purpose. The test I apply: is this a task I want to be able to do myself in a year? If yes, doing it with AI as a crutch is counterproductive. If no — if it’s a task outside my domain that I need done competently but have no intention of developing personal expertise in — AI assistance is entirely appropriate.

Situation Why not to use AI tools Better alternative
Medical treatment decisions Hallucination risk; lacks your specific history Qualified medical professional
Current events and prices Knowledge cutoffs produce outdated or fabricated answers Real-time search; primary sources
Personal condolences or apologies Human effort and genuineness are the point Write it yourself, however imperfectly
Confidential client data analysis Data privacy risk; potential consent violation Enterprise AI tools with appropriate data agreements
Skills you want to develop Outsourcing prevents the learning that comes from struggle Do it yourself; use AI to review, not to produce

The framework underlying all of these: three questions worth asking before starting any AI-assisted task. How serious are the consequences if the AI is wrong? Does the task require something AI tools genuinely can’t provide — human judgment, real relationships, verified expertise? Am I using AI to avoid something I should actually do myself? When the answer to any of these points toward caution, the right decision is usually to use AI only as background support rather than as the primary engine of the work, or not at all.

Our guide on ethical use of AI tools covers the broader framework within which these restrictions sit. For the technical reasons why AI tools fail in high-stakes contexts, our guide on AI tools limitations and risks goes into the specifics of hallucination, knowledge cutoffs, and training bias.

The cost of using AI tools when you shouldn’t

The decision to not use AI tools in specific situations has real costs — the time and effort of doing things manually that AI could accelerate. Understanding this helps make the “when not to use AI” decision deliberately rather than reflexively.

A handwritten condolence note takes longer than an AI-drafted one. A lawyer doing legal research takes longer than an AI doing it. Learning a skill yourself takes longer than having AI do it for you. These costs are real, and the framework for “when not to use AI tools” isn’t about ignoring them — it’s about weighing them against the costs on the other side: the relationship damage from a condolence note that doesn’t feel genuine, the professional liability from legal advice that’s incorrect, the skill gap from outsourcing the work that would have built the capability.

The cases where not using AI has a cost worth paying:

  • When the human process of producing something is what creates its value — a personal apology, a letter of recommendation, creative work with genuine voice
  • When the stakes of being wrong exceed any efficiency benefit — medical decisions, legal strategy, financial planning
  • When the data involved carries obligations that consumer AI tools can’t accommodate — confidential client data, legally privileged information, regulated personal data
  • When the task is specifically meant to develop a skill you want to retain

Outside these categories, the default should lean toward using AI tools and verifying the output — not toward manual alternatives that cost more time without a clear offsetting benefit.

How to check whether you’re in “don’t use AI” territory

When you’re unsure whether a situation falls into the “don’t use AI” category, three questions clarify it quickly:

Could an error cause serious harm? Medical treatment decisions, legal judgments, financial advice — yes. Drafting an internal memo, generating a project outline, summarising a long report — no. If serious harm is possible and AI reliability in the domain is lower than the harm potential, don’t use AI as the primary source.

Is the human effort part of the value? A reference letter, a personal speech, an apology, original creative work where your voice is what the audience came for — the effort signals something. If the recipient would feel the absence of that effort if they knew, the human process matters and AI substitution undermines the value.

Are there data or confidentiality obligations involved? Client data, patient information, legally privileged communications — obligations attach to this data regardless of how you’re processing it. Consumer AI tools are not authorised third parties for most confidentiality obligations. If you’d hesitate to email the content to a stranger, hesitate to paste it into a consumer AI tool.

If none of these three questions produces a “yes,” the situation probably doesn’t fall into the “don’t use AI” category and the productivity case for using AI tools with appropriate review applies.

The nuanced middle: AI as support vs AI as substitute

Most of the “when not to use AI” situations aren’t absolute prohibitions on any AI involvement — they’re about the role AI plays. There’s a significant difference between AI as a support tool and AI as a substitute, and many situations that seem like “don’t use AI” cases are actually “don’t use AI as a substitute” cases where AI as support is entirely appropriate.

Medical decisions: don’t use AI as a substitute for professional medical advice. Do use AI to understand medical terminology, prepare informed questions for a doctor appointment, and understand the general landscape of treatment options before a consultation. The AI provides background orientation; the doctor provides the advice.

Legal matters: don’t use AI to determine whether you have a viable legal claim or how to interpret a specific contract clause in your specific situation. Do use AI to understand the general legal framework, identify what type of legal professional you need, and prepare specific questions for a legal consultation. The AI reduces the time and cost of getting to a productive professional conversation.

Skill development: don’t use AI to produce the work that should be developing the skill. Do use AI to check your work after you’ve done it yourself, understand why something you produced isn’t working, or get explanations of concepts you’re learning. The AI is a study partner and reviewer, not a shortcut past the learning.

This AI-as-support rather than AI-as-substitute framing resolves most of the apparent tension in the “when not to use AI” question. Very few situations call for zero AI involvement. Many situations call for AI involvement that stops short of substituting for the human judgment, expertise, or effort that’s actually needed. Our guide on AI tools vs human judgment covers the specific decision framework for determining when AI handles the work and when humans must — which is the practical extension of the situations covered in this guide.

The clearest signal that AI tools are being used appropriately in a given situation is that the human judgment, accountability, and effort that should be present are still present — and AI is handling the parts that don’t require them. When that’s the case, the efficiency is genuine and the risks are managed. When AI is handling things that should require human judgment, accountability, or genuine effort, that’s where the situations in this guide become relevant.

Nikolas Lamprou

Nikolas Lamprou (MSc; GCFR, SC-200, Security+) has been working with computers professionally since 2009 — starting with web development and e-commerce, and moving into cybersecurity over the years. Based in Greece, he brings over 15 years of real-world IT experience to SolveTechToday, where he writes about Windows fixes, software reviews, security tools, and AI applications. His goal is straightforward: cut through the noise and give readers clear, honest guidance on the tech decisions that matter.

Stay Ahead

Fix your next problem before it starts

Get the week's best Windows fixes, software picks, and security guides delivered straight to your inbox. No noise, just solutions.

Press ESC to close · Try "Windows 11" or "Chrome"