Human Terms

Is that AI headline real? Ask these 7 questions.

Most AI news starts as a company describing its own product. These seven questions take about a minute and catch most of the hype. Each one comes with a real example from this year.

Two are below. Get all seven, and a card you can save to your phone, by subscribing to Human Terms: one free email every Tuesday that runs the week’s AI news through these questions for you.

  1. Who’s making the claim, and what do they sell?

    A company announcing its own product is selling it. That doesn’t make the claim false. It means you wait for someone with nothing to sell.

    Real example. Before the US midterms, Anthropic said its AI scored 99% on a test of fairness about politics. Anthropic built that test and graded itself. It’s not an outside audit. Issue #13

  2. Who ran the test?

    The same AI can score very differently depending on who sets up the test. Look for the score from the people who run the test, not the people who made the AI.

    Real example. In September OpenAI’s GPT-6 Astra posted 99.9% on a puzzle test called ARC-AGI-3. The nonprofit that runs the test scored the same model at 62.7% with its standard setup. Same model, same questions, a 37-point gap. Issue #9

  3. Compared to what?

    “Faster” and “better” mean nothing until you know the comparison. Companies pick the contest they win.

    Real example. In December 2024 Google said its Willow quantum chip did a task in 5 minutes that would take a top supercomputer 10 septillion years. The task was chosen because it’s hard for normal computers, and Google says it has no everyday use yet. Google

  4. Can you use it today?

    “Announced” isn’t “available.” Check who can use it, and whether there’s a date for everyone else.

    Real example. Google’s Gemini 4 Carbon took first place on an independent AI scoreboard on October 28. Only a small group of security teams can use it, and there’s no date for anyone else. Issue #14

  5. How often is it wrong?

    “Smartest” and “most reliable” are different scores. Ask how often it makes things up.

    Real example. On that same scoreboard, when Gemini 4 Carbon didn’t know an answer, it made one up 11% of the time. Claude Opus 5.5 and GPT-6 Astra did so 50% of the time. Issue #14

  6. Is it a rule or a promise?

    A law can be enforced. A pledge can be dropped. Ask who can make them keep it.

    Real example. On September 29 the leaders of six big AI companies signed a safety pledge at the White House. It isn’t legally binding. The President called it “morally binding.” Issue #11

  7. Over what time frame?

    One day makes a headline. It takes weeks to make a trend.

    Real example. On November 5 the S&P 500 fell 4.4%, its worst day since April 2025, led by AI stocks. By November 9 it was less than half a percent below where it stood on October 30. Issue #15

Save it. Download the card (an image for your phone), or bookmark this page.

Every Tuesday at 7am, Human Terms runs the week’s AI news through these seven questions, so you don’t have to. Read past issues.