The Defensibility Test

01.07.26 06:06 PM - By Angela Manchester

How to Trust an AI-Generated EO Output

AI tools that touch Earth Observation data are everywhere now. 

Feed a folder of imagery in, ask a plain-language question ("show me the areas that need attention"), 

and get a finished map or answer back in seconds. 


The speed is real. What is less settled is whether the answer is right, and whether you could stand behind it if someone pushed back.

That is the actual bottleneck right now, and it is worth naming clearly: the limit is not access to data (there is more free EO data available than most people use) and it is not really interpretation either (AI is often decent at that). The limit is trust. Specifically, whether a professional can defend the output: which data it used, which assumptions it made, and whether those choices were the right ones for this question.

This matters more in EO than in a lot of other domains, because the outputs feed real decisions: where to send a maintenance crew, whether a flood claim is valid, whether a piece of land has changed use. An answer that looks clean on screen can still be built on the wrong buffer distance, the wrong cloud mask, or an assumption that quietly does not hold for your area.

The four-question defensibility test

Before you present or act on any AI-generated EO output, run it through four questions. This applies regardless of which tool produced it, this is not about one platform over another.

1. Which datasets did it actually use, and are they the right vintage and resolution for this question? An AI tool will happily use whatever imagery is available in the folder or the default source it was pointed at. That is not the same as the right imagery. A land-use answer built on a two-year-old scene, or a flood answer built on 30m resolution when the flood boundary needs 10m, will look confident and still be wrong.

2. Which assumptions, buffers, filters, or thresholds did it apply, and would you have chosen them yourself? Every "show me the areas that need attention" question hides a dozen small decisions: how wide a buffer around infrastructure, what counts as "change," what threshold separates signal from noise. An AI tool picks something. Your job is to find out what, and check it against how you would have done it.

3. Can you point to the specific pixels or features that support the conclusion? If the answer is "the model said so" and you cannot trace it back to something visible in the data, you cannot defend it in a meeting. You should be able to open the source imagery and show the evidence, not just the output.

4. What would change the answer, and have you checked whether that condition holds? Every EO conclusion is conditional on something: a cloud-free scene, a stable sensor calibration, a correct date range. Name the condition. Then check it, the same way you would check metadata before trusting any other satellite product.

Why this is the actual skill

None of this is anti-AI. AI in EO workflows is only going to get more common, and used well it saves real time. The skill that matters now is not learning to prompt it better, it is learning to interrogate what it hands back before your name goes on it. That is the same discipline that already applies to any EO output, human or automated: treat every result as something you have to defend, not something you get to trust by default.


Open question
What is one AI-generated EO output you have seen, or built yourself, that you would not have signed off on without checking the underlying data first? 
What did you actually check?
This is free, standalone thinking, part of the Earth Observation community's vendor-neutral approach to EO education.

Angela Manchester