Treat violations as real data issues.
There is enough evidence or context to support the relationship.
DATAVEID LAB · APPLIED AI
AI spotted a pattern. Was it actually a data problem?
AI can notice patterns predefined checks may miss. I tested what should happen when it suggested a plausible relationship that was not yet confirmed by the available evidence and context.
WHAT ACTUALLY HAPPENED
A plausible relationship needed more support.
The test dataset was controlled: expected findings and known business rules were defined beforehand. The AI discovery step proposed an additional relationship.
AI suggestion
status → amount_dueThe model inferred a possible relationship between status and amount due. That relationship was not sufficiently supported by the available evidence and context to be treated as a confirmed defect.
The suggestion was plausible, but the available data and context did not establish that it represented a real defect. It therefore remained an AI discovery rather than being treated as a confirmed data-quality problem.
- No quality-score impact
- No automatic data modification
- Not presented as a confirmed data-quality problem
A SIMPLER EXAMPLE
What supports this relationship?
Suppose AI notices that completed records usually have a completion date.
AI suggestionStatus = Completed
→ Completion Date should exist
→ Completion Date should exist
This may genuinely be a required relationship in some organizations. But observing the pattern alone does not prove that a violation is a real data-quality problem.
Support can come from an established business rule, strong evidence in the data, trusted domain context, or human confirmation.
Keep it available for review.
The relationship remains plausible but unconfirmed.
WHAT I TESTED
Controlled cases with known expectations.
I used controlled cases where the expected rules and findings were known before evaluating the AI output.
| Test case | Expected behavior |
|---|---|
| AI discovery is strongly supported by data, context, or an established rule | Allow it into normal review as a supported finding |
| AI discovery is plausible but evidence is insufficient | Keep it as an unconfirmed suggestion for review |
| AI repeats an issue already found by engineered checks | Do not create another issue |
| AI relies on unavailable or insufficient context | Reject as unsupported |
HOW THE DETECTION LAYERS DIFFER
Different layers answer different questions.
Engineered checksKnown structural, validity, consistency, uniqueness and statistical checks.
Business rulesOrganization- or use-case-specific expectations.
AI discoveryPatterns or relationships that may not have been explicitly encoded beforehand.
These layers complement each other; none of them is meant to replace the others.
THE ENGINEERING DECISION
Separate discovery from confirmation.
I wanted AI to remain free to surface problems and relationships that predefined checks or business rules might miss.
The decision was not to require every AI discovery to match something already known. Instead, a discovery needs enough supporting evidence or context before it can be treated as a confirmed data-quality problem.
This keeps AI useful for finding genuinely new issues while preventing plausible but weakly supported suggestions from being treated as established defects.
VALIDATION
How I validated it
Validated against:
- Supported relationship
- Unsupported business relationship
- Duplicate discovery
- Unsupported uniqueness-style claim
- Insufficient-context case
- Reference to unavailable data or context
Automated regression coverage checks that unsupported suggestions do not enter the normal scoring, cleaning, or confirmed-issue path.
Methodology and scope
Expected business rules and findings were defined before evaluating the AI output. This case tested whether AI-generated suggestions were handled appropriately before entering the normal data-quality workflow. It was not designed to measure recall across every possible unknown issue type.