How Do I Validate an AI-Generated Data Visualization Before I Publish It?

The dangerous visualization is often the one that looks finished

AI can now produce a polished chart, map, dashboard, or interactive in minutes. That is useful. It also creates a new kind of review problem.

The output can look completely convincing while being wrong.

The colors are right. The labels are clean. The animation works. The tooltip shows a number with two decimal places. Nothing crashes. The code is sophisticated enough that it would take real effort to recreate by hand.

And the AI may still have used the wrong denominator, joined on the wrong key, treated missing values as zero, mixed dollars and thousands of dollars, compared partial years as if they were complete years, silently dropped records, or drawn the right chart from the wrong field.

That is why "it rendered" is not a validation test and "the AI checked it" is not independent evidence.

The useful question is not whether the AI seems confident. It is: can I trace the important claims in this visualization back to data and definitions I trust?

That is the review problem Rhubarb is designed to make manageable. The current stack is simple to describe: keep the work visible, editable, versioned, and cross-checked.

Visible: make the chain of reasoning inspectable

A visualization is easier to trust when the important steps are not hidden inside an opaque answer.

For a Rhubarb project, that means being able to inspect the relevant data-source logic and the visualization implementation separately. Depending on the source, the data layer may contain a PostgreSQL query, an editable SQL transform, a Data Blend, source configuration, or other preparation that determines the records handed to the visualization. The visualization layer keeps its HTML, CSS, and JavaScript visible.

That separation matters because there are at least two different ways an AI visualization can be wrong:

• the data handed to the visualization is wrong; or
• the visualization misrepresents data that was otherwise correct.

If a percentage is wrong, you want to know whether the data-source logic calculated the wrong numerator or denominator, or whether the JavaScript divided or formatted the correct fields incorrectly. If a category is missing, you want to know whether it disappeared in the query or whether the rendering code filtered it out.

Visible code does not make either layer correct. It makes the mistake findable.

Editable: review has to be able to change the result

Transparency is not very useful if the only response to finding a problem is to regenerate everything and hope the next answer is better.

When you find a bad assumption in Rhubarb, the relevant data-source SQL or transform and the visualization code remain editable. You can correct the field, grouping, filter, label, calculation, scale, or interaction directly. You can also ask the Assistant for a narrow change while preserving the rest of the working implementation.

That changes the role of AI. Instead of treating the generated answer as an oracle, you treat it as a fast first implementation that can be interrogated and corrected.

A useful review request is specific:

Show me exactly where this percentage is calculated. Do not change the visualization. Explain the numerator, denominator, filters, and missing-value behavior, then compare the resulting value for one known category with the data-source preview.

That is very different from: Does this look right?

The first request exposes an assumption you can inspect. The second invites reassurance.

Versioned: make review reversible

AI-assisted editing is fast enough that it is easy to destroy a correct result while trying to fix something else.

Before a substantial change, save a version.

Rhubarb supports saved versions of visualization code and saved versions of data-source logic or configuration. If a revision breaks the visual, changes an established calculation, or removes behavior you wanted to preserve, you have a known prior state to compare against or restore.

Versioning is also useful during validation because it lets you make the review process explicit:

1. save the version that produced the original claim;
2. identify the assumption you are testing;
3. make one narrow correction;
4. cross-check the changed result;
5. save the validated revision.

But versioning has an important limit: a saved code or datasource-logic version is not automatically a historical snapshot of every external source value. If the authoritative database, spreadsheet, or file changes, you may need that source system's own history, a retained input file, or documented extraction date to reconstruct exactly what was true at publication time.

Versioning protects the implementation history. It does not manufacture source provenance that never existed.

Cross-checked: make the important numbers meet evidence outside the artifact

This is the step an AI-generated visualization cannot do alone.

An AI can write a second calculation. It can inspect its code. It can generate tests. It can point out suspicious joins. It can even compare two methods and tell you they agree.

All of that is useful, but if every check begins from the same wrong assumption, the checks can agree with one another and still be wrong.

A meaningful cross-check introduces evidence that is independent of the visualization being reviewed. That might be:

• a known value from the authoritative source;
• a direct query against the source database;
• a spreadsheet calculation made independently;
• a published total from the organization that owns the data;
• a hand calculation for a small, understandable subset;
• a second query written from the definition rather than copied from the generated query.

You do not need to manually reproduce an entire million-row analysis. You need strategically chosen checks that would expose the kinds of mistakes the visualization could plausibly contain.

Cross-check the boring cases, not only the headline number

The headline value is important, but many visualization errors live at the edges.

Pick a compact validation set. For example:

• one row or category whose value you can identify directly in the source;
• the largest and smallest valid values;
• a total that should reconcile to a source total;
• one percentage or rate, recalculated from its numerator and denominator;
• the first and last date in the intended period;
• a category with missing or null data;
• a zero value if zero has a different meaning from missing;
• a filtered state where you know what should remain;
• a deliberately empty selection.

For maps, add a geography check: verify a handful of identifiers and make sure the value shown for a place actually belongs to that place. For time series, verify the period boundaries and whether partial periods are being compared with complete ones. For weighted survey data, verify the weighting and denominator rather than merely checking that percentages add to roughly 100.

This is not glamorous work. It is exactly the work that turns a plausible visualization into a defensible one.

The data preview and the visualization should tell the same story

Rhubarb keeps the data source and visualization close together so you can compare what the browser receives with what the visual claims.

Use the data-source Table or JSON preview to inspect representative records and fields. When necessary, inspect the full result through the explicit full-preview path rather than assuming a bounded preview represents every edge case. Then compare those records with the labels, tooltips, totals, and interactions produced by the visualization.

A few useful questions are:

• Is the visualization using the field I think it is using?
• Are numeric strings being parsed as numbers correctly?
• Are nulls excluded, shown, or converted to zero?
• Is the visual aggregating records that were already aggregated upstream?
• Are dates interpreted in the intended time zone and period?
• Does a filter change only the intended subset?
• Do tooltips show the same values contained in the prepared data?

If the chart says 47.3%, find the records or aggregate that produce 47.3%. The number should have an explainable path back to the source.

Check the definition before you check the arithmetic

A perfectly executed calculation can still answer the wrong question.

Suppose the AI reports a churn rate of 8.4%. Before checking the division, ask what "churn" means. Customers who cancelled during the month? Customers inactive for 30 days? Accounts lost divided by starting accounts? Average active accounts? Does reactivation count? Are trial users included?

AI is especially dangerous when a business or research term sounds familiar enough that the model can confidently choose a reasonable definition without knowing the definition your organization actually uses.

So validation should move in this order:

1. definition: what exactly is being measured?
2. population: which records are included and excluded?
3. transformation: how are joins, filters, grouping, weighting, and missing values handled?
4. calculation: does the arithmetic implement the definition?
5. visual encoding: does the chart represent that result faithfully?
6. interaction: do filters, hover states, selections, and responsive states preserve the meaning?

The prettiest JavaScript in the world cannot rescue an undefined metric.

Do not use AI confidence as a QA signal

One of the easiest failure modes is asking the model that created the visualization to review it and then treating a confident explanation as validation.

Use AI aggressively during review—but give it jobs that produce inspectable work:

• identify every calculation and its inputs;
• list every filter and exclusion;
• describe the expected grain of one record;
• find places where nulls or type coercion could change a result;
• generate edge-case tests;
• compare the visualization's calculation with a separately written source query;
• explain exactly why one displayed value should equal one independently known value;
• flag code paths that mutate data or depend on hidden state.

Those are useful tasks because you can examine the output.

"I reviewed this and it is correct" is not useful evidence merely because an AI wrote the sentence.

A second AI is a reviewer, not an authority

Using another model can improve code review. Different systems notice different problems, and a fresh context may expose assumptions the first conversation normalized.

But two AIs agreeing is not the same as two independent measurements agreeing.

If both models receive the same incorrect source, the same ambiguous definition, or the same misleading sample, they can converge on the same wrong answer. A second model is best used to generate adversarial questions and alternative calculations. The final cross-check should still terminate in something authoritative: source data, a governed definition, a separately derived result, or a known fact.

The independence has to come from the evidence, not merely from changing the chatbot.

Validate changes, not just the first version

A visualization can be correct today and become wrong after an apparently harmless edit.

Changing a label may be safe. Changing a filter, field mapping, join, date range, aggregation, or data source is not merely cosmetic. Neither is asking the Assistant to "simplify" the code if the simplification changes when calculations occur.

After a substantive change:

1. compare the new version with the previous version;
2. rerun the small cross-check set;
3. confirm that unchanged cases remain unchanged;
4. test the specific edge case the change was meant to address;
5. save the new version only after the result is understood.

You do not need a giant formal QA department to do this. You need a repeatable habit and enough visibility to know what changed.

Know what Rhubarb can and cannot affirm

Rhubarb can make an AI-assisted visualization much easier to review because the relevant implementation does not have to disappear behind the generated result. You can inspect and edit the data preparation, inspect and edit the visualization code, preview the data, save and restore versions, and keep the published result attached to the same project.

Rhubarb does not certify that a chart is true merely because it was built in Rhubarb. It cannot know that an upstream source was collected correctly, that your organization's metric definition was communicated correctly, or that a factual claim is authoritative unless those things are supplied and checked.

The product can support a strong validation process. It cannot replace one.

That boundary is important because the goal is not to create a more convincing AI black box. The goal is to make AI-produced work auditable enough that a person can decide whether to trust it.

A practical pre-publication review

Before you publish an AI-generated data visualization, ask whether you can answer all of these questions:

• Visible: Can I see where the data comes from and the code that turns it into the displayed result?
• Editable: If I find a bad assumption, can I correct it without regenerating the entire artifact?
• Versioned: Can I preserve the working state before a material change and restore it if necessary?
• Cross-checked: Have I reconciled representative values and definitions against evidence outside the generated artifact?
• Bounded: Do I know what this validation does not prove—for example, the correctness of an upstream source I did not audit?

If the answer to the first four is yes, and you understand the fifth, you have something much stronger than an AI-generated chart that merely looks convincing.

You have a visualization whose important claims can be inspected, challenged, corrected, and defended.

Frequently asked questions

Can I trust an AI-generated chart if the code runs and the visualization looks right?

No. A chart can render perfectly while using the wrong denominator, date range, join, aggregation, unit, filter, or field. Rendering proves that the browser can execute the code; it does not prove that the analysis is correct.

What should I cross-check before publishing an AI-generated visualization?

Choose a small set of values that exercise the analysis: a known total, a specific row or category, the largest and smallest values, a date boundary, a null or missing case, and any calculated rate or percentage central to the story. Reconcile those against the authoritative source or an independent query or calculation.

Can another AI validate the visualization for me?

A second AI can be useful for code review, test ideas, and finding suspicious assumptions, but it is not a substitute for independent evidence. If both systems receive the same incorrect definition or source data, they can agree and still be wrong. Cross-check the important claims against an authoritative source or separately derived result.

What does versioning prove in Rhubarb?

Versioning gives you a recoverable history of visualization code and data-source logic or configuration so you can compare changes and revert mistakes. It does not prove that an external source value was correct at a particular moment unless that source itself is retained or versioned.

Does Rhubarb automatically certify that an AI-generated visualization is correct?

No. Rhubarb makes the work inspectable and gives you practical surfaces for review: visible and editable code, data previews, versions, and a workflow for cross-checking results. Correctness still depends on the source data, definitions, transformations, and the review performed before publication.