Prophecy 4.2.10 is all about removing friction from everyday work: writing to existing tables, previewing data, reading pipeline errors, and editing common transforms are each faster and clearer than before. Let’s get into it.
Write to existing tables without the schema guesswork
When you write to a table that already exists, Prophecy now gives you a dedicated Schema Mapping experience, eliminating schema errors and type matching difficulties. Choose to overwrite the target’s schema, or map column-by-column with AI-suggested mappings to get you most of the way there. Type mismatches are reconciled, extra columns are handled predictably, and you see exactly what will happen before you run.
See data previews and record counts for quick validation checks
Data previews and record counts are now under your control, gem by gem. Instead of Prophecy deciding which node to compute, you turn on a Data Preview or Record Count exactly where you want it, and get exact counts for sanity checks after joins or complex formulas. On large pipelines, that means increased efficiency as you skip the wait for previews you didn’t ask for.
A guided, no-code Aggregate gem
One of the most commonly used transforms just got a full makeover. The Aggregate gem now opens in a guided visual mode with clear, drag-to-reorder cards for grouping, aggregations, and filters. No more editing raw configuration to build a group-by. Power users can still drop into advanced mode for complex expression code editing, but the everyday case is now fast, visual, and hard to get wrong.
Understand pipeline failures at a glance
Runtime errors now open with a short, plain-language summary and a gem-by-gem breakdown of what failed. When a pipeline has several failures at once, you can step through them one by one. What used to be a wall of raw Spark or dbt output is now a guided path to the fix.
Questions or feedback? Let us know in the comments below!