Spark Version Atlas — 3.5 to main
One city, five editions. Buildings are Apache Spark subsystems verified per version anchor; routes are real control and data paths. Purple is the photorevision overprint: what changed between the edition you are reading and the edition you compare against.
Revision box
Surveyor’s report
Select a building on the map (click, or Tab + Enter) to read its role, era timeline, and source citations.
Legend
How to read this sheet
Each edition tab renders the city as it stands at that version anchor. The overprint works like a USGS photorevised quadrangle: surveyors reprinted old sheets with changes overprinted in purple rather than redrawing the base map. Here, solid purple marks buildings and routes added since the compared edition, dashed purple ghosts mark removals, and purple triangles mark buildings that moved or changed internally. Set the overprint to none for a clean base map, or compare any two editions directly — including backwards.
Building shapes group subsystems: towers are always-on coordinators, halls are stores, plants process bulk bytes, labs are young machinery, gates are entry/negotiation points, yards hold formats. Footprint and height scale with structural weight, not code size. Dashed black routes carry control (plans, commands, RPCs); solid brown routes carry data (blocks, batches, state), with beads drifting along them. Every claim in a survey report cites a repository path at a pinned tag; headline claims sourced from release notes instead of source are flagged RN.
Why this sheet exists here: the 4.1 edition is where GEOMETRY/GEOGRAPHY become native Spark types with st_* expressions — gated behind spark.sql.geospatial.enabled until 4.2 flips it on by default — landing this workbook’s spatial-pushdown track inside the engine itself.
Editions
Revision ledger
| Edition | Anchor | Build | Survey revisions (source-pinned) | Release headlines RN |
|---|
Sources & method
Base survey: github.com/apache/spark, read per anchor with git ls-tree (module/file presence) and git show (defaults, annotations) — no claims from memory. Presence of every building was checked at all five anchors. Release-note items (RN) come from the official announcements: 4.0.0, 4.1.0, 4.2.0, dated by the news archive. The purple-overprint convention follows USGS photorevised topographic quadrangles.