Boston Leads on Purging Duplicate Images From City Records — But Other Global Cities Are Catching Up Fast

Boston's city government has been quietly wrestling with a problem that sounds mundane until you see the price tag: tens of thousands of duplicate images cluttering the municipal digital infrastructure, from permit databases at City Hall on Cambridge Street to public-facing property records tied to the Assessor's Office in Roxbury. The effort to identify, flag, and replace those redundant files — a process archivists and IT administrators call duplicate image replacement — has accelerated under the Wu administration's broader push to modernize city services, but officials have yet to declare the job anywhere close to finished.

The issue matters now for a specific reason. Boston, like dozens of cities globally, has been migrating legacy paper and analog records into unified digital platforms over the past four years. That migration, accelerated by federal broadband and digital equity funding tied to the 2021 Infrastructure Investment and Jobs Act, flooded municipal servers with redundant scans. Every duplicate image that sits unresolved consumes server storage, slows retrieval times for city workers, and — critically — introduces errors when automated systems mistake an outdated image for a current one.

What Boston Is Actually Doing

The city's Department of Innovation and Technology, based in City Hall Plaza, has been running a deduplication protocol since late 2024 using open-source fingerprinting software to match visually identical or near-identical files across departments. The effort is concentrated in two areas: the Building and Structures Permitting database, which covers active construction zones including the dense redevelopment corridors along Washington Street in Jamaica Plain, and the property record image libraries maintained by the Boston Assessing Department for Dorchester parcels. Early internal reviews found duplication rates running as high as 30 percent in some permit image folders, according to a city IT presentation circulated at a January 2025 public meeting of the Mayor's Office of New Urban Mechanics.

Boston Public Library's Digital Repository Services team, operating out of the Central Library on Boylston Street in Copley Square, has run a parallel effort for its digitized historical collections. Librarians there began systematic deduplication of the Leslie Jones photograph archive in March 2025, working through more than 40,000 scanned glass plate negatives where multiple scans of the same image were catalogued under different accession numbers.

How Boston Compares to London, Amsterdam, and Toronto

Cities that have confronted this challenge longer offer useful benchmarks. Amsterdam's Stadsarchief — the city's municipal archive — completed a large-scale deduplication project across its 750,000-image urban planning database in 2023, reducing stored file volume by 22 percent over 18 months, according to the archive's published annual report. London's Government Digital Service has embedded automated image-hash checking directly into its upload pipelines since 2022, meaning duplicates are flagged before they enter the system rather than after. Toronto's Open Data team published a deduplication methodology report in February 2025 that identified its planning department's image repositories as the single largest source of redundancy, a finding that mirrors what Boston's own internal review turned up.

Boston's approach differs in one meaningful way: the city has chosen to replace flagged duplicates with a canonical master image rather than simply deleting the extras. That distinction matters for public records law. Under Massachusetts General Laws Chapter 66, municipalities must retain accessible copies of certain public documents, and deletion without substitution can trigger compliance questions. The replacement model is slower and more resource-intensive, but it keeps the city on firmer legal ground.

The deduplication work also intersects with MBTA-adjacent projects. The T's Capital Program Office, which maintains visual documentation for infrastructure inspections on the Green Line Extension corridor, has been coordinating with the city's DoIT team to align image management standards — a collaboration that began formally in September 2025.

For residents, the practical upshot is simpler than it sounds. Property owners filing renovation permits through Boston's Inspectional Services Department portal on Inspectional Services Drive in Roxbury should expect faster document processing by the end of this calendar year, as the backlog of mismatched and duplicated images in the permit queue clears. The city has set an internal target of reducing duplicate image volume across core databases by 40 percent before December 31, 2026 — a goal that, given Amsterdam's 22-percent result over a comparable timeline, is ambitious but not obviously out of reach.