Boston City Hall's Digital Archive Push Hits Snag Over Thousands of Duplicate Images This Week

Boston's Office of Digital Innovation and Technology confirmed this week that a systematic audit of the city's public-facing document archive uncovered tens of thousands of duplicate image files embedded across municipal records dating back to the early 2000s. The discovery has temporarily slowed a project that city administrators had hoped to complete before the fiscal year closed on June 30.

The timing matters. Mayor Michelle Wu's administration has staked part of its open-government platform on making city records fully searchable and accessible online — a priority that gained urgency after years of criticism that Boston's permitting and zoning records were difficult for residents in neighborhoods like Jamaica Plain and Dorchester to navigate without physically visiting City Hall on Cambridge Street. Redundant image files, officials acknowledged, are not a cosmetic problem: they inflate storage costs, break search indexing, and in some cases surface the wrong version of a document when residents query the system.

Where the Problem Surfaced

Archivists working under contract with the Boston Public Library's Norman B. Leventhal Map and Education Center flagged the duplication issue during a routine quality-control pass in late June. The Leventhal Center has been a key institutional partner in the digitization effort, particularly for historical records. Staff identified that a batch-upload process used between 2021 and 2024 had failed to check for existing files before adding new ones, meaning some property documents and zoning maps were copied into the system anywhere from two to eleven times.

The Inspectional Services Department, whose records are among the most frequently accessed by homeowners and developers in neighborhoods like South Boston and Roxbury, is believed to hold the largest share of affected files. The department processes thousands of building permits and code-violation records each year, and archivists say the duplication is concentrated in scanned PDFs that were converted to JPEG image files during an earlier, now-discontinued workflow.

Massachusetts General Laws Chapter 66 requires municipalities to retain public records for specified periods, and any deletion — even of a confirmed duplicate — must follow a formal disposition schedule approved by the Secretary of State's office. That legal requirement means city IT staff cannot simply run an automated script to purge redundant files. Each flagged image must be verified against the original before removal, a process that archivists estimate will take until at least September 2026 to complete across all affected departments.

What Comes Next for Residents and Developers

The city has not taken the archive offline. Records remain accessible through Accela, the permitting platform Boston uses, and through the separate Boston.gov document portal. However, users searching for property records in ZIP codes 02130 and 02121 — covering Jamaica Plain and Dorchester respectively — may encounter duplicate search results that return the same image file under different record numbers. The Office of Digital Innovation and Technology is advising anyone who finds conflicting documents to contact the relevant department directly rather than assuming the first result returned is the authoritative version.

The broader digitization project, which received a $2.3 million allocation in the city's fiscal year 2025 capital budget, has digitized roughly 1.4 million pages of municipal records since its formal launch in January 2023, according to the project's public progress dashboard. The duplicate-image problem does not affect those totals — the scanned pages themselves are intact — but it does complicate the indexing work that makes those pages retrievable in a useful way.

Boston's experience echoes challenges faced by other large American cities that moved quickly to digitize legacy paper records without adequate deduplication protocols built into the pipeline from the start. The Massachusetts State Archives in Dorchester, which manages its own separate digitization program, moved to a content-addressed storage system in 2022 specifically to prevent this category of error, and city IT officials say they are now evaluating whether to adopt a similar approach for the municipal archive going forward.

Residents with urgent permit or property-record needs are encouraged to use the Inspectional Services Department's in-person service counter at 1010 Massachusetts Avenue or to file a public records request through the city's online portal, where staff can pull source documents directly from the verified master file regardless of the database duplication.