Boston Agencies Race to Purge Duplicate Images From Public Records as New State Guidance Takes Hold

Boston's municipal agencies and several major research institutions along the Longwood Medical Area corridor are working through a backlog of duplicate digital images in their public-facing databases, following updated guidance issued by the Massachusetts Office of Digital Services in late June. The directive, which took effect July 1, sets new standards for how state-funded entities must manage image metadata and remove redundant files from publicly accessible records systems.

The timing matters. The city has spent the past three years digitizing tens of thousands of documents — everything from Jamaica Plain neighborhood planning submissions to MBTA accessibility audits — and duplication crept in at nearly every stage. When the same photograph of a Dorchester streetscape appears under four different file names in a planning database, it slows search tools, inflates storage costs, and creates legal uncertainty about which version is the official record. The new state guidance is the first formal attempt to address that systematically.

Who Is Affected and What Has to Change

The directive covers any agency or institution that receives state digital-infrastructure funding. That pulls in Boston City Hall, the Boston Public Library's digital collections on Copley Square, the MBTA's public communications archive, and university-adjacent bodies like the Boston Redevelopment Authority's successor, the Boston Planning Department. Several Fenway-area biotech firms that license data to state health programs also fall under the rules because of grant funding arrangements.

At the Boston Public Library, staff in the Norman B. Leventhal Map & Education Center began an internal audit in early June anticipating the guidance. The center holds more than 200,000 digitized map images, and librarians identified a significant subset that had been scanned twice — once from an older digitization project completed around 2019 and again during a 2023 refresh that used higher-resolution equipment. Under the new rules, the library must designate a canonical file for each asset and retire the duplicate within 90 days, attaching a deprecation notice so researchers know which version to cite.

The Boston Planning Department faces a different kind of problem. Neighborhood rezoning hearings in Dorchester and Roxbury generated thousands of site photographs submitted by community members and project architects between 2022 and 2025. Those images landed in at least three separate folder structures across two content-management systems, and duplicate detection software flagged roughly 14,000 files as probable matches, according to figures shared at a June 25 city technology briefing. Staff now have until September 30 to resolve conflicts, prioritizing images tied to active permit applications on streets like Blue Hill Avenue and Talbot Avenue where construction timelines are tight.

What Comes Next for Researchers and the Public

The practical stakes for ordinary Bostonians are modest but real. Anyone searching the city's Inspectional Services Department portal for property photographs — a common first step for buyers in neighborhoods like East Boston and Mattapan — has occasionally pulled up mismatched or outdated images. Clearing duplicate records should reduce that friction, though the cleanup work itself may temporarily interrupt search results on the ISD portal through mid-August while the indexing is rebuilt.

For the city's biotech and university sector, the implications are sharper. Research institutions along the Longwood corridor that share imaging data with state public health programs must now implement hash-based duplicate detection — essentially a fingerprinting system for image files — as a condition of continued data-sharing agreements. Several hospitals connected to Harvard Medical School were already using similar internal tools, but the state guidance standardizes the process across partners for the first time.

Technology staff at affected agencies say the September 30 deadline is achievable for most, though the Dorchester and Roxbury planning files represent the largest single workload. The Massachusetts Office of Digital Services plans a compliance review in October. Agencies that miss the deadline face a suspension of new digital-infrastructure grant approvals — a sanction with real bite in a city where the next round of MBTA accessibility digitization funding is expected to open for applications in November.