Boston Archives and City Agencies Tackle Duplicate Image Problem Across Digital Records This Week

Boston's municipal records offices and several of the city's major universities have accelerated efforts this week to address a persistent but under-discussed problem: thousands of duplicate and mismatched images clogging digital archives, slowing public-records requests, and creating legal exposure around copyright and consent. The push, which involves the City of Boston's Department of Innovation and Technology as well as digital asset teams at institutions along the Longwood Medical Area corridor, comes as a July 1 federal guidance update clarified obligations for publicly funded bodies to audit digitized records for redundant content.

The timing matters. Massachusetts has seen a significant expansion of digital record-keeping requirements since the passage of the state's 2023 Public Records Modernization Act, which set a 2026 compliance deadline for agencies managing more than 500,000 digital assets. For a city like Boston — which digitized roughly 1.2 million historical photographs and planning documents between 2019 and 2024 through the Boston City Archives on School Street — the accumulation of duplicates was essentially inevitable. Scanning batches at different resolutions, by different vendors, across different fiscal years, tends to produce exactly this kind of redundancy.

This week, staff at the Boston Public Library's Digital Repository Services unit on Boylston Street confirmed they are running automated deduplication software across the BPL's online image collections, which include the Leslie Jones photography archive and decades of Boston Globe prints transferred under a 2018 preservation agreement. The library has not publicly disclosed how many duplicate entries have been flagged, but the process is expected to take through late August. Separately, Northeastern University's library system, operating out of Snell Library on the main Huntington Avenue campus, launched a parallel review of its digital special collections on July 2.

Why Duplicates Are More Than a Housekeeping Problem

The practical stakes extend well beyond tidy file folders. Duplicate images in government planning records — say, two near-identical aerial photographs of Dorchester's Neponset River corridor filed under different parcel numbers — can complicate zoning reviews and slow approval timelines for housing projects. The Wu administration has made accelerating housing production in neighborhoods including Jamaica Plain and Dorchester a central policy priority, and planning staff have flagged digital-records bottlenecks as one friction point in the permitting pipeline. The Boston Planning Department manages an image database tied directly to its Article 80 development review process, and redundant entries require manual reconciliation before they can be formally archived.

Copyright is the other pressure point. When duplicate images carry conflicting metadata — one copy tagged as public domain, another flagged as rights-restricted — institutions face real legal ambiguity about what they can publish online. The U.S. Copyright Office issued a technical bulletin in March 2026 reminding federally funded archives that inconsistent metadata on duplicate records does not provide a safe-harbor defense. That bulletin has been circulating among Boston-area digital asset managers since spring, and several described it, in general terms at public forums, as a catalyst for moving deduplication projects from the backlog to the active queue.

What Comes Next for Local Institutions

For residents and researchers, the near-term effect is likely to be temporary gaps in online collections. The BPL has advised users that portions of the Leslie Jones archive may be intermittently unavailable through the end of summer as records are reviewed and re-ingested. The library's Digital Repository team expects to publish an updated catalog with corrected metadata by September 15.

The City Archives on School Street is taking a phased approach. Staff are prioritizing records tied to active permit reviews first, then moving to historical photograph collections. The department is using open-source image-fingerprinting tools rather than commercial software, a decision driven partly by budget constraints — the city's FY2026 technology budget allocated roughly $2.4 million to digital records infrastructure, a figure that leaves limited room for enterprise licensing.

For anyone who has submitted a public-records request involving photographs or scanned documents in recent weeks and received a delayed response, the city's Public Records Office on City Hall Plaza is the point of contact. Requesters can ask specifically whether a delay is tied to the deduplication review, and staff are required under state law to provide a written status update within ten business days of any such inquiry.