Boston Agencies Grapple With Duplicate Image Problem as Officials Warn of Risks to Public Records

Boston's city government is confronting a sprawling and largely invisible problem: thousands of duplicate and mislabeled digital images embedded in public-facing systems, permit portals, and archival databases — and the officials responsible for cleaning them up say the task is more urgent than most residents realize.

The issue has moved from a back-office nuisance to a genuine administrative headache across multiple departments, touching everything from the Boston Planning Department's development review files to the MBTA's station-documentation archives. With the city accelerating housing production in Jamaica Plain and Dorchester under Mayor Michelle Wu's housing agenda, the volume of project imagery flowing into municipal systems has surged, and duplicate records are multiplying faster than staff can flag them.

Why the Problem Is Getting Worse

The volume surge is not accidental. Since the Wu administration launched its housing production push — which set a target of 69,000 new units by 2030 — planning staff have been processing permit applications, environmental filings, and community impact reports at a rate that strains legacy document management infrastructure. Each application can carry dozens of attached photographs. When applicants resubmit after revisions, duplicate image files often enter the system under different file names but identical content, making automated deduplication tools unreliable without manual review.

Boston's Inspectional Services Department, headquartered on City Hall Plaza, has been working with the Office of Digital Transformation to address the backlog, though the scope of the problem across all city systems has not been publicly quantified. Digital records specialists at institutions including Northeastern University's library system, which manages its own large-scale archival imaging projects along Huntington Avenue, say the municipal challenge mirrors what private institutions face but is complicated by public records law obligations. Under Massachusetts General Law Chapter 66, city agencies are required to maintain complete and accurate public records — a standard that duplicated or misidentified images can compromise.

Boston Public Library's Digital Commonwealth program, which digitizes historical collections from institutions across the state, has dealt with similar deduplication challenges for years. Archivists there have described the process of reconciling duplicate scans as one of the most labor-intensive aspects of large-scale digitization — a lesson that city planners are now absorbing in real time.

What Experts and Administrators Are Recommending

Records management specialists advising city agencies broadly recommend three steps: adopting hash-based deduplication software that identifies identical files regardless of their name; establishing naming conventions at the point of upload rather than after the fact; and conducting periodic audits tied to existing budget cycles rather than waiting for problems to accumulate.

The MBTA, which is separately managing its own digital infrastructure overhaul as part of the Federal Transit Administration's ongoing oversight agreement — an agreement that has been in place since 2022 — has begun piloting image-management protocols at select stations including Back Bay and Forest Hills. The authority has not publicly disclosed the cost of that pilot program or its timeline for system-wide rollout.

Community organizations in Dorchester and Jamaica Plain that interact with city permitting systems have noted practical downstream effects. Neighborhood councils reviewing development proposals sometimes receive packets containing duplicate site photographs, which can slow the already lengthy community review process and create confusion about whether updated images reflect current site conditions.

City technology staff have pointed to fiscal year 2027 — which begins October 1 — as the earliest point at which dedicated funding for a comprehensive image audit could be incorporated into the departmental budget. That timeline gives agencies roughly three months to build a formal proposal.

For residents and civic groups dealing with permit applications or public records requests in the meantime, records management advisers suggest requesting that submitting parties confirm image uniqueness in writing as part of the application checklist — a low-cost procedural fix that some neighborhood groups in Jamaica Plain have already begun requiring for documents submitted to community review boards. It does not solve the systemic problem, but it slows the accumulation of new duplicates while the city works toward a longer-term technical fix.