Boston City Agencies Race to Purge Duplicate Images From Digital Records This Week

Boston's Department of Innovation and Technology launched a targeted sweep this week to remove duplicate images buried inside the city's shared digital asset management system, a problem that officials say has slowed permit processing, inflated storage costs, and complicated the rollout of Mayor Michelle Wu's open-data initiative. The sweep, which began Monday, July 1, is targeting redundant files accumulated across at least eleven municipal departments since the city migrated to its current cloud platform in 2022.

The timing matters. The Wu administration set a July 31 internal deadline for all departments to certify clean digital inventories as part of the Boston Digital Equity and Open Data Framework, a program tied to the mayor's broader technology modernization agenda. Duplicate image files — often the byproduct of staff uploading the same inspection photo or property scan multiple times — have created version-control headaches that slow everything from building inspections in Dorchester to zoning reviews on Washington Street in Jamaica Plain.

Where the Problem Showed Up

The Boston Inspectional Services Department, headquartered on City Hall Plaza, identified the largest cache of duplicates: internal auditors found that roughly 30 percent of image files in the department's property records folder had at least one exact copy stored elsewhere in the system. That figure comes from a preliminary internal review shared with department heads at a June 25 briefing, according to a summary posted to the city's public procurement portal. The duplication problem is not unique to Boston — municipalities across the Northeast that adopted rapid cloud migrations during the pandemic encountered similar issues — but the scale here caught managers off guard.

The MBTA's Customer Technology office flagged a related issue in May: duplicate wayfinding and accessibility images uploaded by contractors during the Blue Line station refresh project had inflated the agency's managed-file repository by an estimated 18 gigabytes, adding unnecessary licensing costs on a per-seat storage contract. The T has its own separate remediation underway, coordinated partly with the state Executive Office of Technology Services and Security on Causeway Street.

Meanwhile, Northeastern University's Burnes Center for Social Change, which has been partnering with the city on neighborhood-level data mapping in Roxbury and Nubian Square, flagged the duplication issue as a practical barrier to publishing clean datasets. When city image libraries feed into public-facing mapping tools, duplicate files generate broken or redundant pins that erode public trust in the data.

What the Cleanup Involves — and What Comes Next

The Department of Innovation and Technology is running automated deduplication scripts across roughly 2.4 terabytes of image data this week, using hash-matching software to flag identical files before flagging them for human review. Staff at the city's 1 City Hall Square offices are manually confirming deletions for any file attached to an active permit or legal record, a precaution required under the city's records-retention policy.

Agencies have until July 18 to submit remediation reports to DoIT, giving the central office two weeks before the July 31 certification deadline to consolidate findings. Departments that miss the window risk being flagged in the fall budget review cycle, which begins in September.

For residents and businesses that regularly interact with city permitting — contractors pulling permits for triple-deckers in Dorchester, small business owners filing signage applications along Hyde Park Avenue — the practical upside is faster upload processing and fewer instances of documents getting lost inside cluttered digital folders. The city estimates the cleanup could free up enough storage to defer a planned infrastructure expansion that was budgeted at roughly $400,000 in fiscal year 2027.

DoIT is also expected to release updated image-upload guidelines for all departments by August 1, establishing file-naming conventions and size limits intended to prevent the problem from recurring. Whether those standards get adopted consistently across a city government with dozens of semi-autonomous agencies is the harder question — one the fall audit cycle will likely answer.