City agencies and universities are sitting on thousands of redundant digital files — and how Boston handles the cleanup will shape public access to records for years.
Boston's network of city agencies, public libraries, and research universities is confronting a mounting backlog of duplicate digital images stored across incompatible systems, a problem that has quietly consumed storage budgets and complicated public records requests for at least three years. The immediate question is no longer whether to act — it's who decides what gets deleted, who keeps the authoritative copy, and what standards govern the whole process.
The issue has sharpened in 2026 because several major digitization contracts are expiring simultaneously. The Boston Public Library's Copley Square branch concluded a multi-year imaging project for its historic photograph collection in early spring. The City of Boston's Archives office, based on West Roxbury Parkway, is mid-cycle on a separate initiative covering building permits and zoning records dating to the 1940s. Both projects have generated substantial overlap, with some images appearing in three or more repositories under different filenames and metadata tags.
Duplicate files are not merely a storage nuisance. When a journalist, a resident in Dorchester, or a developer pulling permits in Jamaica Plain submits a public records request, staff must verify which version of an image is the authoritative one. That verification step adds time and cost to every fulfillment. Massachusetts public records law requires responses within ten business days, a deadline that agencies have struggled to meet consistently when file provenance is unclear.
The Massachusetts Secretary of State's office sets statewide retention schedules, but granular decisions about which digital asset management platform to use, which metadata schema to adopt, and which department holds the master copy all fall to individual agencies. That decentralization is exactly the problem advocates for open government have been raising since at least 2023.
Northeastern University's library system on Huntington Avenue and the Harvard Divinity School's archives in Cambridge have both navigated similar rationalization exercises in recent years, providing rough templates. Northeastern completed a deduplication audit of its digital collections in late 2024 that reduced redundant files by roughly 40 percent, according to information the university published in its annual library report. That figure is widely cited in conversations among local archivists as a benchmark for what municipal projects might realistically achieve.
Three choices are bearing down on city officials and library administrators before the fiscal year closes on September 30. First, Boston's Department of Innovation and Technology must decide whether to extend its current contract with a cloud storage vendor or issue a new RFP that requires interoperability with the BPL's existing systems. Second, the Archives office needs a written protocol establishing which agency holds the master record when two departments have independently digitized the same physical document — currently there is no such protocol. Third, the city must determine whether residents and researchers will get a single public-facing search portal or continue navigating siloed databases.
Mayor Michelle Wu's administration has signaled interest in consolidating city digital infrastructure as part of a broader technology modernization push, though no formal policy specific to image archives has been announced publicly. The relevant budget lines sit inside the Department of Innovation and Technology's capital plan, which the City Council will review before the summer recess.
Community groups in Jamaica Plain and Dorchester have a practical stake in the outcome. Both neighborhoods are the focus of active housing production initiatives, and historical building records stored as digital images are frequently requested by developers, community land trusts, and neighbors contesting zoning variances. Delays in records fulfillment have real costs — attorneys' fees, project timelines, and public participation windows that close before documents arrive.
Archivists and technology officers are expected to present recommendations to city leadership this fall. The window between now and September is when advocates, neighborhood organizations, and researchers should weigh in — through public comment periods, City Council testimony, or direct correspondence with the Archives office on West Roxbury Parkway. How Boston resolves the duplicate image question will set the baseline for digital records management well into the next decade.
How does this story make you feel?
Spread the word
About this article
Published by The Daily Boston
Daily brief
Free, in your inbox before 7am. Weekdays.
More in News
