Boston Is Quietly Building One of the Country's Better Systems for Flagging Duplicate Street Images — But Other Cities Are Pulling Ahead

Boston's city mapping office processed roughly 14,000 duplicate-image removal requests from its public-facing street-level database in the 12 months ending this past March, according to figures the city's Department of Innovation and Technology posted to its open-data portal in April. That number sounds substantial. Against a total archive that municipal staff estimate at more than 2.4 million georeferenced images — many captured during successive MBTA infrastructure surveys and neighborhood condition assessments — it barely moves the needle.

The issue is not trivial. Duplicate or near-duplicate street photography clogs GIS workflows, inflates storage costs, and — critically — can expose identifiable faces, license plates, and private property details multiple times over in public-facing civic tools. With the city's ambitious Jamaica Plain and Dorchester housing production push generating new construction photography at scale, the backlog is only growing.

What Boston Is Actually Doing

The city's primary deduplication effort runs through a partnership between the Department of Innovation and Technology and Northeastern University's Civic AI Lab on Huntington Avenue, which has been piloting a perceptual-hashing algorithm since January 2025 to flag near-identical frames automatically before they enter the public archive. The program, funded through a $340,000 federal Smart Cities grant awarded in the fall of 2024, is designed to reduce manual review time by flagging duplicates for a human analyst rather than deleting them outright — a deliberate choice given ongoing debates about data retention under Massachusetts public records law.

The Northeastern partnership covers imagery captured along roughly 40 priority corridors, including Washington Street in Dorchester and Centre Street in Jamaica Plain, where the Wu administration has concentrated its housing and transit-oriented development work. Everything outside those corridors still relies on staff manually reviewing flags submitted through the city's existing 311 portal, which was never designed for this purpose and shows it.

Boston's approach is conservative by design. City staff have pointed to legal caution around deletion as a reason for the slow pace. Under the Massachusetts Public Records Law, Chapter 66 of the General Laws, municipalities must follow approved retention schedules before destroying any document, and street photography captured during official surveys can qualify as a public record. That creates a bottleneck that simpler deletion pipelines in other jurisdictions don't face.

How Other Cities Compare

London's Geospatial Commission, working alongside Transport for London, deployed an automated deduplication layer across its Streetscape Intelligence database in mid-2024 and has since reported reducing redundant imagery by about 38 percent in borough-level datasets, according to a progress report the commission published in February 2026. The system uses a combination of perceptual hashing and GPS-proximity filtering and operates on a continuous ingestion model rather than batch processing.

Seoul's Smart City Division, under the city's Digital Master Plan adopted in 2023, went further — embedding deduplication directly into the cameras mounted on municipal survey vehicles, so duplicates are rejected at the point of capture rather than cleaned up afterward. The capital investment was substantial, reported at roughly 4.2 billion Korean won (approximately $3 million USD) for the first phase, but city officials there have argued the upstream savings in storage and staff time justify the cost within a five-year window.

Chicago, a more direct peer for Boston in terms of civic tech budget and political structure, launched its own image-deduplication initiative through the Department of Assets, Information and Services in March 2025. That program, tied to the city's INVEST South/West corridor photography initiative, has processed an estimated 800,000 images since launch — a faster clip than Boston's, though Chicago benefits from a larger dedicated GIS staff.

Boston's advantage, if there is one, is the Northeastern partnership's academic rigor. The perceptual-hashing model being tested on Washington Street and Centre Street is being documented and peer-reviewed in a way that London's and Seoul's proprietary systems are not, which may make it more adaptable and auditable over time.

The practical question for residents and city planners is what happens when the federal grant runs out. The $340,000 award covers the Northeastern pilot through June 2027. Without a successor funding mechanism — either a line item in the city's capital budget or a second grant cycle — the 40-corridor pilot is likely to stall, leaving the remaining 2-plus million images in the archive with no automated review at all. The city's 2027 budget process begins in earnest this coming fall, and advocates for expanded civic data infrastructure say that is when the decision will effectively be made.