A few months ago, researchers from the MIT Senseable City Lab published a study about pollution in New York City featuring some new methods. For instance: With machine learning, they identified the types of vehicles appearing in 331 traffic cameras in the city, and estimated the emissions coming from each automobile. Given enough cameras, these visual artificial intelligence techniques could monitor emissions with an unprecedented combination of precision and scale.
For that matter, visual AI today can address all kinds of questions for urban planners. Why exactly is traffic snarling? What are the most dangerous aspects of different intersections? Which parts of plazas or parks attract the most people?
Across cities, more images means more data, more insight — and more concerns about privacy and fairness.
“We can treat these digital images as data and quantify features of the city,” says Fábio Duarte, an MIT researcher and co-author of a new book about visual AI and urban studies. “With computer vision techniques, each image is a dataset.” Still, he adds, “We have to be careful about it.”
And while urbanists have long used visual analysis to inform their thinking, now it’s possible to an unprecedented extent.
“Everybody has been observing the urban environment and trying to get some insight,” says Martina Mazzarello, an MIT scholar and a co-author of the new book. “But what if we can do that at a large scale and get some insight everywhere?”
The scholars explore these topics in “How AI Sees the City: Urban Visual Intelligence,” published this month by Routledge. The authors are Duarte, a principal research scientist and associate director of the MIT Senseable City Lab; Mazzarello, a research scientist and lead of MIT Senseable City Lab global initiatives; Carlo Ratti, a professor of the practice and founder and director of the MIT Senseable City Lab; and Fan Zhang, an assistant professor at the Institute of Remote Sensing and GIS at Peking University.
“Great urbanists such as Kevin Lynch and Willian H. Whyte showed us the extraordinary value of ‘looking’ at the city,” Ratti says, referring to two prominent thinkers about city dynamics whose work is described in the book. “Today, visual AI gives us new ways to build on that tradition — allowing us to observe cities at a scale and with a level of detail that was previously impossible.”
New tool, long tradition
“How AI Sees the City” stems from the work of the MIT Senseable City Lab, founded in 2004, which uses data to better understand urban dynamics. As the authors discuss in the book, there is a long history of visual representations that shape the way we think about cities, from Romans building marble maps to the introduction of photography — which produced influential urban images about things like Hausmann’s reshaping of Paris or the crowding of tenements in New York City’s Lower East Side during the 19th century.
More recently, some scholars have used visual studies to better understand city form, including Lynch, a former MIT professor whose 1960 book, “The Image of the City,” influenced many scholars. Whyte, a sociologist famous for his book “The Organization Man,” then became an urbanist closely examining public spaces.
By explicitly placing AI in a continuum with these visual urban studies, the authors are making a point: Powerful as it might be, we can still think of AI primarily as a tool serving human purposes, as we seek to design and refine urban form.
“Kevin Lynch at MIT was only using paper and pen,” Duarte says. “We can now scale up what he was doing, with visual AI, while also looking at many different dimension of cities.”
There are extensive possibilities for applying visual AI to urban planning, ranging from emissions to traffic flow, safety, better imagery of street-level activity and sidewalks, and much more. The book also examines, for instance, urban greenery. While satellite imagery can show us how much tree cover and green spaces cities have, near-ubiquitous images from phones and other sources can also reveal to what extent people glimpse greenery in everyday life, a factor in reported wellness.
“The real promise of visual AI is not simply that computers can look at millions of images,” Zhang says. “It is that we can connect what is visible in those images — streets, buildings, greenery, traffic, public space — with larger questions about how cities function and how people experience them.”
Better image recognition by AI even extends to urban interiors. By using images from 400,000 AirBnB listings across the world, one recent Senseable City study shows that, contrary to some claims, interior design styles are not becoming globally more homogeneous, but reflect significant geographic differences.
“No matter what it is, we can learn from what we can see and then use it as urban designers, planners, policymakers, and citizens,” Mazzarello says. “It can be our eyes, or cameras with computers, but in the end it’s the same methodology, and now we are trying to optimize the ways we can use these tools.”
Promise and pitfalls
If the promise of visual AI for urban studies is vast, the pitfalls are concerning. In “How AI Sees the City,” the authors outline multiple potential problems with the technology, including the intrusiveness of widespread visual surveillance and the potential for bias being reinforced through AI systems.
The installation of ubiquitous cameras can quickly raise concerns about surveillance. London, an early adopter of CCTV, has about 210 cameras per square mile. But eight of the world’s 10 most camera-heavy cities are in China; Shanghai has over 5,000 cameras per square mile. Such surveillance practices have raised controversy in other parts of the world, with debate over the uses of traffic cameras bubbling up in the U.S. this year as well.
In evaluating the potential safety gains from intensive video recording, the authors write, “the benefits must be weighed against the significant erosion of personal freedom and the potential for abuse inherent in a system of constant monitoring.”
Meanwhile, AI systems can reinforce social biases as well, leading to the production of data that reinforce prior perceptions as much as underlying realities — about people, neighborhoods, and whole cities. If AI models are trained on majority population groups, they may not evaluate minority groups the same way.
“We need to teach AI to see, and depending on how you teach it, it will see what what is embedded in the culture,” Duarte says. “AI is not neutral.”
Still, as Mazzarello adds, “our eyes are not neutral, either. Every tool has to be guided in the right way, and trained in the best way.”
Other scholars have praised “How AI Sees the City.” Michael Batty of University College London has called it a “fascinating book” that “shows how we are beginning to interpret the world of urban design, suggesting ways in which we might improve design using urban analytics, AI and large language models.”
Ultimately, though the authors think there is great value in deploying visual AI to learn more about our cities, how they function, and how they might be improved. With caution and independent thinking, progress is possible. Or, as they conclude in the book, “We should explore this wisely, critically, and creatively.”
