Community Partners in Residence (CPR) Program – Meeting 6

On Tuesday, February 20, 2024, the NSF Center for Smart StreetScapes (CS3) held its sixth Community Partners in Residence (CPR) Program workshop. The session was convened and led again by Ester Fuchs, Chief Social Impact Officer of the CS3 and Professor of International and Public Affairs and Political Science at Columbia University. The focus of this week’s workshop was on the subjects of public safety, emergency response, and individual privacy as it pertains to the deployment of streetscape technology.

After opening remarks by Ester Fuchs, Mubarak Shah, Chair Professor in University of Central Florida, Director at the Center for Research in Computer Vision, Situational Awareness Research Lead, CS3, presented a summary of his research on the use of “computer vision situational awareness” in order to interpret and identify actions that occur in an image or a video. He discussed that the aim of classifying human actions can take several forms such as activity recognition, that is, classification of human actions and their application in the real world to situations, and anomaly detection, recognition when someone is behaving abnormally or an accident occurs. However, he acknowledged that along with development of this technology are the twin drawbacks of privacy and bias; it’s important to protect the privacy of those being captured on camera as the majority of the time their actions are benign, while the accuracy of an activity recognition program suffer from bias in its ability to identify certain actions based on the traits of the subjects in the video. Specifically, bias is caused largely by the system learning false associations between subject attributes and behavior and could be combated by using more materials to better inform the program. Mubarak highlighted that the future direction of the team is to continue to work to resolve the issues of privacy and bias.

Following Professor Mubarak’s presentation, CPRs posed questions centered around the potential of computer vision situational awareness for public safety, such as “What if we don’t want the identity of shoplifters to remain anonymous?” and “How does this converge with the law and help to combat recidivism?” He clarified that if there should be something where the police wanted the video, it can be revealed, ideally while the privacy of people who are not doing anything illegal is preserved. In response to concerns about malicious training data, he explained that the idea of this research is to make systems more robust against these kinds of attacks by producing a system that looks at the activity of the individual and not the person’s physical characteristics.

The second presentation was delivered by Andrew Smyth, Professor of Civil Engineering and Mechanics at Columbia University, Principal Investigator & Chair of the CS3, offering a use case from his prior work where additional camera technology and machine learning can be used to improve city services beyond the current capabilities. Initiated in 2021 as response times became increasingly slower, his project aimed to optimize the FDNY EMS response operations using historical data from the 1.1 million hospital transport emergencies that occur per year. The typical decision for the dispatch is to send the patient to the nearest hospital based on location and issue as identified through “critical care codes” using a static computerized map, while the location choices do not take into account traffic patterns or times of day/week. He demonstrated that new routing patterns developed using data analytics demonstrate a crucial one minute improvement to routing times, though they are not currently being used by the city. Without having access to the Google Maps data, having more accurate/real-time data through cameras would go a long way to advancing response times, which ideally could operate streetlights or change traffic patterns to assist. Andrew concluded his presentation by introducing his second aspect of the project focusing on pandemic hospital load balancing where streetscapes camera data can be used to route patients to alternative hospitals when the nearest option is at-capacity.

CPRs reacted to Andrew’s presentation with questions such as “How does level of trauma impact response times?” and “How do street alterations such as planters, bike lanes, etc. impact response times?“ He clarified that the ambulance will take different approaches to how they move on the street based on the level of care needed and that the more data there is the more it can accommodate changes to the street, and we can collect some forms of data in real-time from Streetscape-level cameras.

The meeting then moved into breakout sessions, where CPRs shared ideas, feelings and questions about public safety, emergency response, and individual privacy. As for new learning from today’s presentations, many showed their awareness of bias issues around computer vision applications and concerns about privacy when collecting data. Moving to the discussion to identify practical problems on the streetscapes that you think might be addressed through this kind of research, there were many suggestions using real-time data from CPRs such as finding alternative routes for emergency vehicles and during festivals, the most appropriate hospitals, putting those ideas in application for consumer use.

CPR workshops are invite-only. If you are interested in attending a workshop or learning more about CS3’s community engagement process, please contact our team at streetscapes@columbia.edu.