Professors Zoran Kostic, Gil Zussman, Javad Ghaderi, and their students share their latest findings at the Computer Vision and Pattern Recognition Conference (CVPR).
By Xintian Tina Wang, Columbia Electrical Engineering
July 09, 2024
Imagine a “smart city” of the future where intelligent machines like autonomous vehicles and robots navigate with assistance by street infrastructure. This scenario would require that autonomous systems integrate into our human-populated world, where they would have to participate in social interaction-aware motion planning, and be able to perceive and respond to human actions. Real-world training of these machines and supporting computing models is fraught with difficulties, from capturing the myriad ways humans behave to ensuring safety during interactions.
At the 2024 Computer Vision and Pattern Recognition Conference (CVPR) held in Seattle, the researchers and students from Professor Zoran Kostic’s lab in Columbia University’s Electrical Engineering Department (EE) presented their research progress on tackling the challenges of smart cities. The conference is a premier event sponsored by the IEEE Computer Society and the Computer Vision Foundation, attracting over 10,000 scientists and engineers from around the world and featuring a comprehensive technical program that includes papers, tutorials, workshops, and an expo showcasing cutting-edge technologies and innovations.
In the workshop, “Populating Empty Cities: Virtual Humans for Robotics and Autonomous Driving,” EE and CS3 students Chengbo Zang and Mehmet Kerem Türkcan, and professors Kostic, Gil Zussman, and Javad Ghaderi presented a data-driven simulation environment for modeling traffic in metropolitan street intersections. Using real-world tracking data collected over an extended period of time in the NSF-funded COSMOS testbed, the team has been training trajectory forecasting models to learn agent interactions and environmental constraints that are difficult to capture conventionally. Trajectories of new agents are first coarsely generated by sampling from the spatial and temporal generative distributions, then refined using state-of-the-art trajectory forecasting models. The simulations can run either autonomously, or under explicit human control conditioned on the generative distributions, and can be an integral component of the Urban Hybrid Twin research program led by Professor Sharon Di in Columbia University’s Civil Engineering Department. The team presented the experiments for a variety of model configurations.
In the presentation, “Evaluating Zero-Shot Image Tagging for Urban Traffic Intersection Scenes” by EE students William Ho and Türkcan, and professors Ghaderi, Zussman, and Kostic, the use of zero-shot image tagging was explored to enhance the understanding and management of urban traffic intersections. By leveraging advanced computer vision techniques, the research aims to improve traffic monitoring and anomaly detection in complex urban environments, contributing to safer and more efficient city infrastructures.
The startup/university collaborative project, “A Computer Vision Testbed for New York City Street Intersections” presented by Türkcan, Mahshid Ghasemi Dehkordi, Sophia Kleisarchaki, Thibaut Calmant, Ludovic Gürgen, Ghaderi, Zussman, and Kostic, showcased a testbed deployed in New York City for the collection, analysis and real-time monitoring of traffic data. Utilizing state-of-the-art object detection and tracking models, the testbed and monitoring platform by Kentyou provides valuable insights into traffic patterns and helps develop solutions for smart city applications, paving the way for more intelligent and responsive urban traffic systems.
Kostic delivered an invited talk at The 20th Embedded Vision Workshop, titled “Real Time for Urban Streetscape Video-Based Applications.” He discussed the challenges and solutions for real-time video-based applications in dense urban environments. Kostic emphasized the integration of infrastructure-installed sensors, high-bandwidth communications, and AI-enabled computing to enhance the safety and efficiency of urban traffic systems. The talk highlighted the stringent demands of safety-critical applications, focusing on the accuracy, reliability, and robustness required for future smart cities.
The work presented at CVPR 2024 is a sampling of the research themes funded in part by the Center for Smart Streetscapes, an NSF Engineering Research Center based at Columbia University.