Thrust 5: Streetscape Applications

Many CS3 applications will rely upon datasets that live beyond the administrative boundaries of the CS3 team. As one example, in the case of the Smart Intersection application, the application requires access to traffic data beyond the intersection instrumented by CS3. The team is developing standard toolsets to ingest, quality check, and integrate datasets like this (and beyond). Data from NYC department of transportation video streams, and from the Florida testbed is actively being integrated.

Robust decisions in streetscapes and related applications require high fidelity estimation of states and processes in the environment. This activity proposes communication-efficient distributed algorithms to combine sets of cooperative heterogeneous sensors under bandwidth limitation, with provable guarantees. The overarching goals are to manage limited communication bandwidth without suffering performance loss.

Many CS3 applications, including those designed by partners external to the team, depend only upon anonymized, synoptic information about the streetscape. Our Mobility Intelligence application, for example, is concerned only about the objects (e.g., pedestrians, scooters) and activities (e.g., walking, turning left) identified within the streetscape – not the specific characteristics of the participants (e.g., visual information). CS3 is developing a privacy-preserving scene and activity recognition pipeline that simplifies the development of these applications, including those built by external partners. This supports privacy preservation while broadening access for streetscape application developers.

This research is to develop a digital twin platform to support smart streetscape application testing. The digital twin platform is built based on high-resolution LiDAR or photogrammetry scan of the infrastructure and road environment. Trajectories of vehicles, pedestrians, and other road users generated from high-resolution LiDAR and computer vision sensors are used to reconstruct the real-world traffic conditions in the high-resolution infrastructure background. Key research problems lie in addressing the spatial-temporal synchronization and matching of multi-sensor data, predict and infer trajectory data due to line-of-sight coverage loss, and the next-generation traffic simulation models that can simulate high-fidelity traffic behavior in responses to the comprehensive 3D environment in the new digital twin models.