Thrust 3: Privacy, Security, and Fairness

Building the first public-space data collection and automation systems protected by design against security vulnerabilities and privacy infringements arising from normal operation, malicious attacks, or fusion with external data sources.

Project 1: Analysis of Emerging Smart Streetscape Threats

Streetscape applications, which necessarily operate within the public space, present new privacy, security, and fairness risks. As examples, edge infrastructure may be compromised, machine learning models may be biased, and sensing systems may reveal information that communities wish to hold private. While many of these risks are obvious, others are subtle and not yet understood. CS3 is leading a comprehensive analysis of emerging threats specific to modern and future streetscape applications – an essential first step in ensuring trustworthy streetscapes.

CitiBike Data Release Vulnerability Study

We analyze CitiBike’s publicly available ride datasets to identify privacy vulnerabilities. These datasets include only anonymous information: the start and end date/time and locations of individual rides. Although this data might seem harmless, our analysis shows that it can reveal sensitive information depending on an adversary’s knowledge. For example, if an abusive partner observes someone starting a ride near their home, they could track where and when that ride ends. We found that 80% of NYC rides are uniquely identifiable by their start date/time and location, meaning adversaries could deduce exact ride details with full confidence. Similarly, knowing an end date/time and location often reveals the ride’s origin. We also uncovered risks in group rides, which are uncommon and thus can be traced with minimal information. This study underscores the limitations of relying on anonymity alone in releasing mobility data and informs the privacy architecture in CS3 (Project 2 below), guiding us toward safer practices for incorporating data in public releases and semi-trusted applications.

Project 2: Mitigation of Emerging Smart Streetscape Threats

As streetscape technologies and applications evolve, new risks and threats are likely to emerge. Malicious actors may seek to gain access to privileged information (e.g., raw video feeds), control physical infrastructure (e.g., signal timing, autonomous vehicles), bias decision-making algorithms in favor or against specific populations, or engage in other malicious activities. As these risks and threats emerge, mitigation strategies must be developed to ensure adequate protections. CS3 is advancing the fundamental science of privacy, security, and fairness for emerging streetscape applications. The team’s emphasis is on enabling rigorous, verifiable privacy, security, and fairness guarantees applicable to a broad range of future applications.

Urban Sensing Network Self-Monitoring

Streetscape technology deployed broadly within a (harsh) environment will be under threat from malicious individuals. This project explores principles for self-monitoring, to use Streetscape sensors to sense the status and potential threats against neighboring sensors. For example, cameras positioned within the environment should be viewed by other cameras that continuously inspect the state of the cameras in view.

Privacy Architecture based on Web-Informed API Separation and Individual Privacy Loss Accounting

Smart cities increasingly incorporate function-specific technologies that gather data from public spaces to enhance city infrastructure, often without public awareness and at the expense of personal privacy. This gradual data collection resembles the early, unchecked days of web advertising, which has only recently begun to address privacy through new APIs. Having worked on web privacy architectures, we aim to prevent smart cities from repeating these privacy mistakes, which are difficult to correct once systems are entrenched.

CityOS is our proposed privacy architecture for smart cities, designed with privacy as a foundational element. Taking cue from new web APIs, CityOS distinguishes between three types of functions needed by smart city applications and situational awareness technologies, each exhibiting unique data access needs and privacy risks. Accordingly, CityOS opens three specialized APIs, each with tailored privacy safeguards aligned with these risks. A core principle of CityOS is that each user's device retains ultimate control over privacy, monitoring privacy impacts as users interact with CityOS applications across the city. This project, if successful, will impact how most CS3 applications and situational awareness technologies operate, pushing them through well-defined APIs with clear privacy characteristics.

Project 3: Community Legibility of Threats and Guarantees

Community-based co-production of streetscape applications is a core CS3 tenet. For community partners to meaningfully engage in application co-production, they must have a clear understanding of the associated risks to privacy, security, and fairness. CS3 is advancing new explanatory methods to convey these risks and the associated mitigation options. This includes methods for conveying the underlying formal guarantees afforded by CS3’s mitigation solutions.

Explanatory Methods for Privacy Protections Under DP

CityOS’s privacy architecture uses advanced privacy technologies like sandboxing, secure multi-party computation, and differential privacy. However, these protections are complex and often difficult to communicate to broad audiences due to the nuanced, sometimes imperfect guarantees they provide and the inherent tradeoffs between functionality and privacy. Through scientific methods and user studies, we are developing ways to clearly convey these privacy benefits and tradeoffs to the public, for the purpose of enabling our communities to provide more informed guidance that will ultimately feed back into our privacy architecture design. For instance, our privacy nutrition labels—which highlight which data flows differential privacy restricts—have shown promise for setting accurate privacy expectations. Paired with a brief explanation of how differential privacy works, these labels have been found to build trust. With a better understanding of these privacy technologies, the community will be equipped to provide more informed guidance on how to resolve critical design tensions in CityOS.

Recent Publications

Tholoniat, Pierre, et al. “Cookie Monster: Efficient On-Device Budgeting for Differentially-Private Ad-Measurement Systems.” Proceedings of the ACM SIGOPS 30th Symposium on Operating Systems Principles, ACM, 2024, pp. 693–708, https://doi.org/10.1145/3694715.3695965.

Tholoniat, Pierre, et al. DPack: Efficiency-Oriented Privacy Budget Scheduling. Oct. 2024, https://doi.org/10.48550/arXiv.2212.13228.

Hao, Luoyao, and Henning Schulzrinne. “Poster: Identity-Independent IoT for Overarching Policy Enforcement.” 2024 IEEE Security and Privacy Workshops (SPW), IEEE, 2024, pp. 296–296, https://doi.org/10.1109/SPW63631.2024.00036.

Ou, Tingting, et al. “Thompson Sampling Itself Is Differentially Private.” International Conference on Artificial Intelligence and Statistics, PMLR, 2024, pp. 1576–84, https://proceedings.mlr.press/v238/ou24a/ou24a.pdf.

Cummings, Rachel, et al. ATTAXONOMY: Unpacking Differential Privacy Guarantees against Practical Adversaries. May 2024, https://doi.org/10.48550/arXiv.2405.01716.

Hao, Luoyao, and Henning Schulzrinne. “Poster: Identity-Independent IoT for Overarching Policy Enforcement.” 2024 IEEE Security and Privacy Workshops (SPW), 2024, pp. 296–296, https://doi.org/10.1109/SPW63631.2024.00036.

Ou, Tingting, et al. Thompson Sampling Itself Is Differentially Private. PMLR, 2024, pp. 1576–84, https://proceedings.mlr.press/v238/ou24a/ou24a.pdf.

Kostopoulou, Kelly, et al. “Turbo: Effective Caching in Differentially-Private Databases.” Proceedings of the 29th Symposium on Operating Systems Principles, 2023, pp. 579–94, https://dl.acm.org/doi/10.1145/3600006.3613174.