GrayTrack · Beyond Direct Sensing

A vehicle follows a route through a neighborhood. Sparse blue cameras provide direct observations, while gray third-party cameras provide indirect passage events along the route. — Tracking beyond direct sensor coverage, using weak cues from third-party sensing assets.

Abstract

Directly accessible sensors are often sparse, leaving long gaps in vehicle tracking. Additional third-party sensors may be nearby, but their raw data remain inaccessible. GrayTrack combines weak, anonymous passage events from these “gray assets” with sparse direct observations using a road-constrained particle filter. We build a CARLA–Mininet-WiFi pipeline that infers passages from encrypted camera-traffic metadata and evaluates their value for downstream tracking. The detector achieves an event-level F1 of 0.989. Incorporating indirect observations reduces trajectory RMSE by 60.1% and catastrophic track loss from 35.8% to 0.3%, demonstrating that even weak observations can extend tracking coverage.

From indirect observations to vehicle tracks

Encrypted third-party camera traffic passes through packet grouping and vehicle-passage detection. A road-constrained particle filter fuses these events with time, location, and identity from accessible cameras and a road map to track vehicles. — Packet timing and size yield anonymous passage events. Road-network constraints connect these events with sparse, identified direct observations.

Key results

Lower trajectory RMSE: 60.1%; 92.3 m → 36.8 m
Passage-detection F1: 0.989; 242 held-out camera sequences
Catastrophic track loss: 35.8% → 0.3%; Trajectories with RMSE > 100 m

Tracking results compare Road-PF with direct observations alone against direct observations plus noisy indirect events at measured detector error rates.

Code

Part 1

Testbed & passage detection

CARLA capture, Mininet-WiFi replay, learned packet grouping, and anonymous passage-event export.

Explore Part 1

Part 2

Road-constrained tracking

Road-PF and tracking baselines, with experiments on sensing noise, blind gaps, and multiple vehicles.

Explore Part 2

Paper & citation

The arXiv version is coming soon. Download BibTeX

@misc{dong2026graytrack,
  title  = {Beyond Direct Sensing: Harnessing Indirect Observations from Third-Party Sensors in Vehicle Tracking},
  author = {Dong, Gaofeng and Eyunni, Vamsi and Sharma, Pragya and Yang, Kang and Srivastava, Mani},
  year   = {2026},
  url    = {https://nesl.github.io/GrayTrack/},
  note   = {arXiv version forthcoming}
}