The Internet measurement community is increasingly sensitive to the privacy implications of both active and passive measurement. Research into the drawbacks of network data anonymization has led the community to investigate data sharing techniques, as well as to focus on active measurements and active measurement datasets. A key metric in these datasets is round-trip-time (RTT) as measured e.g. by ping or traceroute. This paper examines the assumption that the analysis of Internet RTT data is safe for open research by posing the question, "what potentually-private inferences can be made about a remote target given periodic latency measurements from known vantage points under one’s control?" We explore the risks to end-user privacy both through a review of diverse literature touching on the subject as well as on the analysis of RTT data from fixed and mobile Internet measurement infrastruture. While we find that the common assumption of safety generally holds, we explore caveats and give recommendations for mitigation in those cases where it may not.