No, VoIP is not unsafe that way. However remember that VoIP works through internet and is prone to all of the threats and attacks that are associated with internet - viruses, identity theft, phishing, spam, malware. In other words your VoIP phone is as safe once your pc. Are you