I know my question is broad, but I have no clue where to start my research.
How can one implement VoIP on a server? I am fairly certain it does not use the http/https protocol. If so, what standard/famous protocols one can use? are there any open-source ones? What are good references to start working on that?
Start by looking at the SIP,RTP and RTCP protocols. I believe they form a minimalistic set of required protocols for VoiP
Some relevant open source projects:
http://www.fsf.org/campaigns/priority-projects/priority-projects/highpriorityprojects#Replaceskype
"There are a number of such programs, such as Ekiga, Twinkle, Coccinella, QuteCom, and Jitsi. Unfortunately, these programs only replace some of Skype's functionality, and only in some situations. WebRTC has a mission to enable rich, high quality, Real-Time Communications (RTC) applications to be developed in the browser via simple Javascript APIs and HTML5. Developers should consider helping free software VoIP and video, chat, and multimedia communications projects."