I have been trying to setup a server where users can send sign in using websockets, but I don't want to do this using ws. I want to be able turn on wss without having https. Sadly, there aren't any options to do this. And so the question is how would one do this on the client side without using https protocol.
From the Websocket protocol specification:
A
wss
URI identifies a WebSocket server and resource name, and indicates that traffic over that connection is to be protected via TLS (including standard benefits of TLS such as data confidentiality and integrity, and endpoint authentication).
Emphasis mine
Now you can understand the absurdity of your request: wss is https.
Of course the terminology is wrong (https is a different protocol than wss) but the bottom of the line is that both are simply the version of their respective TCP plain protocols (http and ws) over TLS.
So the answer is no.
As a matter of fact security is a complex thing.
Very experienced programmers refrain from inventing or exploring new ways and, based on the kind of question you asked, you don't appear to have much expertise this field.
So it's better to do things as best-practices say, it they say to use "https" use "https".
Starting studying security seriously (or hiring a contractor) is advised, inventing new ways to perform secure authentication is not, unless you have a PhD in abstract algebra and several years of experience in developing cryptographic schemes.