Let me explain my question in such way:
In a system we have two machines Alice and Bob. Conversation between two is a transaction sort of - request from one, response from another. Anyone can start conversation - if Alice first asks then Bob reply and vice versa.
// Alice machine (pseudocode)
bind(socket, 90909)
listen(socket)
clientSocket = accept(socket)
read(clientSocket, buffer, len(buffer)
// everything is cool?
// Oh no read is a block call, but how can i write? I have important info for Bob
Another feature of this system is that Bob have higher priority in conversation so if Alice began to ask Bob but didn't finish and Bob think he have more important message, then he interrupt Alice and asks first.
// Alice machine (pseudocode)
write(socket, buffer, len(buffer))
// in the call above Bob can`t interrupt Alice during write to socket
// read response
read(socket, rBuffer, len(rBuffer))
So here is the system described above. It's RPC kind of system between two computers in a network. How can I implement it using sockets?
It's RPC kind of system between two computers in a network. How can I implement it using sockets?
You'll need to abstract the communication away from the sockets layer, so that the details of when and how the messages are sent are no longer so tightly coupled to the explicit/individual socket-calls of the calling programs. In particular, you need to get away from the idea of blocking inside I/O calls; rather you need a mechanism such that your sending and receiving happens in the background as far as your application logic is concerned, so that the timing of the I/O operations doesn't interfere with your application's business logic.
What you are looking for is a messaging system -- i.e. a middleware layer that Alice can just hand a complete message off to and rely on to deliver it to Bob as quickly as possible (and vice versa, of course). That is, you'd like your Alice program to be able to send a message to Bob using code something like this (pseudocode):
Message * msg = new Message;
msg->setSourceAddress("Alice");
msg->setDestinationAddress("Bob");
msg->setContents("Hi Bob, how are you today?");
mailbox->postMessage(msg); // always returns immediately!
... and on the other side, you'll want some mechanism by which Bob can be notified that a Message has arrived for him to look at. This would be a little trickier, since in order to receive notifications, Bob has to have some sort of event loop that lets him block until a Message that has arrived for him to look at (pseudocode):
Message * msg;
while((msg = GetNextIncomingMessage()) != NULL) // we'll block here until a Message is received
{
if (msg->getSourceAddress() == "Alice")
{
printf("Hey, Alice sent me a letter!\n");
printf("It says: [%s]\n", msg->getContents());
}
delete msg;
}
... note that GetNextIncomingMessage() could be written to block until a Message has arrived for Bob to handle, or it could be written to never block, but rather return NULL if no incoming Message has arrived, or perhaps wait for a specified maximum amount of time, or etc.
There are various free software libraries (ZeroMQ, RabbitMQ, MUSCLE, etc) available that will implement this kind of functionality on top of sockets for you, and if you're just looking to get the job done, you're well advised to just pick your favorite one and use it, because rolling your own implementation is a non-trivial amount of work and can be tricky to get right.
That said, I'll assume in the remainder of this answer that you really do want to do it yourself, either as a learning exercise, or because none of the existing libraries suit you -- and that you're willing to spend somewhere between dozens and thousands of hours working on it.
A) The first thing you'll want to do is decide how you want to represent a Message (where "a Message", for our purposes, means a chunk of data that is meant to be sent and received as an atomic unit; i.e. the receiver won't be able to take meaningful action based on its contents until he has received the entire thing). Depending on your use case, a "Message" could be as simple as a NUL-terminated ASCII string, or as complex as a full SQL database or some other complex data structure. Regardless of what you choose, you should be aware that the only thing sockets know how to send or receive are sequences of bytes; so you'll need to write a Flatten-the-Message function (to be called by the sender) that takes a Message and converts it into an equivalent sequence-of-bytes, and also a Unflatten-the-Message function (to be called by the receiver) that takes a sequence-of-bytes and from it reconstructs the Message that those bytes intended to represent. (When implementing these functions, keep in mind potential cross-platform-compatibility gotchas such as differing endian-ness, differing sizeof(int/long/float/etc), differing struct member packing across compilers, etc)
B) Next you'll need to decide what transport mechanism you want to use. For messaging across the network, you really have only two choices, UDP and TCP; and UDP is going to present you with some difficult-to-resolve challenges (firewalls that block incoming UDP packets, dropped UDP packets, packets received out-of-order, maximum-packet-size limitations, etc), so I recommend TCP unless you have a really good reason why you can't use it.
C) Once you've decided that, you'll next need to decide upon (and document, if only for your own sanity) an over-the-wire protocol that you're going to use to send/receive the Messages. For TCP, this needs to include some way for the receiver to know where a given Message's data ends and the next one begins. (e.g. if your Messages are really just NUL-terminated text strings, then the receiver can look for the NUL byte to see where a Message ends; or if they are something more complex, a common way to frame them would be to send the number-of-bytes-in-the-Flattened-byte-sequence first, as part of a fixed size header, then send the flattened-byte-sequence itself, and repeat again for the next Message, as necessary. That way the receiver could read the fixed-size header first, and it would tell the receiver how many bytes of payload data to read next before trying to Unflatten the payload data back into a Message object)
D) Once you've got the protocol decided on, you can start working on the sending part of the I/O event loop. Typically you're going to want to have an in-memory FIFO data structure (i.e. linked-list, double-ended queue, or similar) of outgoing Message objects; that way when the calling code calls postMessage(), you can just add the caller's Message object to the end of the FIFO queue, rather than trying to send it across the socket right away. That way the caller won't have to wait for the Message to be sent; rather it can be sent asynchronously. Of course that brings up the question of how you want to implement the sending code to work in parallel with the user's own code (since you don't want the I/O actions to interfere with the user's code, or vice versa). One way to handle that is to do the I/O in a separate thread; alternatively you could use select()/poll()/etc to integrate the I/O into the user's thread, but that can get pretty invasive, so I'd recommend against it if possible. Your send-event-loop might look something like this (pseudocode):
current_byte_sequence_to_send = NULL;
while(true)
{
// If we have no byte sequence that we're currently sending,
// try to pop the next Message out of the queue and convert it
// into bytes to send
if ((current_byte_sequence_to_send == NULL)&&(outgoing_message_queue.length() > 0))
{
Message * next_msg = outgoing_message_queue.pop_front();
current_byte_sequence_to_send = FlattenMessage(nextMsg);
delete next_msg;
}
if (current_byte_sequence_to_send != NULL)
{
send_more_bytes_from_sequence(current_byte_sequence_to_send);
if (current_byte_sequence_to_send->numBytesSent() == current_byte_sequence_to_send->size())
{
delete current_byte_sequence;
current_byte_sequence_to_send = NULL;
}
}
}
Once you've got the above working, then you'll have two event loops operating in parallel: your regular Alice (or Bob) user program, doing whatever it does, plus the send-I/O event loop whose only job is to drain the outgoing-messages FIFO queue as quickly as it can, by converting each Message, in turn, into bytes and sending those bytes out across the socket.
E) After that, the final step is the receive I/O loop. It's pretty much the inverse of the send I/O loop; it reads byte-sequences from the socket, Unflatten()'s them into Message objects as quickly as it can, and then hands the Message object back to the user's own code (or maybe just adds them to a FIFO for the user's code to pick up later, depending on how you want to handle that)
That's the bare bones of it; there's lots of other things to deal with (e.g. thread-safety issues, efficient cross-thread signaling when more data has arrived, handling network errors gracefully, handling poor network performance gracefully, deciding where it's appropriate to block waiting for input and where it isn't, making the protocol architecture-neutral so that it will work on any CPU, and so on). It took me several years of development to get my implementation fully working to my satisfaction.