Search code examples
c++networkingarchitecturezeromqdecoupling

Using ZeroMQ in a crossplatform desktop/mobile app-suite for architecture concerns

I need to make an architecture decision on a cross-platform app-suite. I basically want to try new way of decoupling modules and implement network I/O using ZeroMQ, knowing it's a message queue for in-process, inter-process and networking applications. But I'm not sure how it can fit in with my case.

I'd appreciate it if anyone could clarify a few things before I spend the next week reading their intriguing book: http://zguide.zeromq.org/page:all

I've checked these questions but didn't get my answers:

My requirements:

  • Desktop hosts on Windows and macOS, as separated console backend and GUI frontend; the backend must be written in C++;
  • Mobile guests on iOS and Android, backend written in C++;
  • Desktop talks with mobile using TCP;

Old Way

As for the desktop backend (the console app), a few years back, my first step would be writing a multithreaded architecture based on Observer/Command patterns:

  • Set the main thread for UI and spawn a few threads.
  • One "scheduler" thread for message handling: a queue to get notifications from other modules and another queue for commands. Each command type introduces its own dependencies. The scheduler pumps messages and issues commands accordingly.
  • Other "executor" threads for device monitoring, multiplex network I/O between one desktop and multiple mobile devices, all sending messages to scheduler to have real work scheduled.

I would then need to implement thread-safe message queues, and will inevitably have coupling between schedulers and a bunch of Command classes that are essentially just function wrappers of those executors' behaviors. With C++, this would be a lot of boilerplate code.

New Way to Validate

But it's 2019 so I expect less hand-written code and would try something new. With ZeroMQ, I'd love to see if my expectation holds. I'd love to ...

  • Remove the need of writing a scheduler and message/command queues from scrach, by just passing ZeroMQ requests between in-process modules across threads, because writing scheduling from scratch is tedious and unproductive.
  • Simplify network I/O between desktop and mobile devices. For this part I've tried ASIO and it wasn't significantly more convenient than raw socket and select, plus it's C++-only.
  • Decouple GUI and console app with ZeroMQ-based IPC, so that GUI can be rewritten using different technologies in various languages.
  • Perceive low-latency for both desktop and mobile users.

Is my expectation reasonable?

Solution

  • If new to ZeroMQ domains, feel free to review this and best enjoy a very first look at "ZeroMQ Principles in less than Five Seconds" before diving into further details

    An above referred post has presented an assumption, that:

    ZeroMQ is based on the assumption that there is an while (1) ... loop that it is inserted into

    is completely wrong and misleading any Architecture planning / assessment efforts.

    ZeroMQ is a feature-rich signaling/messaging metaplane, that is intended to provide a lot of services for the application-level code, that may enjoy a light-weight re-use of the smart, complex on low-level, efficient handling of signaling/messaging infrastructure, be it used for in-process, inter-process and inter-node multi-agent distributed fashion, using for that goal many already available transport-class protocols:

    { inproc:// | ipc:// | tipc:// | vmci:// | tcp:// | pgm:// | epgm:// | udp:// }

    This said, let's follow your shopping-list :

    My requirements:

    • c++ ZeroMQ: [PASSED] Desktop hosts on Windows and macOS, as separated console backend and GUI frontend; the backend must be written in C++;
    • c++ ZeroMQ: [PASSED] Mobile guests on iOS and Android, backend written in C++;
    • tcp ZeroMQ: [PASSED] Desktop talks with mobile using TCP;

    I'd love to ...

    • Remove the need of writing a scheduler and message/command queues from scrach, by just passing ZeroMQ requests between in-process modules across threads, because writing scheduling from scratch is tedious and unproductive.
    • Simplify network I/O between desktop and mobile devices. For this part I've tried ASIO and it wasn't significantly more convenient than raw socket and select, plus it's C++-only.
    • Decouple GUI and console app with ZeroMQ-based IPC, so that GUI can be rewritten using different technologies in various languages.
    • Perceive low-latency for both desktop and mobile users.

    Is my expectation reasonable?

    Well :

    • there is obviously no need to write scheduler+Queues from scratch. Queue-management is built-in ZeroMQ and actually hidden inside the service-metaplane. Scheduling things among many-actors is on the other hand your design-decision and has nothing to do with ZeroMQ or other technology of choice. Given your system-design intentions, you decide the way ( "autogenerated magics" are still more a wishful thinking than any near-future system design reality )

    [PASSED] QUEUES : built-in ZeroMQ
    [NICE2HAVE] SCHEDULER : auto-generated for any generic distributed many-agent-wide ecosystem (yet, hard to expect in any near future)

    • network ( and any in principle ) I/O is simplified already in the ZeroMQ hierarchy of services

    [PASSED] : SIMPLIFIED NETWORK I/O - ZeroMQ provides already all abstracted Transport-Class related services hidden to the transparent use of the signaling/messaging metaplane,
    so the application code enjoys to "just" { .send() | .poll() | .recv() }

    [PASSED] : Decoupling GUI from any other part of the ParcPlace-Systems-pioneered-MVC-architecture. Using this since ZeroMQ v2.11 for a (far)remote keyboard over TCP/IP network and even possible to integrate into actor-based GUI, like Tkinter-GUI actors may well serve this distributed local-Visual/remote-distributed-Controller/remote-distributed-Model. If mobile-terminal O/S introduces more complex constraints on the local-Visual MVC-component, proper adaptations ought be validated with domain-experts on that particular O/S properties. ZeroMQ signaling/messaging metaplane has not been considered so far to contain any constraints per se.

    [PASSED] : LATENCY - ZeroMQ was designed from the very start for delivering ultimately low-latency as a must. Given it can feed HFT-tranding ecosystems, the Desktop/Mobile systems are orders of magnitude less restrictive in the sense of E2E lump sum accumulation of all the visited transport + O/S-handling latencies.