Sending very long strings from MetaTrader 4 to Python backend using ZeroMQ Push/Pull Pattern

I have a quite weird issue related to ZeroMQ messaging between MetaTrader 4 and a python backend.

The below shown code creates three strings which should be forwarded to a ZeroMQ PUSH socket. However, it only sends the live_trades string via the socket. Both the account_info and the historical_trades strings are not sent.

While debugging the weirdness only increased:

When I print the historical_trades string, it is shown in the MT4 expert journal, but not in the MT4 log editor
When I add Print( historical_trades ) before or after the pushSocket.send(StringFormat("%s", historical_trades, true)); it sends the string.

Possible issues which I actually exclude:

The PUSH socket is too slow: I decreased the timer to 10 seconds and more, same result
MT4 itself is too slow to create the strings: It prints them correctly, so it is fast enough
The ZeroMQ SNDHWM ( HighWaterMark ) blocks the EA/socket from sending multiple strings in no time: Already increased it to 100 which should be more than sufficient

Any other ideas I cannot verify with debugging:

The MT4 Print() function does some other stuff behind the scenes which enables the EA to send the string?
Maybe there is a maximum string size in MT4 which blocks the string creation loop (my string is around 35.000 characters long). But then why does it work when I just Print() the string once within the onTimer() function?...
Any chance that the account related information is not accessible on weekends?

Or it is just my code which might be buggy..

Any ideas much appreciated!

int OnInit()
  {
//---

   EventSetTimer(2);     // Set Second Timer as push intervall

   context.setBlocky(false);

   // Send data to PULL_PORT that consumer is listening to.
   Print("Connecting MT4 Dashex Feeder to Dashboard on Port " + IntegerToString(PUSH_PORT) + "..");
   pushSocket.connect(StringFormat("%s://%s:%d", ZEROMQ_PROTOCOL, HOSTNAME, PUSH_PORT));

   pushSocket.setSendHighWaterMark(100);
   pushSocket.setLinger(0); 

//---
   return(INIT_SUCCEEDED);
  }


//+------------------------------------------------------------------+
//| Expert deinitialization function                                 |
//+------------------------------------------------------------------+
void OnDeinit(const int reason)
{
//---

   Print("Disconnecting MT4 Dashex Feeder on Port " + IntegerToString(PUSH_PORT) + "..");
   pushSocket.disconnect(StringFormat("%s://%s:%d", ZEROMQ_PROTOCOL, HOSTNAME, PUSH_PORT));

   // Shutdown ZeroMQ Context
   context.shutdown();
   context.destroy(0);

   EventKillTimer();
}


//---

//+------------------------------------------------------------------+
//| Expert timer function                                             |
//+------------------------------------------------------------------+
void OnTimer()
{

   /*
      1.) Account information
   */   

      string account_info = "";   
      account_info = account_info +

         "account_info|" +
         version + "|" +
         DID + "|" +
         IntegerToString(AccountNumber()) + "|" +
         AccountInfoString(ACCOUNT_COMPANY) + "|" +
         IntegerToString(AccountInfoInteger(ACCOUNT_LEVERAGE)) + "|" +
         DoubleToString(AccountInfoDouble(ACCOUNT_BALANCE),2) + "|" +
         DoubleToString(AccountInfoDouble(ACCOUNT_PROFIT),2) + "|" +
         DoubleToString(AccountInfoDouble(ACCOUNT_EQUITY),2) + "|" +
         DoubleToString(AccountInfoDouble(ACCOUNT_MARGIN),2) + "|" +
         DoubleToString(AccountInfoDouble(ACCOUNT_MARGIN_FREE),2) + "|" +
         DoubleToString(AccountInfoDouble(ACCOUNT_MARGIN_LEVEL),2) + "|" +
         AccountInfoString(ACCOUNT_CURRENCY) + "|" +
         IntegerToString(IsDemo()) + "|" +

         pushSocket.send(StringFormat("%s", account_info, true));
         Print("Pushing Account Information To Dashex.Finance Dashboard For Account No. " + IntegerToString(AccountNumber()));


//+------------------------------------------------------------------+
//+------------------------------------------------------------------+         

   /*
      2.) Pending and running trades
   */

      int live_orders = OrdersTotal();
      string live_trades = "";

      for(int i=live_orders; i >= 0; i--)
      {
         if(OrderSelect(i,SELECT_BY_POS)==false) continue;

           live_trades = live_trades +

           "live_trades|" +
           version + "|" +
           DID + "|" +
           IntegerToString(AccountNumber()) + "|" +
           IntegerToString(OrderTicket()) + "|" +
           TimeToString(OrderOpenTime(), TIME_DATE|TIME_SECONDS) + "|" +
           TimeToString(OrderCloseTime(), TIME_DATE|TIME_SECONDS) + "|" +
           IntegerToString(OrderType()) + "|" +
           DoubleToString(OrderLots(),2) + "|" +
           OrderSymbol() + "|" +
           DoubleToString(OrderOpenPrice(),5) + "|" +
           DoubleToString(OrderClosePrice(),5) + "|" +
           DoubleToString(OrderStopLoss(),5) + "|" +
           DoubleToString(OrderTakeProfit(),5) + "|" +
           DoubleToString(OrderCommission(),2) + "|" +
           DoubleToString(OrderSwap(),2) + "|" +
           DoubleToString(OrderProfit(),2) + "|" +
           "<" + OrderComment() + ">|";
      }

      pushSocket.send(StringFormat("%s", live_trades, true));
      Print("Pushing Live Trades To Dashex.Finance Dashboard For Account No. " + IntegerToString(AccountNumber()));

//+------------------------------------------------------------------+
//+------------------------------------------------------------------+

   /*
      3.) History Trades      
   */


         int hstTotal = OrdersHistoryTotal();
         string historical_trades = "";

         Print(hstTotal);
         for(int i=hstTotal; i >= 0; i--)
         {

           if(OrderSelect(i,SELECT_BY_POS,MODE_HISTORY)==false) continue;

           historical_trades = historical_trades +

              "historical_trades|" +
              version + "|" +
              DID + "|" +
              IntegerToString(AccountNumber()) + "|" +
              IntegerToString(OrderTicket()) + "|" +
              TimeToString(OrderOpenTime(), TIME_DATE|TIME_SECONDS) + "|" +
              TimeToString(OrderCloseTime(), TIME_DATE|TIME_SECONDS) + "|" +
              IntegerToString(OrderType()) + "|" +
              DoubleToString(OrderLots(),2) + "|" +
              OrderSymbol() + "|" +
              DoubleToString(OrderOpenPrice(),5) + "|" +
              DoubleToString(OrderClosePrice(),5) + "|" +
              DoubleToString(OrderStopLoss(),5) + "|" +
              DoubleToString(OrderTakeProfit(),5) + "|" +
              DoubleToString(OrderCommission(),2) + "|" +
              DoubleToString(OrderSwap(),2) + "|" +
              DoubleToString(OrderProfit(),2) + "|" +
              "<" + OrderComment() + ">|"; 
         }

         pushSocket.send(StringFormat("%s", historical_trades, true));
         Print("Pushing History Trades To Dashex.Finance Dashboard For Account No. " + IntegerToString(AccountNumber()));

         Sleep(1);
}

MetaTrader 4 log.editor:

MetaTrader 4 console:

Solution

Q : Any chance that the account related information is not accessible on weekends?

This has an easy proof: do a Comment( aStringUnderTEST ); + check all account / Broker-related items.

Q : Maybe there is a maximum string size in MT4 which blocks the string creation loop...?

This is, except a case of an MT4-Build-X.Y.Z release-bug awaiting a BugFIX, very low probability hypothesis.

Yet it has an easy proof: do a loop of growing string-lengths and test until what size the processing keeps working. The indirectly proved size-limit will help you track the root cause, MT4 not being the SPoF here, is it?

Q : The MT4 Print function does some other stuff behind the scenes which enables the EA to send the string?

The MT4 Support ought be contacted if this is to get confirmed or denied. The software is licensed as an as-is product, so do not expect any rapid response or a rocket science to take place if you try to drill down to the bolts and nuts, inside their closed and sealed product.