Google Measurement Protocol for GA4 stopped recording or partially excludes data randomly

Question

I have reason to believe Google Analytics updated their algorithm to partially or entirely filter out events received via the measurement protocol on high traffic'd Properties. I can send the same request to two environments ( except for the obvious credentials differentials ). Both Properties reveal via Debug View that they are successfully receiving these events and all values, but they are absent in RealTime View and from the Reports.

High Traffic'd Property

It will record,

• Device, UserAgent, EngagementTimeMSec, and PageReferrer 50% of the time.
• page_location was recorded 20% of the time.
• Session ID was recorded 10% of the time.
• All other property are ignored entirely.

Low Traffic Property

It records all attributes 100% of the time.

An example of the event that is sent :

{:client_id=>"1699346174.6829318294",
 :events=>
  [{:name=>"foo_bar",
    :params=>
     {:debug_mode=>1,
      :engagement_time_msec=>42,
      :page_location=>"/l/triptest?foo=bar",
      :page_referrer=>"mobile",
      :user_agent=>
       "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36",
      :session_id=>"1699349952",
      :device=>"Trip: Spaceship"}}]}

I can confirm we made zero changes to our Google Analytics Admin or Properties. And that the data became partially or entirely ignored as of September 22nd. Which leads me to believe that Google perhaps released an update that ended collection in whatever unique circumstance my data structure is in.

Both Properties High and Low are equal in all settings and are mirrors of each other except for the data stream ID's, API secrets, etc.

Can anyone determine if there is a missing or malformed attribute or if Google MP is known for filtering out events that otherwise successfully appear in Debug View?

Answer 1

The reason this does not work is severalfold:

In order of importance :

1. custom_events in GA4 depend on your explicitly defining custom definition for preset values page_location and page_referrer etc.
2. This only becomes a problem in high traffic'd properties, and only then when they have received X amount of traffic. So what would seemingly be normal will abruptly stop working eventually with no explanation whatsoever.
3. RealTime Events is in no way "real time". It can take 10 to 30+ minutes to receive data. If it goes beyond 30+ min. then it excludes it entirely. In addition, its results are filtered. In that, what it does receive data, only "some" of the actual content may or not be presented. In short, it does not represent what the API is receiving. Like GA4 as a whole, it should not be taken seriously. As the product is far below standard of what most analytic driven apps have to offer.