Search code examples
rgtfs

Reading GTFS-realtime files using R?

I want to analyze GTFS-realtime files using R, compared to the static GTFS, these files are compiled and reading them is trickier.

Googling around, I have only found this package to deal with GTFS https://github.com/ropenscilabs/gtfsr

But again, this is just for static GTFS.

Are you aware of a cran/github R package that deals with GTFS-realtime?

An alternative solution would be to convert the GTFS-RT into a more readable format like json streaming gtfs real time data into human readable format

Solution

  • The GTFS realtime feeds are binary Protocol Buffers, that can be processed by the RProtoBuf package.

    A simple worked example using my local South-east Queensland Translink feed:

    library(RProtoBuf)

    Load the actual proto file which specifies the format the feed files actually follow:

    download.file(url="https://gtfsrt.api.translink.com.au/api/realtime/protobuf", destfile="translink-gtfs-realtime.proto")
readProtoFiles("translink-gtfs-realtime.proto")

    Check all the 'Descriptors' that are now available for loading feeds in the 'Descriptor Pool'

    ls("RProtoBuf:DescriptorPool")
## [1] "GTFSv2.Realtime.Alert"             "GTFSv2.Realtime.EntitySelector"   
## [3] "GTFSv2.Realtime.FeedEntity"        "GTFSv2.Realtime.FeedHeader"       
## [5] "GTFSv2.Realtime.FeedMessage"       "GTFSv2.Realtime.Position"
## ...

    Read the actual feeds - stored in the 'FeedMessage'/'entity' in this case

    download.file(url="https://gtfsrt.api.translink.com.au/api/realtime/SEQ/TripUpdates", destfile="SEQ-TripUpdates.pb")
download.file(url="https://gtfsrt.api.translink.com.au/api/realtime/SEQ/VehiclePositions", destfile="SEQ-VehiclePositions.pb")

vehicle_position_feed <- read(GTFSv2.Realtime.FeedMessage,  "SEQ-VehiclePositions.pb")[["entity"]]
trip_update_feed  <- read(GTFSv2.Realtime.FeedMessage,  "SEQ-TripUpdates.pb")[["entity"]]

    When read, each object is just a set of pointers to parts of the binary file:

    str(vehicle_position_feed)
##List of 6
## $ :Formal class 'Message' [package "RProtoBuf"] with 2 slots
##  .. ..@ pointer:<externalptr> 
##  .. ..@ type   : chr "GTFSv2.Realtime.FeedEntity"
## $ :Formal class 'Message' [package "RProtoBuf"] with 2 slots
##  .. ..@ pointer:<externalptr> 
##  .. ..@ type   : chr "GTFSv2.Realtime.FeedEntity"
## ..

    You can then extract info from each data point by looping over the file to construct datasets to work with, e.g.:

    data.frame(
  id = sapply(vehicle_position_feed, \(x) x[["id"]] ),
  latitude = sapply(vehicle_position_feed, \(x) x[["vehicle"]][["position"]][["latitude"]] ),
  longitude = sapply(vehicle_position_feed, \(x) x[["vehicle"]][["position"]][["longitude"]] )
)
##                   id  latitude longitude
##1     VU-2123549587_1 -27.06561  153.1595
##2    VU-1176076363_10 -27.30158  152.9881
##3   VU--1272517086_10 -27.49080  153.2397
## ...