I'm trying to create an Arduino-based display that shows the time of the next few trains at a Metro North stop in NYC. Metro North is run by the MTA (MTA is the NYC subway and bus agency) and I had no trouble creating a similar display for MTA busses. But, the train data is very difficult. It can't be filtered via GET commands in the URL. I have to parse the data myself to get stop-level data. (it's organized by train.)
I have webpage where I can run simple PHP, java etc. I think I need to create a page that will parse the "json"/"XML" file from the MTA for me. Then the Arduino can use the simplified data. I can get real-time data using a URL and my api key. This is what it looks like:
{"header":{"gtfs_realtime_version":"1","timestamp":1442890458},"entity":[{"id":"1392","trip_update":{"trip":{"start_time":"2251","start_date":"09212015","schedule_relationship":0,"route_id":"3"},"vehicle":{"label":"1392"},"stop_time_update":[{"departure":{"delay":180,"time":1442890260},"stop_id":"112"},{"departure":{"delay":180,"time":1442890500},"stop_id":"114"},{"departure":{"delay":180,"time":1442890680},"stop_id":"115"},{"departure":{"delay":180,"time":1442890920},"stop_id":"116"},{"departure":{"delay":180,"time":1442891100},"stop_id":"118"},{"departure":{"delay":180,"time":1442891280},"stop_id":"120"},{"departure":{"delay":180,"time":1442891400},"stop_id":"121"},{"departure":{"delay":180,"time":1442891880},"stop_id":"124"}]}},{"id":"1394","trip_update":{"trip":{"start_time":"2254","start_date":"09212015","schedule_relationship":0,"route_id":"3"},"vehicle":{"label":"1394"},"stop_time_update":[{"departure":{"delay":0,"time":1442890440},"stop_id":"4"},{"departure":{"delay":0,"time":1442890920},"stop_id":"56"},{"departure":{"delay":0,"time":1442891400},"stop_id":"105"},{"departure":{"delay":0,"time":1442891580},"stop_id":"106"},{"departure":{"delay":0,"time":1442891760},"stop_id":"108"},{"departure":{"delay":0,"time":1442892000},"stop_id":"110"},{"departure":{"delay":0,"time":1442892180},"stop_id":"111"},{"departure":{"delay":0,"time":1442892360},"stop_id":"112"},{"departure":{"delay":0,"time":1442892600},"stop_id":"114"},{"departure":{"delay":0,"time":1442892780},"stop_id":"115"},{"departure":{"delay":0,"time":1442893020},"stop_id":"116"},{"departure":{"delay":0,"time":1442893200},"stop_id":"118"},{"departure":{"delay":0,"time":1442893380},"stop_id":"120"},{"departure":{"delay":0,"time":1442893500},"stop_id":"121"},{"departure":{"delay":0,"time":1442893980},"stop_id":"124"}]}},{"id":"1395","trip_update":{"trip":{"start_time":"2303","start_date":"09212015","schedule_relationship":0,"route_id":"3"},"vehicle":{"label":"1395"},"stop_time_update":[{"departure":{"delay":0,"time":1442890980},"stop_id":"4"},{"departure":{"delay":0,"time":1442891640},"stop_id":"1"}]}},{"id":"1586","trip_update":{"trip":{"start_time":"2247","start_date":"09212015","schedule_relationship":0,"route_id":"3"},"vehicle":{"label":"1586"},"stop_time_update":[{"departure":{"delay":60,"time":1442890020},"stop_id":"144"},{"departure":{"delay":60,"time":1442890200},"stop_id":"145"},{"departure":{"delay":60,"time":1442890620},"stop_id":"190"},{"departure":{"delay":60,"time":1442891220},"stop_id":"149"}]}},{"id":"1588","trip_update":{"trip":{"start_time":"2254","start_date":"09212015","schedule_relationship":0,"route_id":"3"},"vehicle":{"label":"1588"},"stop_time_update":[{"departure":{"delay":0,"time":1442890440},"stop_id":"136"},{"departure":{"delay":0,"time":1442890620},"stop_id":"137"},{"departure":{"delay":0,"time":1442890860},"stop_id":"138"},{"departure":{"delay":0,"time":1442891040},"stop_id":"188"},{"departure":{"delay":0,"time":1442891520},"stop_id":"140"},{"departure":{"delay":0,"time":1442891880},"stop_id":"143"},{"departure":{"delay":0,"time":1442892120},"stop_id":"144"},{"departure":{"delay":0,"time":1442892300},"stop_id":"145"},{"departure":{"delay":0,"time":1442892720},"stop_id":"190"},{"departure":{"delay":0,"time":1442893380},"stop_id":"149"}]}},{"id":"1597","trip_update":{"trip":{"start_time":"2203","start_date":"09212015","schedule_relationship":0,"route_id":"3"},"vehicle":{"label":"1597"},"stop_time_update":[{"departure":{"delay":0,"time":1442887380},"stop_id":"144"},{"departure":{"delay":0,"time":1442890560},"stop_id":"124"},{"departure":{"delay":0,"time":1442892900},"stop_id":"4"},{"departure":{"delay":0,"time":1442893560},"stop_id":"1"}]}},{"id":"1699","trip_update":{"trip":{"start_time":"2220","start_date":"09212015","schedule_relationship":0,"route_id":"20"},"vehicle":{"label":"1699"},"stop_time_update":[{"departure":{"delay":60,"time":1442888400},"stop_id":"40714"},{"departure":{"delay":60,"time":1442889600},"stop_id":"40712"},{"departure":{"delay":60,"time":1442889960},"stop_id":"40710"},{"departure":{"delay":60,"time":1442890740},"stop_id":"40704"},{"departure":{"delay":60,"time":1442892000},"stop_id":"149"}]}},{"id":"1795","trip_update":{"trip":{"start_time":"2145","start_date":"09212015","schedule_relationship":0,"route_id":"4"},"vehicle":{"label":"1795"},"stop_time_update":[{"departure":{"delay":4200,"time":1442886300},"stop_id":"124"}]}},{"id":"588","trip_update":{"trip":{"start_time":"2248","start_date":"09212015","schedule_relationship":0,"route_id":"2"},"vehicle":{"label":"588"},"stop_time_update":[{"departure":{"delay":300,"time":1442890080},"stop_id":"4"},{"departure":{"delay":300,"time":1442890740},"stop_id":"1"}]}},{"id":"591","trip_update":{"trip":{"start_time":"2250","start_date":"09212015","schedule_relationship":0,"route_id":"2"},"vehicle":{"label":"591"},"stop_time_update":[{"departure":{"delay":0,"time":1442890200},"stop_id":"56"},{"departure":{"delay":0,"time":1442890320},"stop_id":"57"},{"departure":{"delay":0,"time":1442890500},"stop_id":"58"},{"departure":{"delay":0,"time":1442890680},"stop_id":"59"},{"departure":{"delay":0,"time":1442890800},"stop_id":"61"},{"departure":{"delay":0,"time":1442890980},"stop_id":"62"},{"departure":{"delay":0,"time":1442891100},"stop_id":"64"},{"departure":{"delay":0,"time":1442891220},"stop_id":"65"},{"departure":{"delay":0,"time":1442891340},"stop_id":"66"},{"departure":{"delay":0,"time":1442891520},"stop_id":"68"},{"departure":{"delay":0,"time":1442891700},"stop_id":"71"},{"departure":{"delay":0,"time":1442891880},"stop_id":"72"},{"departure":{"delay":0,"time":1442892060},"stop_id":"74"},{"departure":{"delay":0,"time":1442892480},"stop_id":"76"}]}},{"id":"67","trip_update":{"trip":{"start_time":"0035","start_date":"09222015","schedule_relationship":0,"route_id":"17"},"vehicle":{"label":"67"},"stop_time_update":[{"departure":{"delay":80880,"time":1442896500},"stop_id":"149"},{"departure":{"delay":80880,"time":1442899260},"stop_id":"124"},{"departure":{"delay":80880,"time":1442905200},"stop_id":"15001"},{"departure":{"delay":80880,"time":1442906400},"stop_id":"17016"},{"departure":{"delay":80880,"time":1442907360},"stop_id":"17015"},{"departure":{"delay":80880,"time":1442908980},"stop_id":"17012"},{"departure":{"delay":80880,"time":1442911200},"stop_id":"17010"},{"departure":{"delay":80880,"time":1442912760},"stop_id":"17009"},{"departure":{"delay":80880,"time":1442916600},"stop_id":"17006"},{"departure":{"delay":80880,"time":1442917380},"stop_id":"17005"},{"departure":{"delay":80880,"time":1442918520},"stop_id":"17004"},{"departure":{"delay":80880,"time":1442921400},"stop_id":"17003"},{"departure":{"delay":80880,"time":1442929500},"stop_id":"17002"},{"departure":{"delay":80880,"time":1442937000},"stop_id":"17001"}]}},{"id":"687","trip_update":{"trip":{"start_time":"2254","start_date":"09212015","schedule_relationship":0,"route_id":"2"},"vehicle":{"label":"687"},"stop_time_update":[{"departure":{"delay":0,"time":1442890440},"stop_id":"85"},{"departure":{"delay":0,"time":1442890620},"stop_id":"86"},{"departure":{"delay":0,"time":1442890860},"stop_id":"88"},{"departure":{"delay":0,"time":1442891100},"stop_id":"89"},{"departure":{"delay":0,"time":1442891280},"stop_id":"90"},{"departure":{"delay":0,"time":1442891640},"stop_id":"91"},{"departure":{"delay":0,"time":1442892000},"stop_id":"94"}]}},{"id":"689","trip_update":{"trip":{"start_time":"2256","start_date":"09212015","schedule_relationship":0,"route_id":"2"},"vehicle":{"label":"689"},"stop_time_update":[{"departure":{"delay":120,"time":1442890560},"stop_id":"74"},{"departure":{"delay":120,"time":1442890740},"stop_id":"76"},{"departure":{"delay":120,"time":1442890980},"stop_id":"78"},{"departure":{"delay":120,"time":1442891220},"stop_id":"80"},{"departure":{"delay":120,"time":1442891460},"stop_id":"81"},{"departure":{"delay":120,"time":1442891640},"stop_id":"83"},{"departure":{"delay":120,"time":1442892000},"stop_id":"84"},{"departure":{"delay":120,"time":1442892240},"stop_id":"85"},{"departure":{"delay":120,"time":1442892420},"stop_id":"86"},{"departure":{"delay":120,"time":1442892660},"stop_id":"88"},{"departure":{"delay":120,"time":1442892900},"stop_id":"89"},{"departure":{"delay":120,"time":1442893080},"stop_id":"90"},{"departure":{"delay":120,"time":1442893440},"stop_id":"91"},{"departure":{"delay":120,"time":1442893800},"stop_id":"94"}]}},{"id":"692","trip_update":{"trip":{"start_time":"2253","start_date":"09212015","schedule_relationship":0,"route_id":"2"},"vehicle":{"label":"692"},"stop_time_update":[{"departure":{"delay":0,"time":1442890380},"stop_id":"80"},{"departure":{"delay":0,"time":1442890620},"stop_id":"78"},{"departure":{"delay":0,"time":1442890860},"stop_id":"76"},{"departure":{"delay":0,"time":1442891100},"stop_id":"74"},{"departure":{"delay":0,"time":1442892780},"stop_id":"4"},{"departure":{"delay":0,"time":1442893440},"stop_id":"1"}]}},{"id":"791","trip_update":{"trip":{"start_time":"2252","start_date":"09212015","schedule_relationship":0,"route_id":"1"},"vehicle":{"label":"791"},"stop_time_update":[{"departure":{"delay":0,"time":1442890320},"stop_id":"18"},{"departure":{"delay":0,"time":1442890440},"stop_id":"19"},{"departure":{"delay":0,"time":1442890620},"stop_id":"20"},{"departure":{"delay":0,"time":1442890800},"stop_id":"22"},{"departure":{"delay":0,"time":1442890980},"stop_id":"23"},{"departure":{"delay":0,"time":1442891100},"stop_id":"24"},{"departure":{"delay":0,"time":1442891220},"stop_id":"25"},{"departure":{"delay":0,"time":1442891460},"stop_id":"27"},{"departure":{"delay":0,"time":1442891580},"stop_id":"29"},{"departure":{"delay":0,"time":1442891820},"stop_id":"30"},{"departure":{"delay":0,"time":1442892000},"stop_id":"31"},{"departure":{"delay":0,"time":1442892480},"stop_id":"33"}]}},{"id":"792","trip_update":{"trip":{"start_time":"2249","start_date":"09212015","schedule_relationship":0,"route_id":"1"},"vehicle":{"label":"792"},"stop_time_update":[{"departure":{"delay":180,"time":1442890140},"stop_id":"622"},{"departure":{"delay":180,"time":1442890560},"stop_id":"4"},{"departure":{"delay":180,"time":1442891220},"stop_id":"1"}]}},{"id":"887","trip_update":{"trip":{"start_time":"2245","start_date":"09212015","schedule_relationship":0,"route_id":"1"},"vehicle":{"label":"887"},"stop_time_update":[{"departure":{"delay":180,"time":1442889900},"stop_id":"39"},{"departure":{"delay":180,"time":1442890560},"stop_id":"42"},{"departure":{"delay":180,"time":1442890800},"stop_id":"43"},{"departure":{"delay":180,"time":1442891280},"stop_id":"46"},{"departure":{"delay":180,"time":1442891760},"stop_id":"49"},{"departure":{"delay":180,"time":1442892780},"stop_id":"51"}]}},{"id":"983","trip_update":{"trip":{"start_time":"2253","start_date":"09212015","schedule_relationship":0,"route_id":"2"},"vehicle":{"label":"983"},"stop_time_update":[{"departure":{"delay":60,"time":1442890380},"stop_id":"101"},{"departure":{"delay":60,"time":1442890560},"stop_id":"176"},{"departure":{"delay":60,"time":1442891160},"stop_id":"177"}]}}]}
Like drinking from a firehose.
It's in a format called GTFS which is supposed to be "universal" but I'm finding it difficult to find anything that works with it. For example, simplexml_load_file() in PHP won't work.
Arduinos are not good for parsing text. I can't force the Arduino to do all of the work.
What method should I learn? I do not have much control over my server. I can't change the way PHP runs on my server easily. The methods I've found so far seem to require extensions and other things I just can't do.
Yes, "drinking from a firehose" is a good metaphor for working with GTFS data. One aspect of GTFS is to put the least load on the transit agency's system and have the app developer's system (you) do as much of the work as possible. Both pieces (the "static" GTFS file with the current route information and the GTFS Realtime files) can be static data files served from a "dumb" web server (no web application code required) which makes serving the data very scalable.
Some transit agencies (like MTA, in the case of their bus routes) are nice and do some of the work for you, providing XML or JSON API's to give you current realtime status on specific routes. But, if you learn how to process GTFS data on your end, you don't need to rely on their API and your system becomes "universal", able to deal with any agency serving GTFS realtime information.
My guess is you have already looked at the docs but, just in case, here's the reference: https://developers.google.com/transit/
Ordinarily, you would use the combination of the GTFS file (containing the calendar, routes, trips, stops, and stop_times) and the GTFS realtime files (specifically, the tripUpdate file) to determine what the current status of a vehicle was in relation to the stops on its current trip. They might just give you the vehicles' delay times on the their current trips and then you would need to look up the stop time for each trip for the target stop and adjust it by the corresponding delay.
However, MTA is being nice again and they are giving you enough information just in the realtime feed to show when a vehicle (e.g. train 1586) is going to arrive at a stop (e.g. stop 144) at a time (e.g. unix time 1442890020 = Tue, 22 Sep 2015 02:47:00 GMT) and a delay (e.g. 60 seconds).
So, if your PHP process is trying to create a list of upcoming departure times for a target stop, you would:
json_decode())That would be a fast-track solution for you for this single case.