How to speed up a slow loading PHP page that is processing several large arrays?

How one would go about loading this sort of site properly. Currently it is taking a page roughly 10 second to load, and that is obviously far too long.

Some things to just assume for now:

The server is not causing the slowness
Media like images aren't causing the slowness
The slowness is due to the poor way I currently have things set up in my current 'pass' at learning HTML/CSS/PHP (i don't know what I'm doing, but im trying to learn as I go and redoing everything via full 'passes' as I gain knowledge to reduce code etc and im due for another pass)

Basically I am making a site that people can goto to get useful information about a game. This site uses the games API to pull data that is mainly in the form of arrays.

Here is a sample at the starter array (Career): http://pastebin.com/X33k0NKw

One of the arrays comes from that (One of the heros): http://pastebin.com/fA4tjgxd

The item array that comes from that (also large): http://pastebin.com/gaqba3i7

Now from the item array I further get file that gives me data that I can use in tooltips.

Now I have all this data coming from the server, and its taking 10~sec to all get finished. Way too long. You’re thinking I need to set up a database and get the data in there. And I plan to. However I can’t possibly be doing this correctly, so I want to be able to get this done right before I start doing the MySQL stuff.

So my question is what should I be searching for to help with this sort of thing?

Solution

My rule of thumb when dealing with any RESTful API data is this: A RESTful API is not a database and should never be used for direct real-time calls for data on request. They are simply not built for speed. Especially for arrays of data this large. The data array is simply so large the combined weight of fetching the data & acting on the data is slowing down your page.

The solution? If the RESTful API calls are filled with data you need to cache those in your local database—or on the file system—and then act on the data you have stored locally in the database or file system. Even if the caching time between accesses of the API is 5 seconds, the speed of accessing them locally will still be great.

So that all means your setup is like this:

[Your Page] -> [Your Code] -> [Your Call to the RESTful API]

You need to add some local storage/caching layer:

So that all means your setup is like this:

[Your Page] -> [Your Code] -> [Your Local Storage of API Data] -> [Your Call to the RESTful API to Update Local Storage]

Now how long you cache the data locally in a database or on the file system is your call. I have dealt with RESTful APIs where data only needs to be updated once a day as well as systems where the caching adds up to maybe just a minute. But even if the caching is a minute, that could be enough breathing room to help your system deal with the size of the data.

And past all of that, you should also consider adding a AJAX layer to your front-end functionality that would be encompassed in the [Your Page] areas of those short diagrams I have above.

AJAX stands for Asynchronous JavaScript. And the general concept for data issues like this is the core page HTML is rendered by standard methods in whatever language you are using. Then after the HTML loads, a JavaScript call is made on some basic—time interval, user interaction, a mix of both, something else…—which requests data via JSOP from your core application framework & then the JavaScript updates elements on the page based on that data.

So of you were yo break down the [Your Page] functionality it could be like this:

[Your Server Side Script Renders a Page] -> [Browser Loads HTML, CSS & JavaScript] -> [JavaScript AJAX Calls Adjust the Content in Your Renered HTML]

The beauty of this is your core page is never reloaded again. It stays there like a framework waiting to be filled with content. And AJAX calls fill that framework with content.