If the sites reside on two different servers, the following factors may weigh in:
- Old server - Old hardware=slower
- High server load - More load=slower server.
- Load balancing (more than one server) - More servers=faster response.
- Available bandwidth - Low bandwidth=slower download. This could be normal or you might have routers that are underperforming.
- Number of routers between the server and the Internet - Each hop takes time.
- Distance between server and your browser - Longer=slower (more jumps between servers).
- Internet load - If the servers in between you and the server are under load, it's getting slower.
- High load on your ISP's network - More traffic=slower download
Make sure to host your sites with a reliable hosting service.