Search code examples
ruby-on-railsfeed

How can I speed up this feed-building process, which integrates multiple model types?


I have a feed that is displayed to the user, which includes 4 different model types.

It simulates grouping the entries by day, by inserting a day object into the feed.

The feed is sorted chronologically and paginated.

This is how I currently build the feed.

def get_feed initial_feed_position=0, number_of_entries_to_get=30
  # display is called on each model to get them all into a standard format
  feed = (first_models + second_models + third_models + forth_models).map { |feed_entry| feed_entry.display }
  feed += day_entries_for_feed(feed)
  end_feed_position = initial_feed_position + number_of_entries_to_get
  (feed.sort_by { |feed_entry| -feed_entry[:comparison_time].to_i })[initial_feed_position...end_feed_position]
end

def day_entries_for_feed feed
  # iterate over a set of dates that contain feed entries
  feed.map{ |feed_entry| feed_entry[:date] }.uniq.map do |day|
    # building the day object in the standard feed entry format. fields that are not relevant to this question have been left out.
    {
      type: 'day',
      comparison_time: (day + 24.hours - 1.second).time # to ensure that the day appears above it's corresponding entries in the feed, the comparison time is set to 1 second before the day ends
    }
  end
end

Over time, the number of objects in the system has built up, and now the feed takes a long time to build using this method. Is there a better way to do it?

I'm using Rails 3.2.13, Ruby 1.9.3 & PostgreSQL 9.1.9.


Solution

  • Because you're getting all the entries in the database a lot of models are loaded into memory, to solve this problem you would have to look in to UNION (which is a pain to maintain and you will have to have literal SQL in your codebase). A good example of it is here: PosgreSQL: How to union 3 tables sorted by date

    Another option would be to derive a base class and do the querying on this. Which would result in something like this:

                                  +-------------+
                                  |  BASE FEED  |
                                  |             |
                                  +------^------+
                                         |
            +-------------------+--------+---------+----------------+
            |                   |                  |                |
      +-----+-------+    +------+------+   +-------+-----+   +------+-----+
      |  MODEL ONE  |    |  MODEL TWO  |   | MODEL THREE |   | MODEL FOUR |
      |             |    |             |   |             |   |            |
      +-------------+    +-------------+   +-------------+   +------------+
    

    Once you have your models set up like this it's a simple matter of querying this base table. Which could look something like this:

    def get_feed(initial_feed_position = 0, number_of_entries_to_get = 30)
      feeds = BaseFeed.
        limit(number_of_entries_to_get).
        offset(initial_feed_position).
        order("DATE(date_field) DESC")
    end
    

    The above example is not the exact solution but if you elaborate a bit more on what you're trying to get as a result set I can adjust it but it's more about the approach to take.

    Hope this helps.