Search code examples
ruby-on-railsamazon-s3carrierwave

Ruby on Rails memory leak with Carrierwave on AWS S3 in production environment

I have a simple Rails API image deployed on Render with the Starter plan (0.5 CPU / 512 MB).

The User model has a picture field that's a managed field by carrierwave and is hosted on AWS S3.

Whenever my app calls the API to retrieve the User model and its associated picture (which is 4 kB), my server uses all the memory available to it. If pushing with some more requests afterwords, it sometimes crashes. This happens AFTER the request completed correctly to retrieve the user. In this graph we can see it goes from around 75% usage to 100%, so in this case around 120Mb usage for the request.

enter image description here

Here is my Carrierwave config:

CarrierWave.configure do |config|
  if Rails.env.development?
    config.storage = :file
    config.asset_host = 'http://localhost:3000'
  else
    config.storage    = :aws
    config.aws_bucket = "#{ENV.fetch('S3_BUCKET_NAME')}-#{Rails.env}"
    config.aws_acl    = 'private'

    config.aws_authenticated_url_expiration = 60 * 60 * 24 * 7

    config.aws_attributes = -> { {
      expires: 1.week.from_now.httpdate,
      cache_control: 'max-age=604800'
    } }

    config.aws_credentials = {
      access_key_id:     ENV.fetch('AWS_ACCESS_KEY_ID'),
      secret_access_key: ENV.fetch('AWS_SECRET_ACCESS_KEY'),
      region:            ENV.fetch('AWS_REGION'), # Required
      stub_responses:    Rails.env.test?
    }
  end
end

When running the same container locally in development environment (with S3 hosted assets), the memory usage stays stable. But whenever I switch to production, the memory usage goes up like on Render. I also tried to use carrierwave with :file mode in local container with production environment (so without S3), and memory usage does not increase.

Any idea what this could be related to? I've looked into the carrierwave-aws gem issues but didn't find anything...

Solution

  • I ended up switching to fog to handle S3 storage and my app's memory usage got stable again 🎉