Search code examples
mysqlruby-on-railsbulk-load

Why ActiveRecord Model.import is failing to insert bulk records in MySQL?

I am trying to insert about 200,000 records using ActiveRecord import method. It's failing with TypeError: can't quote Hash error. I searched in the net, but nothing meaningful related to my scenario.

I started with smaller size and was successful up to 120,000 records. If I keep trying I think I can find a limit. Is there anything else I can do other than inserting the records into smaller batch?

Here is my configuration - Rails 5.0.3, Ruby 2.3.0, Mysql 5.1.73. I know these are very outdated.

Anyone had similar issue and how did you fix it?

Solution

  • "TypeError: can't quote Hash" is thrown by the connection adapter. Source.

    This error can be easily recreated:

    Arel::Nodes.build_quoted({}).to_sql. 
#=> TypeError: can't quote Hash

    This means that it is likely that somewhere in the records you are trying to import there is a bare Hash, rather than JSON (which is probably what was intended).

    If this is the case you may need to search through the records to determine where these offending Hashes are and modify them accordingly. For instance if they are intended to be JSON data you can use Hash#to_json to convert them.