Search code examples
ruby-on-railscsvruby-on-rails-4jsonbsmartercsv

Ruby: If column exists use it, if not add to other column


I'm parsing a CSV and trying to distinguish between columns in Model and "virtual" columns that'll be added to a JSONB :data column. So far I've got this:

rows = SmarterCSV.process(csv.path)
rows.each do |row|
  row.select! { |x| Model.attribute_method?(x) } # this ignores non-matches
  Model.create(row)
end

That removes columns from the CSV row that don't match up with Model. Instead, I want to add the data from all those into a column in Model called :data. How can I do that?

Edit

Something like this before the select! maybe?

row[:data] = row.select { |x| !Model.attribute_method?(x) }

Solution

  • There are a number of ways you could do this. One particularly straightforward way is with Hash#slice! from Rails' ActiveSupport extensions, which works like Array#slice! and returns a Hash with those keys that weren't given in its arguments, while preserving the keys that were given:

    rows = SmarterCSV.process(csv.path)
    attrs = Model.attribute_names.map(&:to_sym)
    
    rows.each do |row|
      row[:data] = row.slice!(*attrs)
      Model.create(row)
    end
    

    P.S. This could probably be filed under "Stupid Ruby Tricks," but if you're using Ruby 2.0+ you can take advantage of the double-splat (**) for this compact construction:

    rows.each do |row|
      Model.create(data: row.slice!(*attrs), **row)
    end
    

    P.P.S. If your CSVs are big and you find yourself having performance concerns (calling create a few thousand times—and the subsequent database INSERTs—ain't cheap), I recommend checking out the activerecord-import gem. It's designed for exactly this sort of thing. With it you'd do something like this:

    rows = SmarterCSV.process(csv.path)
    attrs = Model.attribute_names.map(&:to_sym)
    
    models = rows.map do |row|
      row[:data] = row.slice!(*attrs)
      Model.new(row)
    end
    
    Model.import(models)
    

    There are other, faster options as well in the activerecord-import docs.