Search code examples
ruby-on-railssqliteruby-on-rails-3.2acts-as-taggable-onrake-task

Rake Tagging from CSV task to much for SQLite?


I have a rake task that foreach through a CSV file of IDs. I then query the database to see if that id exists in my database. if it exists i use act_as_taggable to add a tag. the script runs fine until it gets to the first match and attempts to write a tag to my SQLite DB. and i get a database locked error. I am wondering if i am hitting the I/O limit of SQLite and need to switch over to a fullblown MySQL db.

if defined?(Rails) && (Rails.env == 'development')
  Rails.logger = Logger.new(STDOUT)
end

require 'csv'
desc "Tag Voters that early voted from the Secretary of State Website"
task :tag_early => [:environment] do
  file ="db/AllAbsentees.csv"

  CSV.foreach(file, :headers=> true) do |row|
    @voter_id = row[1]
    puts "Working on Row" + row[1]
    @voter = Voter.where(:state_id => @voter_id).first()
    unless @voter.blank?
      @voter.tag_list = "2012_GEN_EARLY_VOTER"
      @voter.save()
      puts "Voter #" + @voter_id + "tagged"
    end
  end
end


    Voter Load (1.2ms)  SELECT "voters".* FROM "voters" WHERE "voters"."state_id" = '00008030' LIMIT 1
  ActsAsTaggableOn::Tag Load (0.2ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 11944 AND "taggings"."taggable_type" = 'Voter' AND (taggings.context = 'tags' AND taggings.tagger_id IS NULL)
   (0.1ms)  begin transaction
   (0.4ms)  UPDATE "voters" SET "updated_at" = '2012-11-23 00:02:33.438114' WHERE "voters"."id" = 11944
  ActsAsTaggableOn::Tag Load (0.1ms)  SELECT "tags".* FROM "tags" WHERE (lower(name) = '2012_gen_early_voter')
  ActsAsTaggableOn::Tag Load (0.2ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = 11944 AND "taggings"."taggable_type" = 'Voter' AND (taggings.context = 'tags' AND taggings.tagger_id IS NULL)
  ActsAsTaggableOn::Tagging Exists (0.1ms)  SELECT 1 AS one FROM "taggings" WHERE ("taggings"."tag_id" = 19 AND "taggings"."taggable_type" = 'Voter' AND "taggings"."taggable_id" = 11944 AND "taggings"."context" = 'tags' AND "taggings"."tagger_id" IS NULL AND "taggings"."tagger_type" IS NULL) LIMIT 1
  SQL (1.4ms)  INSERT INTO "taggings" ("context", "created_at", "tag_id", "taggable_id", "taggable_type", "tagger_id", "tagger_type") VALUES (?, ?, ?, ?, ?, ?, ?)  [["context", "tags"], ["created_at", Fri, 23 Nov 2012 00:02:33 UTC +00:00], ["tag_id", 19], ["taggable_id", 11944], ["taggable_type", "Voter"], ["tagger_id", nil], ["tagger_type", nil]]
   (5053.1ms)  commit transaction
SQLite3::BusyException: database is locked: commit transaction
   (99.7ms)  rollback transaction
rake aborted!
SQLite3::BusyException: database is locked: commit transaction

Solution

  • SQLite has not much concurrency; for a transaction to write to the DB, there must be no other reading or writing connections.

    Ensure that no other program is reading from or writing to the DB at the same time, and that all DB accesses in your own program use the same database connection.