Search code examples
rspecrspec-rails

Testing "validates ..., format => with"

I have a User model with this validation

validates :name, :lastname, :format => {:with => /^[a-zA-Z]+$/, :message => 'Only letters and spaces allowed.'}

I'm not sure how to properly test it.

I've done a function that returns a random string made by 10 chars from an array of a-zA-z chars.

def get_random_name
  "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ".split('').shuffle[0..10].join

end end

and then I get a new name for each run of my specs.

I don't want to test that it uses some regexp, because then I would be testing the implementation and not the behavior, and I also don't want to test only one hardcoded case.

My questions are: Should I do that? Is it really needed? Is it better or useless? Do you know any better way to test that kind of validations?

EDIT: another question, what about generating invalid random names? is there a way to create random names that includes at least one char outside the allowed values? I can't hardcode an array of all invalid values to randomize it since it would be TOO big

Solution

  • Testing against random data is a common testing technique called Fuzzing. I would look at using FuzzBert's Generators to create true random binary data to test against.

    Here is an example of some great random data that you would not want getting past your validations.

    irb> require 'fuzzbert'
       => true
irb> test_string = FuzzBert::Generators.random_fixlen(10)[]
       => "S\x1EU1\x11HVY\x0E\xD0"
irb> puts test_string
     S▲U1◄HVY♫�
       => nil

    Random Is Good

    Because it is converting random binary bits to strings, you are going to get some really funky results to test against. That is a good thing! Not only will it test against using known symbols like ?, but also all kinds of combinations of valid and invalid characters.

    Random is sometimes valid

    There is a chance you will get valid data every now and then, although very unlikely. Also, the odds of getting valid data decrease the longer the random string you create is.

    A first attempt to solve this problem could be to just append an invalid character to every output, but I wouldn't suggest it. For example, if you always append "!", then it would make your test equivalent to making sure the string doesn't have a "!" in it and not a true test of the full regex possibilities.

    I would suggest testing your random strings against that same regex, if it does pass it, generate a different string to test with.

    Finally, you could just ignore the rare chance that the random data is valid. If every once-in-a-while this particular rspec fails, look at what string it failed on, if it was a valid string, then just rerun.

    Test all the possibilities

    You will never be able to test all non-valid strings (unless you have a short maximum length), but by using Fuzz Testing you will be able to test against strings that contain a large variety of valid and invalid characters.