When I try to paste hundreds of lines or more into Vim, even in paste mode, it takes much longer (10 times or more?) than to paste that same text into TextEdit and save it as a file. I would speculate that Vim is trying to insert one character at a time and that this is slowing things down. When it's quite a large paste, I end up having to save the text with TextEdit and then open it up in Vim. Is there a faster way to paste text in?
If your Vim is compiled with clipboard support (:echo has('clipboard') returns 1) you can simply use "+p or "*p.
If it's not (:echo has('clipboard') returns 0), get yourself a proper Vim. On Mac OS X, MacVim is the way to go.