Is it possible to configure smartparens in a way that it doesn't insert a second ' (single quote) if in lisp mode ?
Yes. Here's the easy way:
(require 'smartparens-config)
This sets up smartparens with some default settings, most of which I find either useful or inoffensive (take a look with M-x find-library <RET> smartparens-config). One of them disables the single-quote pair in various Lisp modes.
That should take care of this pair almost everywhere it's going to be a problem. The one place it seems to miss is the eval-expression (M-:) command, so I add this line:
(sp-local-pair 'minibuffer-inactive-mode "'" nil :actions nil)
This will affect all commands that use the minibuffer for input, so if you frequently use single quotes in the minibuffer for some reason, you may want to find a more narrowly-focused solution.
If you want to do more tweaking, here is the relevant smartparens documentation.