Is there no other way to deploy a plugin and make it functional, without restart? Or else can someone give details what happens to plugin.hpi on restart and why restart of Hudson/Jenkins is important?
Many containers will allow you to load plugins without restart. But Jenkins does not guarantee, that new plugin will properly work without restart. This is architectural choice made in Jenkins.
Take a look at this:
Jenkins added the plugin support in 1.44, which is some 400 releases ago and since that time Jenkins required a restart for new plugins to take effect. This is because we load plugins at start-up, doing all kinds of computation to build up the immutable data structure around all the plugins.
And this:
Finally, why is upgrade hard? This is because Jenkins keeps instances of model objects for long time in memory (unlike, say, a typical database application, where those things are request scoped and thus much short-lived.) I like this for a number of reasons, such as more straight-forward object traversals, and the fact that builds take a long time anyway.
See more at: http://blog.cloudbees.com/2011/12/installing-plugins-to-jenkins-without.html#sthash.yFpYe7Sr.dpuf