Keep parallax moving for one second on mouseout and stop smoothly

I would like to make a website with mouse parallax effect like in this page http://brightmedia.pl background mouse parallax is so smooth..

I have two questions:

When you mouseover on a container from, let's say, the top left corner, the image jumps. How can I make a smooth animation?
When you mouseout of a container, how can I make the image move a little bit and stop with a smooth animation?

What would code to solve these problems be?

Here is basic code:

$('.container').mousemove( function(e){
    var xPos = e.pageX;
    var yPos = e.pageY;

  $('#par1').css({marginLeft: -xPos/20});

});

.container {
  position: relative;
  width: 100%;
  height: 800px;
  background: grey;
  overflow: hidden;
  margin: 0 auto;
}

.container img {
  width: 110%;
  height: 100vh;
  position: absolute;
}

body{
  height: 1000px;
}
h1{
  font-size: 60px;
  z-index: 10;
  position: absolute;
  left: 50%;
  top: 30%;
  transform: translate(-50%, -50%);
}

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <title></title>
    <link rel="stylesheet" href="css.css">
  </head>
  <body>

<div class="container" id="container">
  <img id="par1" src="https://www.gettyimages.ca/gi-resources/images/Homepage/Hero/UK/CMS_Creative_164657191_Kingfisher.jpg" alt="">
  <h1>TEXT</h1>

</div>

    <script src="https://code.jquery.com/jquery-3.3.1.min.js"></script>
    <script type="text/javascript" src="script.js"></script>
  </body>
</html>

Solution

As I had solved the problem long time ago and I forgot about this post so I decided to update with the answer. Maybe it will be helpful for anyone else.

Problem solved by using GSAP. Below You can see the code that works exactly as I wanted

    let wrap = document.getElementById('container');
  	let request = null;
  	let mouse = { x: 0, y: 0 };
  	let cx = window.innerWidth / 2;
  	let cy = window.innerHeight / 2;
    
  	document.querySelector('.container').addEventListener('mousemove', function(event) {
  		mouse.x = event.pageX;
  		mouse.y = event.pageY;
      cancelAnimationFrame(request);
      request = requestAnimationFrame(update);
  	});
    
  	function update() {
  		dx = mouse.x - cx;
  		dy = mouse.y - cy;
  		let tiltx = (dy / cy );
  		let tilty = - (dx / cx);

      TweenMax.to("#container img", 1, {x:-tilty*20, y:-tiltx*20, rotation:0.01, ease:Power2.easeOut});
    }
    
    window.addEventListener('resize', function(event){
       window.innerWidth / 2;
    	 window.innerHeight / 2;
    });

* {
  margin:0;
  padding:0;
  box-sizing:border-box;
}
.container {
  position: relative;
  width: 100%;
  height: 100vh;
  overflow: hidden;
  display:flex;
  flex-direction:column;
  justify-content:center;
  align-items:center;
}
.container img {
  width: 110%;
  height: 120vh;
  position: absolute;
}
h1 {
  z-index:100;
  font-size: 6rem;
  z-index: 10;
  color:#333;
}

<script src="https://cdnjs.cloudflare.com/ajax/libs/gsap/latest/TweenMax.min.js"></script>
<div class="container" id="container">
  <img id="par1" src="https://www.gettyimages.ca/gi-resources/images/Homepage/Hero/UK/CMS_Creative_164657191_Kingfisher.jpg" alt="">
  <h1>GSAP Mouse Parallax</h1>

</div>