I want to implement a reversible image watermarking that use IWT transform and filter 5/3. at first I thing this filter is cdf 5.3 that it is in matlab. but after running the code I found out that this is not true and filter 5/3 is different. after search that I had, I found this filter is legall 5/3 that used in JPEG2000. now I should implement filter legall 5/3 at first and then choose subband HL1 to embedding my data for watermarking. I am a little confused how to implement this filter and use of HL.
The LeGall 5/3 wavelet (CDF 5/3) is named as bior2.2 in MATLAB.
The dwt2 command with 'bior2.2' argument performs the two-dimensional CDF 5/3 wavelet decomposition:
[LL,HL,LH,HH] = dwt2(x,'bior2.2');
This returns the requested HL subband. However, you probably do not want to use it.
As you have mentioned, an integer-to-integer approximation of CDF 5/3 wavelet is also used in the JPEG 2000 standard for a lossless compression. This is possible thanks to lifting scheme. In MATLAB, the integer CDF 5/3 lifting scheme is named cdf2.2. Thus, the following commands are what you probably looking for:
ls = liftwave('cdf2.2');
[LL,HL,LH,HH] = lwt2(x,ls);
See lwt2 for more details.