So I have a background in Java and is starting to learn C++ using Visual Studio. From what I’ve seen, you can put C++ classes and function into .lib and .dll so the linker can use them for other programs. In Java, you can also archive stuffs into jars. So is this comparison correct?
It's a good comparison although not perfect.
Jar files are just archives of .class files, and .class files contain all the information that the Java compiler needs so that you can use class A from class B. It's self-contained.
C++ libraries (.a, .dll, .dylib, etc) are not fully self-contained. They are archives of the .o files created after you compile some .cpp files. However, they are not fully self-contained. The lib is not everything you need. You also need the related include files (typically .h or .hpp, depending on your naming convention).
So yes, they are very similar, although there are important differences.