From a MSDN page, about .NET Framework versions, we know that each version includes all features from the previous versions.
Unlike previous versions of the .NET Framework, the .NET Framework 4 does not allow an application that was built with previous versions of the .NET Framework to migrate forward and run on it if the previous version is not installed
Why is that?
As already explained in other answers they are different runtimes so by default the application is not supported because the 2.0 runtime used by .NET 3.5 is not installed.
You can however add the following to the application config to force the application to use the .NET 4.0 runtime:
<startup>
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/>
</startup>
This however may cause the application to crash because of changes between the different runtimes so the safest is to use the runtime the application targets.