I was wondering how I can achieve sandboxing in .net core, since appdomains are not supported in .net core for this. Also, I can't use Virtualization or docket containers because these things are not available on my target machine.
Per .NET Core docs, looks like your options are a bit limited.
Free of problematic tech. .NET Core doesn’t include certain technologies we decided to discontinue because we found them to be problematic, for instance AppDomain and sandboxing. If the scenario still makes sense for .NET Core, our plan is to have replacements. For example, AssemblyLoadContext replaces AppDomains for loading and isolating assemblies.
App Domains
Why was it discontinued? AppDomains require runtime support and are generally quite expensive. While still implemented by CoreCLR, it’s not available in .NET Native and we don’t plan on adding this capability there.
What should I use instead? AppDomains were used for different purposes. For code isolation, we recommend processes and/or containers. For dynamic loading of assemblies, we recommend the new AssemblyLoadContext class.
Since you cannot use containers, it looks like the only option is to run your sandbox in a separate process.
Source: https://blogs.msdn.microsoft.com/dotnet/2016/02/10/porting-to-net-core/