I have created a domain joined HdInsight cluster. This is working great at allowing users from my active director login to the cluster using their AD credentials.
However, there doesn't appear to be anyway to limit user access to various files/directories.
user1 has full access to user2's files and visa versa.
My original hope was to use Ranger to configure HDFS permissions for various users/groups - however, this doesn't seem to be a possibility given blob storage on the back end.
I've also tried setting the user/group/other permissions using commands such as "hadoop fs -chown" however, when ssh'd into the cluster (headnode) all users seem to be able to do whatever they want.
Is this expected behavior? is there any way to restrict user access to various directories on HDFS ?
Using Azure Data Lake on the back end unfortunately isn't an option for me as it is not currently available in my region.
Any help/insight is greatly appreciated !
The underlying file system has to support POSIX-ACL. Azure Blob Storage is not supporting this feature, so I would expect the behavior. With Azure Data Lake Store Gen1 the described scenario would work.