ADLS Client Bulk Download - downloading files but no data

Question

I want to download a directory from Azure Data Lake Store to my local directory and I am using this method- Bulk Download. The issue is that when I run my code its downloading the files as expected but the files have no data and their sizes also show 0KBs.

Here is how I am using the method-

string srcPath = "/souce path included/"; 
string dstPath = "C:/Users/keprasad/Documents/Views/"; 
clientProd.BulkDownload(srcPath, dstPath);

I am fairly confident that I have created the client correctly with right credentials as it is not throwing an error and actually downloading the files. Anyone has an idea as to when this scenario can occur and what needs to be done to remedy it?

PS: Actually my very first question in Stackoverflow(excited for the future!)

Answer 1

If you want to use a service principal to access Azure data lake gen1, we need to configure the right Access control for the sp. For more details, please refer to here and here.

For example. I want to download file (/Seattle/Portland/Data.txt). we need to configure ACL for the sp as below.

Code

download a single file

 string appId = "";
            string appSecret = "";
            string domain = "";

            var serviceSettings = ActiveDirectoryServiceSettings.Azure;
            serviceSettings.TokenAudience = new Uri(@"https://datalake.azure.net/");
            
            var creds = await ApplicationTokenProvider.LoginSilentAsync(domain, appId, appSecret, serviceSettings);

            string accountName = "";
            AdlsClient client = AdlsClient.CreateClient($"{accountName}.azuredatalakestore.net", creds);
            string fileName = "/Seattle/Portland/data.txt";
            string line = null;
            using (var readStream = new StreamReader(client.GetReadStream(fileName)))
            {
                while ((line =  await readStream.ReadLineAsync()) != null) {

                    Console.WriteLine(line);
                }
               
            }
            Console.ReadLine();

2. Bulk download

string appId = "";
            string appSecret = "";
            string domain = "";

            var serviceSettings = ActiveDirectoryServiceSettings.Azure;
            serviceSettings.TokenAudience = new Uri(@"https://datalake.azure.net/");

            var creds = await ApplicationTokenProvider.LoginSilentAsync(domain, appId, appSecret, serviceSettings);

            string accountName = "";
            AdlsClient client = AdlsClient.CreateClient($"{accountName}.azuredatalakestore.net", creds);
            string srcPath = "/Seattle/Portland/";
            string desPath = @"D:\sample";
            TransferStatus status =client.BulkDownload(srcPath, desPath);