Azure Java DirectoryClient slow to list files and directories

Question

I'm using the MS Azure Java API in Spring by including azure-spring-boot-starter-storage-3.6.0 and I'm getting a serious performance bottleneck when iterating the contents of a small directory (40 items).

Taking the MS example from https://learn.microsoft.com/en-us/azure/storage/files/storage-java-how-to-use-file-storage?tabs=java as a starting point:

public static Boolean enumerateFilesAndDirs(String connectStr, String shareName,
                                                String dirName)
{
    StopWatch stopwatch = new StopWatch();
    try
    {
        ShareDirectoryClient dirClient = new ShareFileClientBuilder()
             .connectionString(connectStr).shareName(shareName)
             .resourcePath(dirName)
             .buildDirectoryClient();
        stopwatch.start("Start stream");
        dirClient.listFilesAndDirectories().forEach(

            if(stopwatch.isRunning()) {
               stopwatch.stop();
               log.debug("Time taken to start stream of files and directories: {} ms", stopwatch.getLastTaskTimeMillis());
            }

            fileRef -> System.out.printf("Resource: %s\t Directory? %b\n",
            fileRef.getName(), fileRef.isDirectory())
        );

        return true;
    }
    catch (Exception e)
    {
        System.out.println("enumerateFilesAndDirs exception: " + e.getMessage());
        return false;
    }
}

I've added in some stopwatch log statements, and it is taking around 20 seconds for the forEach to start outputting. Once it's started, then in proceeds at an expected speed to output all the contents of the directory.

I also put in stopwatch log statements around creating the various ShareClient and ShareDirectoryClients that are required to get to the location where my files are, and these interactions are as expected, taking 2ms to complete.

Can anyone shed any insight as to what is happening here, or how I might be able to diagnose where/why this delay is happening?

Answer 1

After this problem disappearing one day and reappearing the next, I did some more digging, and found that this appears to be due to the default netty http client that azure is using.

Updating my POM to the following has resolved this issue:

<dependency>
        <groupId>com.azure.spring</groupId>
        <artifactId>azure-spring-boot-starter-storage</artifactId>
        <version>3.6.0</version>
        <exclusions>
            <exclusion>
            <!-- Removed due long delays in listing folder contents. Using azure-core-http-okhttp instead-->
                <groupId>com.azure</groupId>
                <artifactId>azure-core-http-netty</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <!-- https://mvnrepository.com/artifact/com.azure/azure-core-http-okhttp -->
    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-core-http-okhttp</artifactId>
        <version>1.7.1</version>
    </dependency>