I want to copy blobs from one storage account (ADLS Gen2) to another storage account (same type). I have even WORKING code with SAS token authorization:
import com.microsoft.azure.storage.{CloudStorageAccount,StorageException,StorageCredentials,StorageCredentialsToken}
import com.microsoft.azure.storage.blob.{CloudBlobClient,CloudBlockBlob,CloudBlobDirectory,CloudBlobContainer,ListBlobItem}
val srcStorageAccountName = "<src-account-name>"
val destStorageAccountName = "<dest-account-name>"
// SAS authorization
val srcSas = "<src-sas-token>"
val srcCloudStorageAccount= CloudStorageAccount.parse(s"DefaultEndpointsProtocol=https;AccountName=${srcStorageAccountName};SharedAccessSignature=${srcSas};EndpointSuffix=core.windows.net")
val destSas = "<dest-sas-token>"
val destCloudStorageAccount= CloudStorageAccount.parse(s"DefaultEndpointsProtocol=https;AccountName=${destStorageAccountName};SharedAccessSignature=${destSas};EndpointSuffix=core.windows.net")
val destBlobClient = destCloudStorageAccount.createCloudBlobClient()
val srcBlobClient = srcCloudStorageAccount.createCloudBlobClient()
try {
val srcPath = "<src-path>"
val destPath = "<dest-path>"
val srcContainerName = "<src-container>"
val destContainerName = "<dest-container>"
val srcContainer = srcBlobClient.getContainerReference(srcContainerName)
val destContainer = destBlobClient.getContainerReference(destContainerName)
val blobs = srcContainer.listBlobs(srcPath)
import scala.jdk.CollectionConverters._
copyBlobList(blobs.asScala, destContainer, srcContainer, srcContainerName, srcPath, destPath)
} catch {
case e: StorageException =>
e.printStackTrace()
}
def copyBlobList(blobs: Iterable[ListBlobItem], destContainer: CloudBlobContainer, srcContainer: CloudBlobContainer, srcContainerName: String, srcPath: String, destPath: String): Unit = {
for (blob <- blobs) {
blob match {
case blockBlob: CloudBlockBlob => copySingleBlob(blockBlob, destContainer, srcContainer, srcContainerName, srcPath, destPath)
case blobDirectory: CloudBlobDirectory => copyBlobDirectory(blobDirectory, destContainer, srcContainer, srcContainerName, srcPath, destPath)
case _ => println(s"Unknown blob type")
}
}
}
def copySingleBlob(blob: CloudBlockBlob, destContainer: CloudBlobContainer, srcContainer: CloudBlobContainer, srcContainerName: String, srcPath: String, destPath: String): Unit = {
val srcBlob = srcContainer.getBlockBlobReference(blob.getUri.getPath.replaceFirst(srcContainerName, "").substring(1).replace(srcPath, destPath))
val destBlob = destContainer.getBlockBlobReference(blob.getUri.getPath.replaceFirst(srcContainerName, "").substring(1).replace(srcPath, destPath))
destBlob.startCopy(srcBlob)
}
def copyBlobDirectory(blobDirectory: CloudBlobDirectory, destContainer: CloudBlobContainer, srcContainer: CloudBlobContainer, srcContainerName: String, srcPath: String, destPath: String): Unit = {
val blobsFromDir = srcContainer.listBlobs(blobDirectory.getUri.getPath.replace(srcContainerName, ""))
import scala.jdk.CollectionConverters._
copyBlobList(blobsFromDir.asScala, destContainer, srcContainer, srcContainerName, srcPath, destPath)
}
The issue here is that any other autorization results in exception: com.microsoft.azure.storage.StorageException: Public access is not permitted on this storage account.
I have tried:
// Account key authorization
val destKey = "<dest-account-key>"
val srcKey = "<src-account-key>"
val srcCloudStorageAccount = CloudStorageAccount.parse(s"DefaultEndpointsProtocol=https;AccountName=${srcStorageAccountName};AccountKey=${srcKey};EndpointSuffix=core.windows.net")
val destCloudStorageAccount = CloudStorageAccount.parse(s"DefaultEndpointsProtocol=https;AccountName=${destStorageAccountName};AccountKey=${destKey};EndpointSuffix=core.windows.net")
// Access token authorization
val srcTokenCredentials = new StorageCredentialsToken(srcStorageAccountName, srcToken)
val srcCloudStorageAccount = new CloudStorageAccount(srcTokenCredentials, true)
val destTokenCredentials = new StorageCredentialsToken(destStorageAccountName, destToken)
val destCloudStorageAccount = new CloudStorageAccount(destTokenCredentials, true)
Both of them allow to copy blobs inside same storage account but fails on copy action between two different storage accounts.
Question: What I don't really get is why SAS authorization allows to copy between two storage acounts while Account Key autorization and Access Token authorization fails with this "Public access not permitted" exception. Any ideas?
Thanks
REFERENCES: