We got a use case to retrieve some files from Azure Blob Storage #1 and upload it to another Azure Blob Storage #2. Is it a feasible way in which it can be implemented via Apache Camel.
I have tried with a sample route included below, however its not working in the expected format, ended up in below exception.
java.lang.RuntimeException: Stream mark expired. at com.azure.storage.common.StorageInputStream.reset(StorageInputStream.java:366) ~[azure-storage-common-12.11.1.jar:na] at org.apache.camel.component.azure.storage.blob.BlobUtils.getInputStreamLength(BlobUtils.java:38) ~[camel-azure-storage-blob-3.12.0.jar:3.12.0] at org.apache.camel.component.azure.storage.blob.BlobStreamAndLength.createBlobStreamAndLengthFromExchangeBody(BlobStreamAndLength.java:50) ~[camel-azure-storage-blob-3.12.0.jar:3.12.0] at org.apache.camel.component.azure.storage.blob.operations.BlobOperations.uploadBlockBlob(BlobOperations.java:182) ~[camel-azure-storage-blob-3.12.0.jar:3.12.0] at org.apache.camel.component.azure.storage.blob.BlobProducer.process(BlobProducer.java:89) ~[camel-azure-storage-blob-3.12.0.jar:3.12.0]
from("azure-storage-blob://storage_name/container?accessKey=xxx&blobName=int_1111.csv")
.to("azure-storage-blob://storage_name/container?accessKey=xxx&blobName=Outbound_$simple{date:now:yyyyMMdd_HHmmss}.csv&operation=uploadBlockBlob");
UPDATE : When I convert the body to byte[], I was able to upload the file content to Azure blob, however this might impact the memory when dealing with large files.
from("azure-storage-blob://storage_name/container?accessKey=xxx&blobName=int_1111.csv")
.convertBodyTo(byte[].class)
.to("azure-storage-blob://storage_name/container?accessKey=xxx&blobName=Outbound_$simple{date:now:yyyyMMdd_HHmmss}.csv&operation=uploadBlockBlob");
Any suggestion or feedback to handle the file upload in a better way.
I've raised a ticket with Camel framework team to incorporate the Azure Copy Blob operation in starter pack - https://issues.apache.org/jira/browse/CAMEL-17161.
They have added the feature and its available from 3.14.0 version of azure blob starter pack, if anyone is interested in exploring the functionality, please feel free to check the documentation page for examples.
from("azure-storage-blob://storage_account_name/container_1?accessKey=xxxxxx&blobName=int_1111.csv")
.process(exchange -> {
exchange.getIn().setHeader(BlobConstants.BLOB_NAME, "int_1111.csv");
exchange.getMessage().setHeader(BlobConstants.SOURCE_BLOB_CONTAINER_NAME, "container_1");
exchange.getMessage().setHeader(BlobConstants.SOURCE_BLOB_ACCOUNT_NAME, "storage_account_name");
})
.to("azure-storage-blob://storage_account_name/container_2?accessKey=xxxxxx&operation=copyBlob&sourceBlobAccessKey=xxxxxx");
Copy Blob API : https://learn.microsoft.com/en-us/rest/api/storageservices/copy-blob
Maven Repo link : https://mvnrepository.com/artifact/org.apache.camel.springboot/camel-azure-storage-blob-starter/3.14.0
Examples (refer copy blob section): https://camel.apache.org/components/3.14.x/azure-storage-blob-component.html