I'm trying to generate download and upload link from the Google Cloud, to view and upload files, using the following code:
public class Test {
public static void main(String[] args) throws IOException {
Storage storage = StorageOptions.newBuilder().setCredentials(
ServiceAccountCredentials.fromStream(new FileInputStream("C:/cred/Key.json")))
.build()
.getService();
String filePath = "file/path/";
File file = new File(filePath);
byte[] bytes = Utilities.fileToByteArray(file);
String mimeType = Utilities.getMimeType(bytes);
BlobId blobId = BlobId.of("bucket", file.getName());
BlobInfo blobInfo = BlobInfo.newBuilder(blobId).setContentType(mimeType).build();
URL urlGet = storage
.signUrl(BlobInfo.newBuilder("bucket", "object").build(), 1, TimeUnit.HOURS,
SignUrlOption.httpMethod(HttpMethod.GET));
URL urlPut = storage
.signUrl(blobInfo, 1, TimeUnit.DAYS, SignUrlOption.httpMethod(HttpMethod.PUT),
SignUrlOption.withContentType());
System.out.println(urlGet);
System.out.println(urlPut);
}
}
urlGet contains the download link and urlPut contains the upload link. When I run the program, I get the following output:
https://storage.googleapis.com/roshanbucket/jasperviewpdf?GoogleAccessId=myservice@deft-idiom-234709.iam.gserviceaccount.com&Expires=1552986620&Signature=OZl6M4uMkigu6JBMYoDumgs8P4EC%2BtcK44zHMPkG2xzq3hFfsrj8YYRRtajI8diz64vdCX54nct3wuEpXNRwcnzCmq4KdD53%2B8gbERNuttm8K6%2BlZDLGF3wng%2BCSMzghbGbLnYaZRiZbvjCG%2B3ObBUg9ZiY0qRlif9nyGFftsGUF9GGHvHP6HWP51DJOAurGytSjf9SA5HKPOw4e%2B%2BP1LltfI7m3WjWhxwnSYz4lVxcR4eksec7ILTi66jnwu1gxXtqp75XTxLp9vQa6RC4dCPGoTridFQcMqm89TVzf58c8krk7apQCR6TWp2tAWuFr2xJ1U5FwFfiBaoSX4A33rw%3D%3D
https://storage.googleapis.com/roshanbucket/pi.jpg?GoogleAccessId=myservice@deft-idiom-234709.iam.gserviceaccount.com&Expires=1553069420&Signature=YHsGTgXIBum9t5M7U%2F9fdibDvzBKttQGh0jxzbYgJkevQbpOh8gRQYOlHdjT86byobDE5TNEGF5VrGFAtI5rhRGxLw0xqcNT%2BYGfvHxAIfAJXy5ormXxWVnVEnwGMafyVLOtdIY4asa0niFu%2B36eaIqtD5UzsjUY%2F18OW%2FwvjfQmhlmsvJ7qSkfD1Oif5Rv6c%2F67z1zT7gz7rB4gTCG6mLALuRrOIwCPO%2BkyzOxP9PhEJkoB7j446v%2BhE%2F0pt0wM2nJ29%2BK3HRUShhccJzzZ%2BZRxgOXeUL44CsnYlssaTThU%2FztyUbsXWXbs2hroTcFxVVtOp7aGeCUs1qjdJkXaEg%3D%3D
When I click on the first link (i.e download), it renders the file from the bucket without any problem, but when I use the second link to upload a file from my computer to the Google Cloud, using HTTP PUT with Postman, it gives me the following error, with Status 403:
<?xml version='1.0' encoding='UTF-8'?>
<Error>
<Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature
you provided. Check your Google secret key and signing method.</Message>
<StringToSign>PUT
multipart/form-data; boundary=-------------------------
-025804137217704409263172
1553069420
/roshanbucket/pi.jpg</StringToSign>
</Error>
I have no idea what's causing this. Some help would be really appreciated.
After a while of struggle, finally managed to get it running. Turns out, first I need to generate a signed URL, the equivalent of
gsutil signurl -c 'Content-Type' \
-m RESUMABLE /path/to/your/json_cert_file.json \
gs://your_bucket/file.txt
Then using that signed URL, send an empty POST request with Content-Type and x-goog-resumable:start headers, the equivalent of
curl -v -X 'POST' \
-H 'content-type: text/plain' \
-H 'x-goog-resumable:start' \
-d '' '<signedURL>'
A successful POST will return status 201 with Location header with the actual location where you can upload the file using HTTP PUT.
Below is the Java class I wrote to finish this, with the help of this article
import com.google.api.client.util.Base64;
import com.google.auth.oauth2.ServiceAccountCredentials;
import java.io.DataOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.HttpURLConnection;
import java.net.URL;
import java.net.URLEncoder;
import javax.ws.rs.client.Client;
import javax.ws.rs.client.ClientBuilder;
import javax.ws.rs.client.Entity;
import javax.ws.rs.client.ResponseProcessingException;
import javax.ws.rs.core.Response;
import uploader.Utilities;
public class Uploader {
private ServiceAccountCredentials creds; // Service Account Credentials
private String saEmail; // Service Account email
public Uploader() {
/* Initialize credentials and service account email*/
try (InputStream inputStream = new FileInputStream("C:/cred/Key.json")) {
this.creds = ServiceAccountCredentials.fromStream(
inputStream);
} catch (IOException e) {
e.printStackTrace();
}
this.saEmail = "service account email";
}
/* Sign and return the URL for POST, using credentials from above*/
private String getSignedUrl(String bucketName, String objectName, String mimeType) {
String signed_url = null;
try {
String verb = "POST";
long expiration = System.currentTimeMillis() / 1000 + 60;
String Canonicalized_Extension_Headers = "x-goog-resumable:start";
String content_type = mimeType;
byte[] sr = creds.sign(
(verb + "\n\n" + content_type + "\n" + expiration + "\n" + Canonicalized_Extension_Headers
+
"\n" + "/" + bucketName + "/" + objectName).getBytes());
String url_signature = new String(Base64.encodeBase64(sr));
signed_url = "https://storage.googleapis.com/" + bucketName + "/" + objectName +
"?GoogleAccessId=" + saEmail +
"&Expires=" + expiration +
"&Signature=" + URLEncoder.encode(url_signature, "UTF-8");
} catch (Exception ex) {
ex.printStackTrace();
}
return signed_url;
}
/* Send POST request to the signed URL using custom headers and an empty body, which returns the actual upload location */
public String getLocation(String bucketName, String objectName, String mimeType)
throws IOException {
URL myURL = new URL(getSignedUrl(bucketName, objectName, mimeType));
HttpURLConnection myURLConnection = (HttpURLConnection) myURL.openConnection();
myURLConnection.setRequestMethod("POST");
myURLConnection.setRequestProperty("Content-Type", mimeType);
myURLConnection.setRequestProperty("x-goog-resumable", "start");
// Send post request
myURLConnection.setDoOutput(true);
DataOutputStream wr = new DataOutputStream(myURLConnection.getOutputStream());
wr.flush();
wr.close();
int responseCode = myURLConnection.getResponseCode();
if (responseCode != 201) {
System.out.println("Request Failed");
}
return myURLConnection.getHeaderField("Location");
}
/* Do the actual upload and return the PUT Response*/
public Response doUpload(String url, InputStream inputStream, String mimeType) {
Response response = null;
Client client = ClientBuilder.newClient();
try {
response = client.target(url)
.request()
.put(Entity.entity(inputStream, mimeType));
if (response.getStatus() != 200) {
System.out.println("Request failed with " + response.getStatus());
}
} catch (ResponseProcessingException e) {
e.printStackTrace();
}
return response;
}
}
Now, simply call it in the main method
public static void main(String[] args) throws Exception {
Uploader uploader = new Uploader();
String filePath = "file/path";
File file = new File(filePath);
byte[] bytes = Utilities.fileToByteArray(file); // convert file to bytes
String mimeType = Utilities.getMimeType(bytes); // bytes from above used with tika
String url = uploader.getLocation("bucket", file.getName(), mimeType);
Response r = uploader.doUpload(url, new FileInputStream(file), mimeType);
System.out.println("Response : " + r.getStatus());
System.out.println(r.getHeaders());
}
Hope this helps someone! This method doesn't require sending POST request with Jwt in Authorization Bearer.