How do you take a video from Google Cloud Storage and generate a jpeg image from one of its frames?
The frame can be used as a thumbnail for the video. Or by getting frames at periodic times that can be used to make preview frames when scrubbing the video.
I like to do this with Java server-side (on Google App Engine). This is the code I can use to get the Blob of the video in Google Cloud Storage. What are my options after I do that?
Storage storage = StorageOptions.getDefaultInstance().getService();
BlobId blobId = BlobId.of(BUCKET, OBJECT_NAME);
Blob blob = storage.get(blobId);
I'm looking for a fast and lightweight solution because this will be run server-side in Google App Engine.
Although the answer of @VonC is excellent, I would recommend you using the functionality provided by the Transcoder API instead of a custom solution.
Specifically the Transcoder API gives you the capability of generating a spritesheet of video frames.
As indicated in the documentation, you have two options for generating the thumbnails:
You have two options for generating the spritesheet:
- Generate a set number of thumbnail images distributed evenly across the input video timeline.
- Generate a thumbnail image periodically, that is, every n seconds, across the input video timeline.
The API provides SDK for different programing languages, Java (2) among them. This related example could be of help as well. For reference:
import com.google.cloud.video.transcoder.v1.AudioStream;
import com.google.cloud.video.transcoder.v1.CreateJobRequest;
import com.google.cloud.video.transcoder.v1.ElementaryStream;
import com.google.cloud.video.transcoder.v1.Input;
import com.google.cloud.video.transcoder.v1.Job;
import com.google.cloud.video.transcoder.v1.JobConfig;
import com.google.cloud.video.transcoder.v1.LocationName;
import com.google.cloud.video.transcoder.v1.MuxStream;
import com.google.cloud.video.transcoder.v1.Output;
import com.google.cloud.video.transcoder.v1.SpriteSheet;
import com.google.cloud.video.transcoder.v1.TranscoderServiceClient;
import com.google.cloud.video.transcoder.v1.VideoStream;
import java.io.IOException;
public class CreateJobWithSetNumberImagesSpritesheet {
public static final String smallSpritesheetFilePrefix = "small-sprite-sheet";
public static final String largeSpritesheetFilePrefix = "large-sprite-sheet";
public static final String spritesheetFileSuffix = "0000000000.jpeg";
public static void main(String[] args) throws IOException {
// TODO(developer): Replace these variables before running the sample.
String projectId = "my-project-id";
String location = "us-central1";
String inputUri = "gs://my-bucket/my-video-file";
String outputUri = "gs://my-bucket/my-output-folder/";
createJobWithSetNumberImagesSpritesheet(projectId, location, inputUri, outputUri);
}
// Creates a job from an ad-hoc configuration and generates two spritesheets from the input video.
// Each spritesheet contains a set number of images.
public static void createJobWithSetNumberImagesSpritesheet(
String projectId, String location, String inputUri, String outputUri) throws IOException {
// Initialize client that will be used to send requests. This client only needs to be created
// once, and can be reused for multiple requests.
try (TranscoderServiceClient transcoderServiceClient = TranscoderServiceClient.create()) {
VideoStream videoStream0 =
VideoStream.newBuilder()
.setH264(
VideoStream.H264CodecSettings.newBuilder()
.setBitrateBps(550000)
.setFrameRate(60)
.setHeightPixels(360)
.setWidthPixels(640))
.build();
AudioStream audioStream0 =
AudioStream.newBuilder().setCodec("aac").setBitrateBps(64000).build();
// Generates a 10x10 spritesheet of small images from the input video. To preserve the source
// aspect ratio, you should set the spriteWidthPixels field or the spriteHeightPixels
// field, but not both.
SpriteSheet smallSpriteSheet =
SpriteSheet.newBuilder()
.setFilePrefix(smallSpritesheetFilePrefix)
.setSpriteHeightPixels(32)
.setSpriteWidthPixels(64)
.setColumnCount(10)
.setRowCount(10)
.setTotalCount(100)
.build();
// Generates a 10x10 spritesheet of larger images from the input video.
SpriteSheet largeSpriteSheet =
SpriteSheet.newBuilder()
.setFilePrefix(largeSpritesheetFilePrefix)
.setSpriteHeightPixels(72)
.setSpriteWidthPixels(128)
.setColumnCount(10)
.setRowCount(10)
.setTotalCount(100)
.build();
JobConfig config =
JobConfig.newBuilder()
.addInputs(Input.newBuilder().setKey("input0").setUri(inputUri))
.setOutput(Output.newBuilder().setUri(outputUri))
.addElementaryStreams(
ElementaryStream.newBuilder()
.setKey("video_stream0")
.setVideoStream(videoStream0))
.addElementaryStreams(
ElementaryStream.newBuilder()
.setKey("audio_stream0")
.setAudioStream(audioStream0))
.addMuxStreams(
MuxStream.newBuilder()
.setKey("sd")
.setContainer("mp4")
.addElementaryStreams("video_stream0")
.addElementaryStreams("audio_stream0")
.build())
.addSpriteSheets(smallSpriteSheet) // Add the spritesheet config to the job config
.addSpriteSheets(largeSpriteSheet) // Add the spritesheet config to the job config
.build();
var createJobRequest =
CreateJobRequest.newBuilder()
.setJob(
Job.newBuilder()
.setInputUri(inputUri)
.setOutputUri(outputUri)
.setConfig(config)
.build())
.setParent(LocationName.of(projectId, location).toString())
.build();
// Send the job creation request and process the response.
Job job = transcoderServiceClient.createJob(createJobRequest);
System.out.println("Job: " + job.getName());
}
}
}
The input of the transcoder Job is obtained from Cloud Storage.
As a consequence, in order to start the transcoding process with this code probably the way to go could be defining a [Cloud Function that responds to the creation] - object finalize event - of a video](https://cloud.google.com/functions/docs/calling/storage).
You can find an example of a function that handles this type of events in the GCP documentation.
import com.google.cloud.functions.CloudEventsFunction;
import com.google.events.cloud.storage.v1.StorageObjectData;
import com.google.protobuf.InvalidProtocolBufferException;
import com.google.protobuf.util.JsonFormat;
import io.cloudevents.CloudEvent;
import java.nio.charset.StandardCharsets;
import java.util.logging.Logger;
public class HelloGcs implements CloudEventsFunction {
private static final Logger logger = Logger.getLogger(HelloGcs.class.getName());
@Override
public void accept(CloudEvent event) throws InvalidProtocolBufferException {
logger.info("Event: " + event.getId());
logger.info("Event Type: " + event.getType());
if (event.getData() == null) {
logger.warning("No data found in cloud event payload!");
return;
}
String cloudEventData = new String(event.getData().toBytes(), StandardCharsets.UTF_8);
StorageObjectData.Builder builder = StorageObjectData.newBuilder();
JsonFormat.parser().merge(cloudEventData, builder);
StorageObjectData data = builder.build();
logger.info("Bucket: " + data.getBucket());
logger.info("File: " + data.getName());
logger.info("Metageneration: " + data.getMetageneration());
logger.info("Created: " + data.getTimeCreated());
logger.info("Updated: " + data.getUpdated());
}
}
The final code in the function could be similar to this (forgive for any inaccuracy, I just try combining the two samples):
import com.google.cloud.functions.CloudEventsFunction;
import com.google.events.cloud.storage.v1.StorageObjectData;
import com.google.protobuf.InvalidProtocolBufferException;
import com.google.protobuf.util.JsonFormat;
import io.cloudevents.CloudEvent;
import java.nio.charset.StandardCharsets;
import java.util.logging.Logger;
import com.google.cloud.video.transcoder.v1.AudioStream;
import com.google.cloud.video.transcoder.v1.CreateJobRequest;
import com.google.cloud.video.transcoder.v1.ElementaryStream;
import com.google.cloud.video.transcoder.v1.Input;
import com.google.cloud.video.transcoder.v1.Job;
import com.google.cloud.video.transcoder.v1.JobConfig;
import com.google.cloud.video.transcoder.v1.LocationName;
import com.google.cloud.video.transcoder.v1.MuxStream;
import com.google.cloud.video.transcoder.v1.Output;
import com.google.cloud.video.transcoder.v1.SpriteSheet;
import com.google.cloud.video.transcoder.v1.TranscoderServiceClient;
import com.google.cloud.video.transcoder.v1.VideoStream;
import java.io.IOException;
public class TranscodingFunction implements CloudEventsFunction {
private static final Logger logger = Logger.getLogger(HelloGcs.class.getName());
public static final String smallSpritesheetFilePrefix = "small-sprite-sheet";
public static final String largeSpritesheetFilePrefix = "large-sprite-sheet";
public static final String spritesheetFileSuffix = "0000000000.jpeg";
String projectId = "my-project-id";
String location = "us-central1";
@Override
public void accept(CloudEvent event) throws InvalidProtocolBufferException {
logger.info("Event: " + event.getId());
logger.info("Event Type: " + event.getType());
if (event.getData() == null) {
logger.warning("No data found in cloud event payload!");
return;
}
String cloudEventData = new String(event.getData().toBytes(), StandardCharsets.UTF_8);
StorageObjectData.Builder builder = StorageObjectData.newBuilder();
JsonFormat.parser().merge(cloudEventData, builder);
StorageObjectData data = builder.build();
logger.info("Bucket: " + data.getBucket());
logger.info("File: " + data.getName());
logger.info("Metageneration: " + data.getMetageneration());
logger.info("Created: " + data.getTimeCreated());
logger.info("Updated: " + data.getUpdated());
String inputUri = data.getBucket() + data.getName();
String outputUri = "gs://my-bucket/my-output-folder/";
// Initialize client that will be used to send requests. This client only needs to be created
// once, and can be reused for multiple requests.
try (TranscoderServiceClient transcoderServiceClient = TranscoderServiceClient.create()) {
VideoStream videoStream0 =
VideoStream.newBuilder()
.setH264(
VideoStream.H264CodecSettings.newBuilder()
.setBitrateBps(550000)
.setFrameRate(60)
.setHeightPixels(360)
.setWidthPixels(640))
.build();
AudioStream audioStream0 =
AudioStream.newBuilder().setCodec("aac").setBitrateBps(64000).build();
// Generates a 10x10 spritesheet of small images from the input video. To preserve the source
// aspect ratio, you should set the spriteWidthPixels field or the spriteHeightPixels
// field, but not both.
SpriteSheet smallSpriteSheet =
SpriteSheet.newBuilder()
.setFilePrefix(smallSpritesheetFilePrefix)
.setSpriteHeightPixels(32)
.setSpriteWidthPixels(64)
.setColumnCount(10)
.setRowCount(10)
.setTotalCount(100)
.build();
// Generates a 10x10 spritesheet of larger images from the input video.
SpriteSheet largeSpriteSheet =
SpriteSheet.newBuilder()
.setFilePrefix(largeSpritesheetFilePrefix)
.setSpriteHeightPixels(72)
.setSpriteWidthPixels(128)
.setColumnCount(10)
.setRowCount(10)
.setTotalCount(100)
.build();
JobConfig config =
JobConfig.newBuilder()
.addInputs(Input.newBuilder().setKey("input0").setUri(inputUri))
.setOutput(Output.newBuilder().setUri(outputUri))
.addElementaryStreams(
ElementaryStream.newBuilder()
.setKey("video_stream0")
.setVideoStream(videoStream0))
.addElementaryStreams(
ElementaryStream.newBuilder()
.setKey("audio_stream0")
.setAudioStream(audioStream0))
.addMuxStreams(
MuxStream.newBuilder()
.setKey("sd")
.setContainer("mp4")
.addElementaryStreams("video_stream0")
.addElementaryStreams("audio_stream0")
.build())
.addSpriteSheets(smallSpriteSheet) // Add the spritesheet config to the job config
.addSpriteSheets(largeSpriteSheet) // Add the spritesheet config to the job config
.build();
var createJobRequest =
CreateJobRequest.newBuilder()
.setJob(
Job.newBuilder()
.setInputUri(inputUri)
.setOutputUri(outputUri)
.setConfig(config)
.build())
.setParent(LocationName.of(projectId, location).toString())
.build();
// Send the job creation request and process the response.
Job job = transcoderServiceClient.createJob(createJobRequest);
System.out.println("Job: " + job.getName());
}
}
}
The event trigger and the function should be properly configured.
Please, note that the transcoding job is asynchronous, you may need some additional work to read the results.
Although probably more complex, but based in this idea, this blog post and the companion one could be of help.