Mocking GetObjectCommand response using v3 of the SDK using file system object

Question

I have a method which pulls from s3 and processed the gzipped file that is retrieved. I am attempting to write a test which mocks the retrieval (using a file read from the FS) and can then test the processing of the file.

The aws-sdk-client-mock is definitely the way to go an I have used it in other places, but having difficulty here the resolving the stream types.

import {mockClient} from "aws-sdk-client-mock";

const s3ClientMock = mockClient(S3Client);

test("getJSONFroms3Item returns data correctly with a gzip file from s3", async () => {
    let stream: ReadStream = fs.createReadStream('../resources/persons.json.gz');
    s3ClientMock.on(GetObjectCommand).resolves({ Body: stream});
    const result =  await getJSONFroms3Item();
    expect(s3ClientMock).toHaveReceivedCommandWith(GetObjectCommand, {});
    expect (result).toContain({name: "Jane Doe", address: "happytown", lastUpdated: 1657116641655, isActive: true, status:"ACTIVE"})
});

The error I am getting is

   __test__/src/S3.test.ts:22:54 - error TS2322: Type 'ReadStream' is not assignable to type 'SdkStream<Readable | ReadableStream<any> | Blob | undefined> | undefined'.
  Type 'ReadStream' is not assignable to type 'Readable & SdkStreamMixin'.

I have spent a long time trying to figure out how to get a Readable from the filesystem, but no luck.

I used the answer provided https://stackoverflow.com/a/72196278/10917982 to give me guidance, but can't see to get it working.

sandbox here https://taramcc-ominous-space-umbrella-7vq7j764xj9f799.github.dev/

Any help much appreciated.

Answer 1

Since version 3.188.0 (Oct 22), the S3 client support util functions to consume and parse the response streams (something that we would have to do manually before).

To mock the response, you need to wrap it with the util function sdkStreamMixin()

So now, you need to do this

import { mockClient } from "aws-sdk-client-mock";
import { sdkStreamMixin } from '@aws-sdk/util-stream-node';

const s3ClientMock = mockClient(S3Client);

test("getJSONFroms3Item returns data correctly with a gzip file from s3", async () => {
    const stream = sdkStreamMixin(fs.createReadStream('../resources/persons.json.gz'))
    s3ClientMock.on(GetObjectCommand).resolves({ Body: stream});
    ...
});

Update Jan 28th, 2024

The @aws-sdk/util-stream-node package is deprecated and we should use the sdkStreamMixin() function from @smithy/util-stream instead

import {sdkStreamMixin} from '@smithy/util-stream';