Search code examples
amazon-s3connectionqliksense

Export data from QlikSense cloud to AWS S3 bucket

I am trying to create a pipeline to export Qliksense app data to AWS S3 bucket, but not sure if I can do it directly.

Two options I tried:

  1. Use export API to export data as qvf to local disk, then connect to S3 via python script and push the data.

2.Store the data as csv using qliksense script locally and then push to S3.

Basically my ideal solution would be use a single python script to connect to Qliksense, read the data, convert to csv and export to S3.

Any ideas/code/approach would be helpful.

Solution

  • Qliksense is a tool to consume data and display in general, but when there is need

    STORE INTO vFile (text);

    is easy option. This way you will get csv file saved on drive. If you are running python script on the same machine is easy as you can EXECUTE statement which can run python script and store it to s3 using boto3 library.

    Otherwise just use network drive instead and then in cron put data from there via python to s3.

    Last option is that on qliksense server you can map s3 bucket. There are few tools for that like TnTDrive.

    So much depends on your infrastructure.