Search code examples
influxdbinfluxdb-python

InfluxDB: Export/Import large number of points form one instance to another


I have an instance of InfluxDB installed in one of our customers premise, that I don't have access to.

  • When certain problems are reported, I need to identify a window of time (before and after the problem) and provide them a script that can pull out all data in that window.
  • The script should generate a set of exported data that my customer will share with me.

I then need to import this data into my instance for analysis.

  • I tried to do this using backup/restore but that seems to fail with an error that the database exists.
  • I am now planning to use FOR each measurement: "influx -execute 'select * from ' -format csv -> measurement.txt

    And export all these files as a tar.gz. I still need to figure out how to import this data into my instance but that should not be an issue.

Is there a better way to do this?


Solution

  • Usually backup/restore works for your use case. But you cannot restore the values to already existing database.

    Backup: influxd backup -portable -database telegraf <path-to-backup>

    This is a way suggested by official docs of influxdb to restore into already existing db

    Restore the existing database backup to a temporary database.

    influxd restore -portable -db telegraf -newdb telegraf_bak path-to-backup

    Sideload the data (using a SELECT ... INTO statement) into the existing target database and drop the temporary database.

    USE telegraf_bak

    SELECT * INTO telegraf..:MEASUREMENT FROM /.*/ GROUP BY *

    DROP DATABASE telegraf_bak

    https://docs.influxdata.com/influxdb/v1.8/administration/backup_and_restore/#restore-examples