Search code examples
mysqldatabaseamazon-auroraarchiving

AWS Aurora MYSQL how to deal with growing table

I have a MySQL table that is growing quite fast and I was wondering what would be the best approach regarding ARCHIVING not needed data moving forward. The table has data that is 2 years old, but we only need the data for last year onwards. At the moment, the table has about 4 million rows and is 2.2GB in size.

DB specs:

Engine version

5.7.mysql_aurora.2.07.2

Instance class

db.r4.xlarge

vCPU

4

RAM

30.5 GB

Would anyone have any input in that regard?

Thank you

Solution

  • If the table were already partitioned by, say, month, archiving would be relatively efficient.

    In the absence of that prep work, I recommend:

    PARTITION BY RANGE(..)

    1. Create a new table that is partitioned; cf Partition
    2. Copy the data since a year ago into that table.
    3. Drop the current table
    4. Work on creating a regular monthly process involving "transportable tablespaces". Or, if you don't need to keep the old data, then plan on just DROP PARTITION (and add a new partition). (See link above.)

    Big DELETE

    If, instead, you choose to do something that involves DELETEing millions of rows, I strongly suggest chunking the operation: http://mysql.rjweb.org/doc.php/deletebig

    The above does not say where you will send the data you have removed from this main table. What is your plan for that?