I've been given a very large .txt(csv) (100k+ entries) which is ftp'd to my server nightly then i'm using this query:
LOAD DATA LOCAL INFILE '/x/x/public_html/2013/fluid1/x3export/x3export.txt' REPLACE INTO TABLE x3export FIELDS TERMINATED BY '|'
I want the data in "x3export" to be updated, using replace. However, there isn't a unique field within the data, you have to combine two fields to create a unique id. (CatalogueRef + HierarchyCode) i've no idea how to do this however, as LOAD DATA from what i've read just uses pre-defined unique key or indexed columns and you can't(?) supply what it should replace contents on.
To try and explain a little further:
CatalogueRef for example, might be: 001ABC. HierarchyCode for example, might be: AAA
(don't ask me what these mean, i don't really know they come from another system which requires them.) There might be multiple rows with the same CatalogueRef however adding them together to make 001ABCAAA creates a unique id from what i'm told. Hopefully that can help?
Thanks
See the MySQL documentation here: http://dev.mysql.com/doc/refman/5.5/en/load-data.html
Near the end of the text the SET keyword is explained that allows you some manipulations with the values you're inserting, you might be able to make that into something useful, for example,
LOAD DATA LOCAL INFILE 'x3export.txt'
REPLACE INTO TABLE x3export
FIELDS TERMINATED BY '|'
(Column1, Column2, Column3, @CatalogueRef, @HierarchyCode)
SET ID = concat(@CatalogueRef, @HierarchyCode);