ADF Copy activity - i have two types of flat files stored in the same directory, i am reading it with a wildcard *.txt
SOURCE type 1 has 18 columns, type 2 has 24 columns. the first 18 columns are identical, type 2 only has an additional 6 cols.
SINK azure sql table with 24 columns
Is it possible to use an IF condition to count the columns, and pass each source file into the correct copy activity - I would then have 2 copy activities (1 for each type of file).
You can use GetMetaData activity to get the structure of file and even count of columns. And then via If activity, split them for 2 different copy activities or else use dynamic mapping based on if condition leveraging only 1 copy activity
https://sqlitybi.com/dynamically-set-copy-activity-mappings-in-azure-data-factory-v2/