Loading last 7 days files with specific names in SSIS

Question

I have a folder which gets a daily dump of 7 or 8 .xls files. Among all those files, I just need to load 2 files with a specific name.. ABC_datestamp.xls and XYZ_datestamp.xls.

The problem is I am not able to isolate these 2 file names because there are so many other files in the same folder. The second challenge is that my manager wants me to load all files from last week at once, which means I have to specifically load files from last 7 days only.

Right now I am copying those specific 7 files in to a folder and then loading it using a for each loop container.

Is there anyway, I can use a script task to generate the file names like ABC_02-Dec-2011.xls, ABC_01-Dec-2011.xls, ABC_30-Nov-2011.xls .......last 7 days files and pass it as a variable to a DATA FLOW TASK?

Any better suggestions ?

I know a lot of people have asked similar questions but I am new to SSIS and not able to find good answers.

Thanks a lot..

Answer 1

Rather than try and build a list of all possible file names, I'd loop through all the files in the folder and use a Script Task to determine which files to process further.

The control flow would look like this:

with FELC_InputFolderContents's variable mappings set thusly:

SCR_SetProcessFileFlag's configuration:

SCR_SetProcessFileFlag's script like so:

    public void Main()
    {
        string targetFileFullPath = (string)Dts.Variables["User::TargetFileFullPath"].Value;
        string targetFileName = Path.GetFileName(targetFileFullPath);
        Dts.Variables["ProcessFileFlag"].Value = IsValidFileName(targetFileName);
        Dts.TaskResult = (int)ScriptResults.Success;
    }

    private bool IsValidFileName(string targetFileName)
    {
        bool result = false;
        if (Path.GetExtension(targetFileName) == "xls")
        {
            string fileNameOnly = Path.GetFileNameWithoutExtension(targetFileName);
            if (fileNameOnly.Substring(0, 4) == "ABC_" || fileNameOnly.Substring(0, 4) == "XYZ_")
            {
                string datePart = fileNameOnly.Substring(4);
                DateTime dateRangeStart = DateTime.Today.AddDays(-7);
                DateTime dateRangeEnd = DateTime.Today;
                DateTime fileDate;
                if (DateTime.TryParse(datePart, out fileDate)) 
                {
                    if (dateRangeStart <= fileDate && fileDate <= dateRangeEnd)
                    {
                        result = true;
                    }
                }
            }
        }
        return result;
    }

and the precedence constraint between SCR_SetProcessFileFlag and DTT_ProcessSelectedFile configured thusly:

(Obviously, the specific logic for IsValidFileName depends on your particular requirements.)