Search code examples
c#asynchronousreadlinetxt

Why do I only get results from my first txt?


I am trying to read & use all lines from txt files. With a method I iterate through them asinc, and I try to get the data. My problem is, the output looks like it only contains the data from the first txt file. I just cant find where the problem is. I would appreciate any help.

Here's my code:

string[] files = Directory.GetFiles("C:/DPS-EDPWB05/forlogsearch", "*", SearchOption.AllDirectories);

//data I need from the txts
            List<string> num = new List<string>();
            List<string> date = new List<string>();
            List<string> time = new List<string>();
            List<string> sip = new List<string>();
            List<string> csmethod = new List<string>();
            List<string> csuristem = new List<string>();
            List<string> csuriquery = new List<string>();
            List<string> sport = new List<string>();
            List<string> csusername = new List<string>();
            List<string> cip = new List<string>();
            List<string> csuseragent = new List<string>();
            List<string> csreferer = new List<string>();
            List<string> scstatus = new List<string>();
            List<string> scsubstatus = new List<string>();
            List<string> cswin32status = new List<string>();
            List<string> timetaken = new List<string>();

        int x = 0;
        int y = 0;
        int i = 0;
        int filesCount = 0;
        string v = "";

        //Taking the data from the Log, getting a list of string[] 
        //items with the lines from the txts

        List<string[]> lines = new List<string[]>();

        while (i < files.Length)
        {
            lines.Add(ReadAllLinesAsync(files[i]).Result);
            i++;
        }

        //Trying to get the data from the string[]s
        do
        {
            string line;
            int f = 0;
            string[] linesOfTxt = lines[filesCount];

            do
                {
                line = linesOfTxt[f];

                string[] splittedLine = { };

                    splittedLine = line.Split(' ', 15, StringSplitOptions.None);
                        
                    y = splittedLine.Count();

                    if (y == 15)
                    {
                        num.Add(x.ToString());
                        date.Add(splittedLine[0]);
                        time.Add(splittedLine[1]);
                        sip.Add(splittedLine[2]);
                        csmethod.Add(splittedLine[3]);
                        csuristem.Add(splittedLine[4]);
                        csuriquery.Add(splittedLine[5]);
                        sport.Add(splittedLine[6]);
                        csusername.Add(splittedLine[7]);
                        cip.Add(splittedLine[8]);
                        csuseragent.Add(splittedLine[9]);
                        csreferer.Add(splittedLine[10]);
                        scstatus.Add(splittedLine[11]);
                        scsubstatus.Add(splittedLine[12]);
                        cswin32status.Add(splittedLine[13]);
                        timetaken.Add(splittedLine[14]);

                        x++;
                    }
                
                f++;

                } while (f < linesOfTxt.Length);
            filesCount++;
        }
        while (filesCount < files.Count());

After all this I group these and stuff but that happens AFTER the lists of data i need are filled - so the problem must be here somewhere. Also, my asinc reader (I found here on stackoverflow):

public static Task<string[]> ReadAllLinesAsync(string path)
        {
            return ReadAllLinesAsync(path, Encoding.UTF8);
        }

        public static async Task<string[]> ReadAllLinesAsync(string path, Encoding encoding)
        {
            var lines = new List<string>();

            // Open the FileStream with the same FileMode, FileAccess
            // and FileShare as a call to File.OpenText would've done.
            using (var stream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, DefaultBufferSize, DefaultOptions))
            using (var reader = new StreamReader(stream, encoding))
            {
                string line;
                while ((line = await reader.ReadLineAsync()) != null)
                {
                    lines.Add(line);
                }
            }

            return lines.ToArray();
        }

Solution

  • There are several problems with the example code, thou I'm unsure what is responsible for the actual issue.

    1. Instead of writing your own ReadAllLinesAsync, just use File.ReadAllLinesAsync
    2. You should in general avoid .Result since this is a blocking operation, and this has the potential to cause deadlocks. In this case you should just call the synchronous version, File.ReadAllLines, instead.
    3. When possible, use foreach or for loops. They make it much easier to see if your code is correct, and avoids spreading the loop logic over multiple lines.
    4. As far as I can see, you are processing each line identically, so you should be able to just merge all lines from all files by using List<string> lines and AddRange
    5. Instead of keeping 15 different lists of properties, a more common approach would be to store one list of objects of a class with 15 different properties.
    6. Whenever you need to store data, you should seriously consider using an existing serialization format, like json, xml, csv, protobuf etc. This lets you use existing, well tested, libraries for writing and reading data and converting it to your own types.