Retrieve files from SharePoint Online/365

Question

I'm new to SharePoint and I do realize that there are several posts about this subject and I have also read through them but I'm still not sure how to go about this.

I have attempted to achieve this using CSOM .NET (actually I preferred to use the REST API, but tho whole Azure registration thing proved to be a study of it's own, so I went with managed code instead).

Anyway, there is a SP site that I know the path to "https://(server)/sites/Pictures/". This place (site, list, folder or whatever it's called) contains some images that I would like to be able to list and potentially download.

Doing the following gives me the Name: "Pictures", the Count: 393 and the ServerRelativeUrl: "/sites/Pictures/Pictures".

    using (ClientContext ctx = new ClientContext(site_url))
    {
        ctx.Credentials = new SharePointOnlineCredentials(usr, secure_pwd);

        Web web = ctx.Web;

        List lst_pictures = web.Lists.GetByTitle("Pictures");

        ctx.Load(lst_pictures.RootFolder);

        ctx.ExecuteQuery();

        Console.WriteLine("Name: " + lst_pictures.RootFolder.Name);
        Console.WriteLine("ItemCount: " + lst_pictures.RootFolder.ItemCount);
        Console.WriteLine("ServerRelativeUrl: " + lst_pictures.RootFolder.ServerRelativeUrl);
    }

From here on I have tried several things to list and loop through the files, but whatever I try, I keep getting the same exception saying that the collection hasn't been initialized and I can't seem to get it right.

Then I have tried to see if I could somehow get help using both U2U CAM Query Builder and CAML Designer for SharePoint, but neither of these tools display the Pictures site (or list or folder) that I'm looking for.

I'm thinking that this ought to be pretty straight forward for someone who is used to the not so intuitive way of querying SP for resources via CSOM.

So, can anyone guide me as to what I do from here, so get the files listed in a way so that they can be downloaded (the link to the file should do in that regard).

Any help is appreciated, thanks.

UPDATE 1

I added the lines you suggested and I'm pretty sure that I have already had those lines in my code during previous tests, but I removed them again because it didn't work out for me - and it still doesn't :( See the image here.

// lines not shown in the image
ctx.Load(lst_pictures.RootFolder);
ctx.Load(lst_pictures.RootFolder.Files);

ctx.ExecuteQuery();

Console.WriteLine("Name: " + lst_pictures.RootFolder.Name);
Console.WriteLine("ItemCount: " + lst_pictures.RootFolder.ItemCount);
Console.WriteLine("ServerRelativeUrl: " + lst_pictures.RootFolder.ServerRelativeUrl):

FileCollection files = lst_pictures.RootFolder.Files;

The first part works though, but it did that before changing the code.

UPDATE 2

UPDATE 3 Updated code according to comments, still no go though.

UPDATE 4: the solution

Well it turns out that the actual problem was the nuget package installed called Microsoft.SharePoint.Client v.14.xxxx. It should have been the one just called SharePoint.Client v.15.xxxx (this one has no description but comes from Microsoft).

So, thanks to Adam for trying to help me, I will mark your answer as correct since it finally led me to the problem so it could be fixed. Installing the correct package fixed the issue right away.

Correct package is this one.

UPDATE 5: New subject, custom fields/columns (still using CSOM)

1) I have the following code at the moment, but I need to get some extra properties (custom ones) from the files - what should I add to the code to do this? 2) And should I be using ListItem and not File in this case? 3) And could I even add filtering on custom fields/columns so to only retrieve a subset of the files?

        using (ClientContext ctx = new ClientContext("<url>"))
        {
            ctx.Credentials = new SharePointOnlineCredentials("<usr>", "<pwd>");

            Web web = ctx.Web;

            lst = web.Lists.GetByTitle("<list name>");

            ctx.Load(lst);
            ctx.Load(lst.RootFolder);
            ctx.Load(lst.RootFolder.Folders);

            ctx.ExecuteQuery();

            files = lst.RootFolder.Files;

            foreach (File f in files)
            {
                // doing stuff with some attributes like UniqueId, Name and ServerRelativeUrl, but need custom attributes as well.
            }
        }

Answer 1

Please load also the files of the rootFolder into the context, so like:

    ctx.Load(lst_pictures.RootFolder.Files);

and then You can get the collection of files in root folder of the library like:

    FileCollection fileCollection = lst_pictures.RootFolder.Files;

the complete code based on Your code is like:

    using (ClientContext ctx = new ClientContext(site_url))
    {
        ctx.Credentials = new SharePointOnlineCredentials(usr, secure_pwd);

        Web web = ctx.Web;

        List lst_pictures = web.Lists.GetByTitle("Pictures");

        ctx.Load(lst_pictures.RootFolder);

        // Load files of root folder to context
        ctx.Load(lst_pictures.RootFolder.Files);

        // after execute on context the rootFolder will be populated with file collection
        ctx.ExecuteQuery();

        Console.WriteLine("Name: " + lst_pictures.RootFolder.Name);
        Console.WriteLine("ItemCount: " + lst_pictures.RootFolder.ItemCount);
        Console.WriteLine("ServerRelativeUrl: " + lst_pictures.RootFolder.ServerRelativeUrl);

        // get the file collection of the rootfolder
        FileCollection fileCollection = lst_pictures.RootFolder.Files;
        foreach (File file in fileCollection)
        {
            Console.WriteLine($"file : {file.Name} ");
        }
    }

the output of the code above for my example library was like this:

Please be aware

that if You plan to use folders in this library and put some files in does folders this approach is not enough :). You will need to first load lst_pictures.RootFolder.Folders to context, then execute, after that foreach the collection of folders and for each folder You will need to load the files collection of this folder to get the files. so to sum up You will need to:

    ctx.Load(lst_pictures.RootFolder);
    ctx.Load(lst_pictures.RootFolder.Folders);
    ctx.ExecuteQuery();

and then

    FolderCollection folderCollection = lst_pictures.RootFolder.Folders;
    foreach (Folder folder in folderCollection)
    {
           ctx.Load(folder.Files);
           ctx.ExecuteQuery();
           FileCollection fileCollection = folder.Files;
           foreach (File file in fileCollection)
           {
                 Console.WriteLine($"file : {file.Name} ");
           }
    }

but if You plan only to keep files in the root folder of the library the first code should be what You are looking for. I hope this post is of any help :)

P.S. Welcome to SharePoint.

UPDATE 1

If You would like to include custom columns in the files collection You need to include them. You can either include all custom columns of files like this

    ctx.Load(list.RootFolder);
    ctx.Load(list.RootFolder.Folders);
    ctx.Load(list.RootFolder.Files, includes => includes.Include(i => i.Name, i => i.ListItemAllFields));
    ctx.ExecuteQuery();

and then You have the list of all columns also the custom ones. So You can do a foreach like:

    FileCollection fileCollection = list.RootFolder.Files;
    foreach (File file in fileCollection)
    {
          Console.WriteLine($"file : {file.Name} ");
          Console.WriteLine($"file : {file.ListItemAllFields["SomeCustomColumn"]} ");
     }

to get only the needed custom columns You can include them one by one like

    ctx.Load(list.RootFolder.Files, includes => includes.Include(i => i.Name, i => i.ListItemAllFields["CustomColumn1"], i => i.ListItemAllFields["CustomColumn2"]));

and then for this file collection You should have only the Name, and the two custom columns in the collection.

Please be also aware that the names of this columns are the internal field names (or static names) not the display name, so SharePoint for example can change the space to x0020. So For column display name like "Custom Column 1" the internal name is "Custom_x0020_Column_x0020_1" and this is the name You should use. You can get this name by going to the column settings page (edit page) and You will see the name in the url in the parameter Field=....