Is there a way to process multiple xml feeds asynchoronously?

Question

I am using Atom10FeedFormatter class for processing atom xml feeds calling OData Rest API endpoint.

It works fine, but the api gives the result slow, if there are more than 200 entries in the feed.

That is what I use:

        Atom10FeedFormatter formatter = new Atom10FeedFormatter();

        XNamespace d = "http://schemas.microsoft.com/ado/2007/08/dataservices";

        string odataurl= "http://{mysite}/_api/ProjectData/Projects";

        using (XmlReader reader = XmlReader.Create(odataurl))
        {
             formatter.ReadFrom(reader);
        }

         foreach (SyndicationItem item in formatter.Feed.Items)
         {
              //processing the result
         }

I want to speed up this process at least a little faster by splitting the original request to query the results skipping some entries and limiting entry size.

The main idea is count the number of feeds using $count, divide the feed results into blocks of 20, use the $skip and $top in the endpoint url, iterate through the results, and finally summarize them.

int countoffeeds = 500; // for the sake of simplicity, of course, i get it from the odataurl using $count
int numberofblocks = (countoffeeds/20) + 1;

for(int i = 0; i++; i<numberofblocks){
    int skip = i*20;
    int top = 20;
    string odataurl = "http://{mysite}/_api/ProjectData/Projects"+"?&$skip="+skip+"&top=20";
    Atom10FeedFormatter formatter = new Atom10FeedFormatter();


    using (XmlReader reader = XmlReader.Create(odataurl))
    {
         formatter.ReadFrom(reader); // And this the part where I am stuck. It returns a void so I 
                                     //cannot use Task<void> and process the result later with await 
    }
...

Normally I would use async calls to the api (this case numberofblocks = 26 calls in parallel), but I do not know how would I do that. formatter.ReadFrom returns void, thus I can not use it with Task.

How can I solve this, and how can I read multiple xml feeds at the same time?

Answer 1

Normally I would use async calls to the api (this case numberofblocks = 26 calls in parallel), but I do not know how would I do that. formatter.ReadFrom returns void, thus I can not use it with Task.

Atom10FeedFormatter is a very dated type at this point, and it doesn't support asynchrony. Nor is it likely to be updated to support asynchrony.

How can I solve this, and how can I read multiple xml feeds at the same time?

Since you're stuck in the synchronous world, you do have the option of using "fake asynchrony". This just means you would do the synchronous blocking work on a thread pool thread, and treat each of those operations as though they were asynchronous. I.e.:

var tasks = new List<Task<Atom10FeedFormatter>>();
for(int i = 0; i++; i<numberofblocks) {
  int skip = i*20;
  int top = 20;
  tasks.Add(Task.Run(() =>
  {
    string odataurl = "http://{mysite}/_api/ProjectData/Projects"+"?&$skip="+skip+"&top=20";
    Atom10FeedFormatter formatter = new Atom10FeedFormatter();
    using (XmlReader reader = XmlReader.Create(odataurl))
    {
      formatter.ReadFrom(reader);
      return formatter;
    }
  }));
}
var formatters = await Task.WhenAll(tasks);