How to read numerous xml files from windows appication from URL (multiple requests to server from windows app) c#

Question

I have a windows form where a user will be able to download all currency rates for selected date period. I have this now:

    for (DateTime d = fromDatePicker.Value.Date; d <= toDatePicker.Value.Date; d.AddDays(1))
    {
        if (d.DayOfWeek == DayOfWeek.Saturday || d.DayOfWeek == DayOfWeek.Sunday)
                                return;

        string url = "http://cbar.az/currencies/" + d.ToString("dd.MM.yyyy", CultureInfo.InvariantCulture) + ".xml";
        XmlDocument doc = new XmlDocument();
        doc.Load(url);
        //other stuff
    }

URL format is like this depending on the date: http://cbar.az/currencies/15.07.2015.xml And for example if I select two week period, it gets rates for two days, then skips two days and etc. throwing an error not even reaching the end of the period:

remote server returned an error (503) server unavailable

I could guess that this is kind of server side protection against multiple client requests but do not know how to solve this problem.

It does not throw an error if I select period of 2 or 3 days. But here it also may not get rates for all dates as well.

I would appreciate your help. Thank you.

Here is my whole code:

for (DateTime d = fromDatePicker.Value.Date; d <= toDatePicker.Value.Date; d.AddDays(1))
                {
                    if (d.DayOfWeek == DayOfWeek.Saturday || d.DayOfWeek == DayOfWeek.Sunday)
                        continue;

                    string url = "http://cbar.az/currencies/" + d.ToString("dd.MM.yyyy", CultureInfo.InvariantCulture) + ".xml";

                    #region read rates for the date to the DataTable
                    XmlDocument doc = new XmlDocument();
                    doc.Load(url);

                    XmlElement root = doc.DocumentElement;
                    XmlNodeList nodes = root.SelectNodes("//ValCurs/ValType");

                    DataTable tempRates = new DataTable();

                    foreach (XmlNode node in nodes)
                    {
                        if (node.Attributes["Type"].Value == "Xarici valyutalar")
                        {
                            //create temp table and load new rates
                            tempRates.Clear();
                            tempRates.Columns.Add("Code");
                            tempRates.Columns.Add("Nominal");
                            tempRates.Columns.Add("Name");
                            tempRates.Columns.Add("Value");

                            foreach (XmlNode currency in node.ChildNodes)
                            {
                                DataRow dr = tempRates.NewRow();
                                dr["Code"] = currency.Attributes["Code"].Value;

                                foreach (XmlNode currencyDetailsNode in currency.ChildNodes)
                                {
                                    dr[currencyDetailsNode.Name] = currencyDetailsNode.InnerText;
                                }

                                tempRates.Rows.Add(dr);
                            }
                        }
                    }
                    #endregion

                    DAL dal = new DAL();
                    dal.ClearCurrentRates(d);

                    //insert new values
                    foreach (DataRow currencyRow in StaticValues.dataSet.Tables["Currencies"].Rows)
                    {
                        if (currencyRow["Code"].ToString() == "AZN")
                        {
                            #region Insert the row for AZN
                            try
                            {
                                SqlParameter[] pars = new SqlParameter[3];

                                pars[0] = new SqlParameter("@Date", SqlDbType.Date);
                                pars[0].Value = d.ToShortDateString();

                                pars[1] = new SqlParameter("@CurrencyID", SqlDbType.Int);
                                pars[1].Value = currencyRow["ID"].ToString();

                                pars[2] = new SqlParameter("@Rate", SqlDbType.Decimal);
                                pars[2].Value = 1.0000;

                                dal.InsertData("CurrencyRates", pars);
                            }
                            catch (Exception ex)
                            {
                                StaticValues.WriteEventLogXML(ex, this.Text);
                                switch (StaticValues.user.Language)
                                {
                                    case "English":
                                        MessageBox.Show("Database error", Application.ProductName, MessageBoxButtons.OK, MessageBoxIcon.Error);
                                        break;
                                    case "Russian":
                                        MessageBox.Show("Ошибка базы данных", Application.ProductName, MessageBoxButtons.OK, MessageBoxIcon.Error);
                                        break;
                                    case "Azeri":
                                        MessageBox.Show("Məlumat bazası səhvi", Application.ProductName, MessageBoxButtons.OK, MessageBoxIcon.Error);
                                        break;
                                    default:
                                        break;
                                }
                            }
                            #endregion
                            continue;
                        }
                        foreach (DataRow tempRow in tempRates.Rows)
                        {
                            if (tempRow["Code"].ToString() == currencyRow["Code"].ToString())
                            {
                                #region Insert the row
                                try
                                {
                                    SqlParameter[] pars = new SqlParameter[3];

                                    pars[0] = new SqlParameter("@Date", SqlDbType.Date);
                                    pars[0].Value = d.ToShortDateString();

                                    pars[1] = new SqlParameter("@CurrencyID", SqlDbType.Int);
                                    pars[1].Value = currencyRow["ID"].ToString();

                                    pars[2] = new SqlParameter("@Rate", SqlDbType.Decimal);
                                    pars[2].Value = decimal.Parse(tempRow["Value"].ToString());

                                    dal.InsertData("CurrencyRates", pars);
                                    break;
                                }
                                catch (Exception ex)
                                {
                                    StaticValues.WriteEventLogXML(ex, this.Text);
                                    switch (StaticValues.user.Language)
                                    {
                                        case "English":
                                            MessageBox.Show("Database error", Application.ProductName, MessageBoxButtons.OK, MessageBoxIcon.Error);
                                            break;
                                        case "Russian":
                                            MessageBox.Show("Ошибка базы данных", Application.ProductName, MessageBoxButtons.OK, MessageBoxIcon.Error);
                                            break;
                                        case "Azeri":
                                            MessageBox.Show("Məlumat bazası səhvi", Application.ProductName, MessageBoxButtons.OK, MessageBoxIcon.Error);
                                            break;
                                        default:
                                            break;
                                    }
                                    break;
                                }
                                #endregion
                            }
                        }
                    }
                    d = d.AddDays(1);
                    Thread.Sleep(1000);
                }

Answer 1

Your stuff is completely wrong.

1. method AddDays(x) don't update your "d" variable, so construction of

   for (DateTime d = fromDatePicker.Value.Date; d <= toDatePicker.Value.Date; d.AddDays(1))

produces endless loop

2.

if (d.DayOfWeek == DayOfWeek.Saturday || d.DayOfWeek == DayOfWeek.Sunday)
                            return;

exits the loop completely.

3. it seems, that the remote server has a some kind of performance problem with many requests at a short time. It can be solved by some pause between requests (for example: Thread.Sleep(2000) - two seconds pause)

So, your code would look like this:

for (DateTime d = fromDatePicker.Value.Date; d <= toDatePicker.Value.Date; d = d.AddDays(1))
{
    if (d.DayOfWeek == DayOfWeek.Saturday || d.DayOfWeek == DayOfWeek.Sunday)
                            continue;

    string url = "http://cbar.az/currencies/" + d.ToString("dd.MM.yyyy", CultureInfo.InvariantCulture) + ".xml";
    XmlDocument doc = new XmlDocument();
    doc.Load(url);

            Thread.Sleep(2000);
}