I am using a C# Client class provided by the USPS EPF (Electronic Product Fulfillment) in order to download USPS files through a console application. I use my console app to login with my USPS credentials, specify the file I want to download, and get the file. All of this works great for the two smaller files we have access to (AMS Developer Kit, 47.7 MB, and DPV Developer Kit, 1.59 MB). However, when I try to download the 2.8 GB AMS Commercial DVD file, which is the only one I really care about, I run into problems. The console app stops downloading the file at 1.75 GB every time. Since it's a .tar file, I can open it and see some of the contents, but naturally a lot is missing. The USPS-provided Client class does not throw an exception or error of any kind; it's supposed to read to the end of the file, but it just stops early.
I have tried everything I can think of: altering the HttpWebRequest properties (changing KeepAlive to true, increasing the Timeout value), modifying the getEpfFile method to use an IsolatedStorageFile instead of a MemoryStream to get the file, even checking with our networking people to make sure there isn't some arbitrary network setting causing a timeout. I've tried downloading from my machine and different network servers with the same result. I looked into using WebClient instead, but that requires a parameter of the entire URL of the file to download, which is not known. I have to use HttpWebRequest, as far as I can tell, to access the USPS EPF files at all.
This is the getEpfFile method from the Client.cs class provided by USPS (I apologize for any formatting issues, this is my first post on the site):
// get actual file
public bool getEpfFile(String fileid)
{
bool downloadSuccess = true;
string strUrl = this.strBaseUrl + "/download/epf";
try
{
Console.WriteLine("Starting file download ...");
// add json to URL
Dictionary<string, string> json_value = new Dictionary<string, string>();
json_value.Add("logonkey", this.logon_key);
json_value.Add("tokenkey", this.token_key);
json_value.Add("fileid", fileid);
JavaScriptSerializer jsonSerializer = new JavaScriptSerializer();
string json_string = "obj=" + jsonSerializer.Serialize(json_value);
System.Text.ASCIIEncoding encoding = new System.Text.ASCIIEncoding();
Byte[] byteArray = encoding.GetBytes(json_string);
// set URL
Uri address = new Uri(strUrl);
// web request
HttpWebRequest request = WebRequest.Create(address) as HttpWebRequest;
request.UserAgent = "USPS .NET Sample";
request.KeepAlive = false;
request.Timeout = 100000;
request.ProtocolVersion = HttpVersion.Version10;
request.Method = "POST";
request.ContentLength = byteArray.Length;
request.ContentType = "application/x-www-form-urlencoded";
// add headers
// request.Headers.Add("Akamai-File-Request", filepath);
request.Headers.Add("logonkey", this.logon_key);
request.Headers.Add("tokenkey", this.token_key);
request.Headers.Add("fileid", fileid);
// post request
Stream dataStream = request.GetRequestStream();
dataStream.Write(byteArray, 0, byteArray.Length);
dataStream.Close();
// Get response
HttpWebResponse response = request.GetResponse() as HttpWebResponse;
if (request.HaveResponse == true && response != null)
{
Stream remoteStream = response.GetResponseStream();
Directory.CreateDirectory("c:\\Atemp");
Stream localStream = File.Create("c:\\Atemp\\dd.tar");
//byte[] buffer = new byte[2048];
byte[] buffer = new byte[1000000];
int bytesRead = 0;
do
{
// Read data (up to 1k) from the stream
bytesRead = remoteStream.Read(buffer, 0, buffer.Length);
// Write the data to the local file
localStream.Write(buffer, 0, bytesRead);
} while (bytesRead > 0);
int i = response.Headers.Count;
for (int x = 0; x < i; x++)
{
if (response.Headers.Keys[x].ToString() == "User-Tokenkey")
{
this.token_key = response.Headers[x].ToString();
}
else if (response.Headers.Keys[x].ToString() == "User-Logonkey")
{
this.logon_key = response.Headers[x].ToString();
}
else if (response.Headers.Keys[x].ToString() == "Service-Response")
{
Console.WriteLine("Web service result: " + response.Headers[x].ToString());
}
else if (response.Headers.Keys[x].ToString() == "Service-Messages")
{
Console.WriteLine("Resulting Messages: " + response.Headers[x].ToString());
}
}
// close resources
localStream.Close();
remoteStream.Close();
response.Close();
Console.WriteLine("File Download completed.");
}
}
catch (Exception ex)
{
downloadSuccess = false;
string str = ex.Message;
str += "";
}
return downloadSuccess;
}
Any insight as to why it keeps cutting off early would be massively appreciated.
Turned out that USPS had a Download Manager program the whole time which accomplishes what I'm trying to do 10 times easier. I was emailed the zip file directly by the USPS rep, but the documentation (including a download link) is available here if anyone needs it: https://ribbs.usps.gov/acs/documents/tech_guides/ACS_EPF_DownloadManagerTechnicalGuide.pdf