Having trouble reading http response into input stream

Question

I call a .Net web service from my Blackberry app. After having nothing but trouble with KSoap2 I decided to do everything manually. Here is a snippet of code:

byte [] postDataBytes = soapRequestEnvelope.getBytes();
byte [] dataRetrieved;

    try
    {
        HttpConnection connection = (HttpConnection)Connector.open(URL);

        connection.setRequestMethod(HttpConnection.POST);
        connection.setRequestProperty("soapaction", soapAction);
        connection.setRequestProperty("Content-Type", "text/xml; charset=utf-8");

        os = connection.openOutputStream();
        os.write(postDataBytes);

        int rc = connection.getResponseCode();

        if(rc == HttpConnection.HTTP_OK)
        {
            inputStream = connection.openInputStream();
            dataRetrieved = new byte[(int)connection.getLength()];
            int bytesRead = inputStream .read(dataRetrieved);

        }
        else
        {

            dataRetrieved = null;

        }


        String dataString = new String(dataRetrieved);

//HttpConnection = javax.microedition.io.HttpConnection

//inputStream = java.io.InputStream

The problem I'm having is that the XML I get back from the web service call gets cut short.

When I print the number of received bytes to the screen it says 1170 sometimes (which is actually the response size I'm expecting in this particular situation.)

But other times the received bytes is 702...... why 702???

I've tested about a dozen times in a row and got the following result:

1170, 702, 1170, 1170, 702, 1170, 1170, 702, 1170, 702, 1170, 702, 702, 702, 1170, 702

But why 702?? When it messes up and doesn't work why is it so consistent?? It's always allocating 1170 bytes though, but why does it sometimes only read 702??

This is very strange.

EDIT: I tried printing inputStream.available() to the screen as well for comparison, totally inconsistent. It varies between 0, 702 and 1170. Sometimes the available bytes is 0 or 702 and the bytes read is 1170. I'm totally confused.

Any help would be greatly appreciated.

Thank you

Answer 1

After my sort of vague comment, I suppose I should specify what I meant by "put that read in a loop". As defined, read will return the number of bytes read, or -1 if the end of file is reached.

When reading from a socket, you may be calling read() before all of the data has been returned to you from the server. The practical effect of this is that you call read() and you get back less data than you're expecting. This simply means you need to call it again (and again, and again) until you get all the data.

I would suggest using a simple fixed size buffer and looping through read() until you get -1 and doing something with each chunk (writing it to a file, for example.) This prevents you from needing to have the entire file in memory. For example:

byte buf[1024];
int len;

while((len = read(buf)) >= 0)
{
    /* do something with len bytes of buf */
}

Of course, you may have a guarantee that the file will be small and you really do want the whole thing in memory. In that case, you can (as you did above) just allocate a buffer of connection.getLength() size, however you'll need to call the 3-arg read(byte[], int, int) and do a little more housekeeping:

int remain = (int) connection.getLength();
byte buf[remain];
int offset = 0, len;

while((len = read(buf, offset, remain)) >= 0)
{
    offset += len;
    remain -= len;
}