Status code 400: NSHTTPURLResponse on calls made using URLSession.shared.dataTask

Question

This is an issue that I started having after I updated my iOS 11 (15A5304i) and Xcode 9 (9M137d) to the latest beta.

When I revert back to the previous beta, it goes away.

Making any GET https calls to a website results in a 400 response from the servers.

networkError(FoodAPI.NetworkError.responseStatusError(status: 400, message: "<NSHTTPURLResponse: 0x1c4239700> { URL: https://stackoverflow.com/ } { status code: 400, headers {\n    Date = \"Thu, 22 Jun 2017 17:41:03 GMT\";\n} }"))

This is how I'm making the call

func callTask(_ request: URLRequest, completion: @escaping (ResultType<Data, NetworkError>) -> Void) {
  let task = URLSession.shared.dataTask(with: request) { data, response, error in

    guard let response = response as? HTTPURLResponse else {
      completion(.failure(.responseInvalidFormat))
      return
    }

    if response.statusCode >= 400 {
      let error = response.description
      completion(.failure(.responseStatusError(status: response.statusCode, message: error)))
      return
    }

    guard let data = data else {
      completion(.failure(.responseNoDataError))
      return
    }

    completion(.success(data))
  }
  task.resume()
}

and this is how the request is generated:

  private func generateRequest(urlString: String,
                               method: HttpMethod = .get,
                               contentType: ContentType = .json,
                               headers: [String: String] = ["":""],
                               body: [String: Any]? = nil) throws -> URLRequest {


    guard let url = URL(string: urlString) else {
      throw NetworkError.requestGenerationFail(url: urlString, httpMethod: method)
    }
    var request = URLRequest(url: url)
    request.httpMethod = method.rawValue
    request.setValue("***** by Reza: info: reza@example.com", forHTTPHeaderField: "User-Agent")

    request.allHTTPHeaderFields = headers
    if contentType == .urlEncoded && headers["content-type"] ?? "" != ContentType.urlEncoded.rawValue {
      request.addValue(ContentType.urlEncoded.rawValue, forHTTPHeaderField: "content-type")
    }

    if let body = body {
      request.httpBody = try self.generateBody(body: body, contentType: contentType)
    }

    return request
  }

Again, this works fine on the previous iOS 11 beta, however the new one results in a 400 response on https calls to web servers, with little to no data.

It's worth noting that I first make a call to Yelp's endpoint API which is hosted on https://api.yelp.com/. This connection succeeded without any issue. Any further calls to other host (https://stackoverflow.com, https://www.yelp.com) comes back with a 400.

When I run the code in simulator using the old beta, it works fine, when I deploy to my iPhone 6 that uses the new iOS it results in 400 from servers.

I haven't tested this against a server I own so I can view logs, but something along the way is breaking and causing servers to comeback with a bad request response. These same calls work fine using curl, postman, browsers and previous iOS builds.

I was hoping someone who has faced the same issue could shed some light.

Answer 1

I have pinpoint the issue. Although the reason is still elusive.

In the generateRequest function I pass a dictionary of [String:String] with default values ["":""] for the request headers.

It appears that request.allHTTPHeaderFields = ["":""] will break your request.

In prior builds this did not cause any issues but this will result in a bad request on the current beta.

Update

I filled a bug with Apple when I first came across this issue and this was their response:

The problem was that in CFNetwork-856.4.7 (part of seed 1) we made a mistake and removed headers that were blank. In Seed 2, (CFNetwork-861.1) we limited the headers we would remove to 4 “special” headers (Content-Type, User-Agent, Accept-Language, Accept-Encoding). So, that’s why you saw the change in behavior.

macOS El Capitan also sends the “=“ header that results the 400 HTTP status code. So, this is behaving correctly now in Seed 2. Seed 1 was an anomaly and was wrong.

Please let us know whether the issue is resolved for you by updating your bug report.