I am trying to make a connection with the following URL: https://www.popularmechanics.com/rss/science/?src=rss. In my browser, everything seems fine. However, when I try to retrieve the URL using .get()
function from the http
package. I get the following exception: ClientException: Failed to parse HTTP, 115 does not match 13, uri=https://popularmechanics.com/rss/science/?src=rss
.
Here is a simplified version of my code:
import 'package:http/http.dart' as http;
void main() {
final client = http.Client();
final Uri url = Uri.https('popularmechanics.com', '/rss/science/', {
'src': 'rss'
});
try {
client.get(url);
} catch (e) {
print(e);
}
}
Running the code will give the following:
Unhandled exception:
ClientException: Failed to parse HTTP, 115 does not match 13, uri=https://popularmechanics.com/rss/science/?src=rss
#0 IOClient.send (package:http/src/io_client.dart:121:7)
<asynchronous suspension>
#1 BaseClient._sendUnstreamed (package:http/src/base_client.dart:93:32)
<asynchronous suspension>
Process finished with exit code 255
I thought that it might have something to do with the http
package so I tried the dio
package as well. However, the same problem still persists:
import 'package:dio/dio.dart';
void main() {
final client = Dio();
try {
client.get('https://popularmechanics.com/rss/science/', queryParameters: {
'src': 'rss'
});
} catch (e) {
print(e);
}
}
Running the code will give the following:
Unhandled exception:
DioException [unknown]: null
Error: HttpException: Failed to parse HTTP, 115 does not match 13, uri = https://popularmechanics.com/rss/science/?src=rss
#0 DioMixin.fetch.<anonymous closure> (package:dio/src/dio_mixin.dart:507:7)
#1 _FutureListener.handleError (dart:async/future_impl.dart:180:22)
#2 Future._propagateToListeners.handleError (dart:async/future_impl.dart:858:47)
#3 Future._propagateToListeners (dart:async/future_impl.dart:879:13)
#4 Future._completeError (dart:async/future_impl.dart:655:5)
#5 _SyncCompleter._completeError (dart:async/future_impl.dart:63:12)
#6 _Completer.completeError (dart:async/future_impl.dart:27:5)
#7 Future.any.onError (dart:async/future.dart:618:45)
#8 _RootZone.runBinary (dart:async/zone.dart:1666:54)
#9 _FutureListener.handleError (dart:async/future_impl.dart:177:22)
#10 Future._propagateToListeners.handleError (dart:async/future_impl.dart:858:47)
#11 Future._propagateToListeners (dart:async/future_impl.dart:879:13)
#12 Future._completeError (dart:async/future_impl.dart:655:5)
#13 Future._asyncCompleteError.<anonymous closure> (dart:async/future_impl.dart:745:7)
#14 _microtaskLoop (dart:async/schedule_microtask.dart:40:21)
#15 _startMicrotaskLoop (dart:async/schedule_microtask.dart:49:5)
#16 _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:118:13)
#17 _Timer._runTimers (dart:isolate-patch/timer_impl.dart:405:11)
#18 _Timer._handleMessage (dart:isolate-patch/timer_impl.dart:429:5)
#19 _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:184:12)
Process finished with exit code 255
I am trying to find out what 115
and 13
mean. Because I am struggling to find a solution to this problem. So if you know what these numbers mean I would greatly appreciate an explanation. Of course, a solution to the problem would be even better.
The server is sending a rarely used HTTP header called 'Trailer'.
Connection: keep-alive
Content-Type: text/xml; charset=UTF-8
Expires: Sun, 26 Nov 2023 23:05:57 GMT
Pragma: public
Referrer-Policy: no-referrer-when-downgrade
Accept-Ranges: bytes
Date: Sun, 26 Nov 2023 23:08:39 GMT
Age: 462
X-Cache: HIT, HIT
Server-Timing: time-start-msec;dur=1701040119607,time-elapsed;dur=14,fastly-pop;desc=NYC,hit-state;desc=HIT-CLUSTER
set-cookie: location_data={"country_code":"US","postal_code":"redacted","geo_region":"redacted"}; path=/;
x-robots-tag: all
x-country: US
strict-transport-security: max-age=31557600; includeSubDomains
Cache-Control: max-age=0, must-revalidate, private
alt-svc: h3=":443";ma=86400,h3-29=":443";ma=86400,h3-27=":443";ma=86400
transfer-encoding: chunked
trailer: server-timing
At the end of the body, it includes the data for this trailing header.
server-timing: rtt; dur=38.4, retrans; dur=0, trailer-timestamp; dur=1701040119621
The Dart HttpClient isn't expecting this appended trailer (or it is malformed). Note how the first character is s
from server
and this is the 115
it says isn't the 13
it is expecting.
I wonder if there's some way to disable this server timing header from the server. (I found the above information by simply opening a secure socket to port 443 (note that you need to connect to www.popularmechanics.com
otherwise it just sends you a redirect there) and writing the relevant HTTP request to it and printing the response. If you were desperate, you could do the same and basically parse the HTTP response yourself - it really depends on if this is a one off.) Otherwise, James is correct and you should raise an issue.