I have a Pyramid web app with fail2ban set up to jail ten consecutive 404 statuses (i.e. bots that probe for vulnerabilities), Sentry error logging and, as far as I know, there are no security vulnerabilities. However, every few days I get a notification of a 502 caused by a null byte attack. This is harmless, but it has become very tiresome and I ignored a bizarre but legitimate human-user–generated 502 status as a result.
A null byte attack in Pyramid, in my set-up, raises a URLDecodeError ('utf-8' codec can't decode byte 0xc0 in position 16: invalid start byte) at the url dispatch level, so is not routed to the notfound_view_config decorated view.
Is there any way to capture %EF/%BF in requests in Pyramid or should I block them in Apache?
Comment by Steve Piercy converted into an Answer: A search in the Pyramid issue tracker yields several related results. The first hit provides one way to deal with it.
In brief, the view constructor class exception_view_config(ExceptionClass, renderer) captures it behaving like notfound_view_config or forbidden_view_config (which aren't passed declared routes in contrast to view_config).
So the 404 view could look like:
from pyramid.view import notfound_view_config
from pyramid.exceptions import URLDecodeError
from pyramid.view import exception_view_config
@exception_view_config(context=URLDecodeError, renderer='json')
@notfound_view_config(renderer='json')
def notfound_view(request):
request.response.status = 404
return {"status": "error"}
This can be tested by visiting the browser http://0.0.0.0:👾👾/%EF%BF (where 👾👾 is the port served onto).
However, there are two additionally considerations.
pyramid.includes = pyramid_debugtoolbar in the local configuration ini file).request.path_info gets accessed. So either the response is minimally formatted or request.environ['PATH_INFO'] is assigned a new value before any operation in the view (e.g. usage data etc.).
The view call happens after the debugtoolbar error is raises, however, so the first point still stands even with a request.environ['PATH_INFO'] = 'hacked'.As this is unequivocally an attack, this could be customised to play well with fail2ban to block the hacker IP as described here by using a unique status code, say 418, at the first occurrence.