Django-Storages with SFTP: GET-requests fail

Question

I am trying to use django-storages to access my "Hetzner" Storage Box (https://www.hetzner.com/storage/storage-box) using SFTP which should hold media data, i.e. image files which users of my website can upload dynamically.

The corresponding part of my settings.py file looks like:

DEFAULT_FILE_STORAGE = 'storages.backends.sftpstorage.SFTPStorage'
SFTP_STORAGE_HOST = 'username.your-storagebox.de'
SFTP_STORAGE_ROOT = '/media'

SFTP_STORAGE_PARAMS = {
'username': 'username',
'password': 'password',
'allow_agent': False,
'look_for_keys': False,
}

The strange thing is that, when the user uploads an Image, it is placed on the storage space as I can confirm using SFTP. But getting the images from the storage box fails, no Image is displayed. An excerpt from the console:

[03/Sep/2021 22:34:01] "GET /media/filename.jpg HTTP/1.1" 404 1962

I could figure out that Django is still looking inside my MEDIA_DIR for the files. Againg, the corresponding part of my settings:

MEDIA_DIR = 'media'
MEDIA_ROOT = os.path.join(BASE_DIR, MEDIA_DIR)
MEDIA_URL = '/media/'

So in a nutshell: Using SFTP seems to be working for putting files into storage, but getting them again somehow fails.

EDIT: As requested, I am going to provide some more code snippets: models.py:

class SizeRestrictedImageField(ImageField):

    def __init__(self, *args, **kwargs):
        self.max_upload_size = kwargs.pop('max_upload_size', 0)
        super().__init__(*args, **kwargs)

    def clean(self, *args, **kwargs):
        data = super().clean(*args, **kwargs)

        file = data.file
        try:
            if file.size > self.max_upload_size:
                raise forms.ValidationError(_('Please keep filesize under %s. Current filesize %s'
                        ) % (filesizeformat(self.max_upload_size),
                        filesizeformat(file.size)))
        except AttributeError:
            logger.exception('An Exception occured while checking for max size of image upload. size: `%s`'
                             , file.size)
            pass

        return data


class ImageModel(models.Model):
    image = SizeRestrictedImageField(upload_to=POST_PIC_FOLDER, null=True, blank=True,
                              help_text="Erlaubte Dateitypen: .jpeg, .jpg, .png, .gif", max_upload_size=MAX_IMAGE_SIZE)

And my urls.py:

urlpatterns = [
                  path('defaultsite/', defaultsite_view, name='home'),
                  path('help', help_view, name="help"),
                  path('user/', include('user.urls')),
                  path('sowi/', include('sowi.urls')),
                  path('blog/', include('blog.urls')),
                  path('chat/', include('chat.urls')),
                  path('notifications/', include('notifications.urls')),
                  path('cookies/', include('cookie_consent.urls')),
                  path('', home_view, name="home"),
                  path('about/', AboutUsView.as_view(), name="about-us"),
                  path('impressum/', impressum_view, name="imprint"),
                  path('privacy/', privacy_view, name='privacy'),
                  path('privacy/statement/', privacy_statement_view, name='privacy-statement'),
                  path('agb', agb_view, name="agb")
              ] + static(settings.STATIC_URL, document_root=settings.STATIC_ROOT) + static(settings.MEDIA_URL,
                                                                                           document_root=settings.MEDIA_ROOT)

I tried removing the +static(...)-part from my url-patterns, but that didn't seem to solve the issue.

Answer 1

I want to complete @Paulos Answer. You can create the proxy by using a middleware. Create a file project/middleware.py and add it to your middleware-array in the settings.py.

Then create the middleware:

import mimetypes

from storages.backends.sftpstorage import SFTPStorage
from django.http import HttpResponse



class SFTPMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response
        # One-time configuration and initialization.

    def __call__(self, request):
        # Code to be executed for each request before
        # the view (and later middleware) are called.

        SFS = SFTPStorage()
        
        response = self.get_response(request)

        path = request.get_full_path()

        if SFS.exists(path):
            file = SFS._read(path)
            type, encoding = mimetypes.guess_type(path)
            response = HttpResponse(file, content_type=type)
            response['Content-Disposition'] = u'attachment; filename="{filename}"'.format(filename=path)

        return response

EDIT: It is actually not necessary to open up a new connection on every call. You should rather create a single connection on system startup in the ready()-hook of your config.py and use this one.