CrawlSpider with Splash getting stuck after first URL

Question

I'm writing a scrapy spider where I need to render some of the responses with splash. My spider is based on CrawlSpider. I need to render my start_url responses to feed my crawl spider. Unfortunately my crawl spider stops after rendering of the first responds. Any idea what is going wrong?

class VideoSpider(CrawlSpider):

    start_urls = ['https://juke.com/de/de/search?q=1+Mord+f%C3%BCr+2']

rules = (
    Rule(LinkExtractor(allow=()), callback='parse_items',process_request = "use_splash",),
)

def use_splash(self, request):
    request.meta['splash'] = {
            'endpoint':'render.html',
            'args':{
                'wait':0.5,
                }
            }     
    return request

def start_requests(self):
    for url in self.start_urls:
        yield scrapy.Request(url, self.parse, meta={
            'splash': {
                'endpoint': 'render.html',
                'args': {'wait': 0.5}
        }
    })  


def parse_items(self, response):      
    data = response.body
    print(data)

Answer 1

Use SplashRequest instead of scrapy.Request... Check out my answer CrawlSpider with Splash