Failed to start QThread when using QNetworkAccessManager

Question

I'm currently trying to make a software that downloads a lot of files from Google Drive. Downloading is currently not a problem.

Nevertheless, I encounter an issue when launching 500+ simultaneous downloads. I use a slightly modified version of this tutorial : https://wiki.qt.io/Download_Data_from_URL.

Here is the .h file :

class FileDownloader : public QObject
{
    Q_OBJECT
public:
    explicit FileDownloader(QUrl url, QObject *parent = 0, int number = 0);
    QByteArray downloadedData() const;
    void launchNewDownload(QUrl url);
    QByteArray m_DownloadedData;
    QNetworkReply* reply;

    static QNetworkAccessManager *m_WebCtrl;

signals:
    void downloaded();

private slots:
    void fileDownloaded(QNetworkReply* pReply);
};

And here is the .cpp file :

QNetworkAccessManager* FileDownloader::m_WebCtrl = nullptr;

FileDownloader::FileDownloader(QUrl url, QObject *parent) :
    QObject(parent)
{
    if (m_WebCtrl == nullptr) {
        m_WebCtrl = new QNetworkAccessManager(this);
    }
    connect(m_WebCtrl, SIGNAL (finished(QNetworkReply*)),this, SLOT (fileDownloaded(QNetworkReply*)));

    launchNewDownload(url);
}

void FileDownloader::launchNewDownload(QUrl url) {
    QNetworkRequest request(url);
    this->reply = m_WebCtrl->get(request);
}

void FileDownloader::fileDownloaded(QNetworkReply* pReply) {
    m_DownloadedData = pReply->readAll();

    //emit a signal
    pReply->deleteLater();
    emit downloaded();
}

QByteArray FileDownloader::downloadedData() const {
    return m_DownloadedData;
}

The issue is "QThread::start: Failed to create thread ()" when reaching about the 500th download. I tried to limit the number of downloads which run at the same time - but I always get the same issue. Besides, I tried to delete every downloader when finishing its task - it did nothing else than crashing the program ;)

I think that it is coming from the number of threads allowed for an only process, but I'm not able to solve it !

Does anyone have an idea that could help me ?

Thank you !

Answer 1

QNetworkAccessManager::finished signal is documented to be emitted whenever a pending network reply is finished.

This means that if the QNetworkAccessManager is used to run multiple requests at a time (and this is perfectly normal usage). finished signal will be emitted once for every request. Since you have a shared instance of QNetworkAccessManager between your FileDownloader objects, the finished signal gets emitted for every get call you have made. So, all the FileDownloader objects get a finished signal as soon as the first FileDownloader finishes downloading.

Instead of using QNetworkAccessManager::finished, you can use QNetworkReply::finished to avoid mixing up signals. Here is an example implementation:

#include <QtNetwork>
#include <QtWidgets>

class FileDownloader : public QObject
{
    Q_OBJECT
    //using constructor injection instead of a static QNetworkAccessManager pointer
    //This allows to share the same QNetworkAccessManager
    //object with other classes utilizing network access
    QNetworkAccessManager* m_nam;
    QNetworkReply* m_reply;
    QByteArray m_downloadedData;

public:
    explicit FileDownloader(QUrl imageUrl, QNetworkAccessManager* nam,
                            QObject* parent= nullptr)
        :QObject(parent), m_nam(nam)
    {
        QNetworkRequest request(imageUrl);
        m_reply = m_nam->get(request);
        connect(m_reply, &QNetworkReply::finished, this, &FileDownloader::fileDownloaded);
    }
    ~FileDownloader() = default;

    QByteArray downloadedData()const{return m_downloadedData;}

signals:
    void downloaded();
private slots:
    void fileDownloaded(){
        m_downloadedData= m_reply->readAll();
        m_reply->deleteLater();
        emit downloaded();
    }
};

//sample usage
int main(int argc, char* argv[]){
    QApplication a(argc, argv);

    QNetworkAccessManager nam;
    FileDownloader fileDownloader(QUrl("http://i.imgur.com/Rt8fqpt.png"), &nam);
    QLabel label;
    label.setAlignment(Qt::AlignCenter);
    label.setText("Downloading. . .");
    label.setMinimumSize(640, 480);
    label.show();
    QObject::connect(&fileDownloader, &FileDownloader::downloaded, [&]{
        QPixmap pixmap;
        pixmap.loadFromData(fileDownloader.downloadedData());
        label.setPixmap(pixmap);
    });

    return a.exec();
}

#include "main.moc"

---

^{If you are using this method to download large files, consider having a look at this question.}