FileOutputStream throws FileNotFoundException (Too many open files)

Question

My application somehow throws „java.io.FileNotFoundException: /file/path (Too many open files)“ and I have no idea which part is causing this exception, because the try-with-resources is supposed to close a file after writing, but it somehow doesn’t. I have no other resources that are writing to files, so it can’t be caused by an external application.

I also tried replacing the streams as well as changing the method writeToFile(String filename, Double data) to non-static which is why class B is now creating a new instance for DataFileWriter in each loop. I was hoping that this somehow forces the FileOutputStream to close, but it didn’t help.

public class A {
    public static void main(String[] args) {
        List<Data> dataList = dao.getAll();
        B classB = new B();
        dataList.stream().forEach(data -> {
            classB.someMethod(data);
        });
    }
}

public class B {
    public void someMethod(Data data) {
        // Some transformation happens here which results in transformedData
        writeToFile(filename, transformedData);
    }

    private void writeToFile(String filename, Double[][] data) {
        Arrays.stream(data).forEach(d -> {
            DataFileWriter dataFileWriter = new DataFileWriter();
            dataFileWriter.writeToFile(filename, d);
        });
    }
}

public class DataFileWriter {
    public void writeToFile(String filename, Double data) {
        File file = new File(filename);
        if (!file.getParentFile().isDirectory()) file.getParentFile().mkdirs();
        try (FileOutputStream fOut = new FileOutputStream(filename, true) {
            byte b[] = data.getBytes();
            fOut.write(b);
        } catch (IOException e) {
            // Exception handling
        }
    }
}

It would be nice if you could help me resolving this issue. Modifying the maximum number of allowed opened files is no option, because the application is supposed to handle much more data than at its current state.

UPDATE

I changed a few things to reduce use of memory, but still I get the same error after the program has written to exactly 5016 files. I went through the whole code, but couldn't figure anything out. The only place where I can imagine the exception is triggered somehow, is during calling the API where a connection is established, but I don't see any errors here either.

protected static CoreEntity[] sendGET(CoreEntity[] resultList, String path) throws IOException, JSONException {
    return handleResponse(resultList, getConnection(path, GET_REQUEST));
}

private static HttpURLConnection getConnection(String path, String requestMethod) throws IOException {
    URL url = new URL(REQUEST_URL + path);
    HttpURLConnection connection = (HttpURLConnection) url.openConnection();
    connection.setDoOutput(true);
    connection.setDoInput(true);
    connection.setRequestProperty("Content-Type", "application/json");
    connection.setRequestProperty("accept", "application/json");
    connection.setConnectTimeout(50000);
    connection.setReadTimeout(50000);
    connection.setRequestMethod(requestMethod);
    initializeGSON();
    return connection;
}

private static CoreEntity[] handleResponse(CoreEntity[] resultList, HttpURLConnection connection) throws IOException {
    final int status = connection.getResponseCode();
    if (status == HttpURLConnection.HTTP_OK) { // Success
        try (InputStreamReader reader = new InputStreamReader(connection.getInputStream()); BufferedReader in = new BufferedReader(reader)) {
            String inputLine;
            StringBuilder response = new StringBuilder();
            while ((inputLine = in.readLine()) != null) { response.append(inputLine); }
            reader.close();
            in.close();
            JSONArray jsonArray = getJSONAsArray(response.toString());
            resultList = (CoreEntity[]) Array.newInstance(resultList.getClass().getComponentType(), jsonArray.length());
            for (int i = 0; i < jsonArray.length(); i++)
                resultList[i] = (CoreEntity) GSON.fromJson(jsonArray.get(i).toString(), resultList.getClass().getComponentType());
        } catch (IOException | JSONException e) { e.printStackTrace(); }
    } else { // Error
        System.out.println("Request failed with error code: " + status);
    }
    connection.disconnect();
    return resultList;
}

Answer 1

I am writing multiple files to a single directory where the name of a file corresponds to the amount of files within the directory. To get this length I used Files.list(Paths.get(directory)).count(). Apparently this opens all files within the directory which caused the mentioned exception. I replaced it with new File(getRootDirectory() + directory + "/").list().length and now everything works as expected. A great help was apple's activity monitor where I noticed that the opened files came from a specific method.

Although you couldn’t help me directly, thank you guys for pointing me in the right direction.