Reading file from and saving file to FTP at the same time

Question

I have problem with downloading file from FTP, processing it in memory, and then saving a couple of files to the same FTP server.

What I'm trying to achieve: I want to download csv file (lets say 1000 rows), split it to smaller files (100 rows each) and save them to diffrent directory.

I'm succesfully connecting to FTP, succesfully downloading filee and processing it but I cannot save those smaller files.

Code:

public boolean getFile1(final String filePath, final String fileName) {
    String remoteFile = filePath + "/" + fileName;
    BufferedReader reader = null;
    try{
        InputStream is = ftpClient.retrieveFileStream(remoteFile);
        reader = new BufferedReader(new InputStreamReader(is));
        String line;
        String file ="";
        int i = 1;
        int max = 100;
        int current = 0;
        while((line = reader.readLine()) != null){
            if(current < max){
                file+=line+"\n";
                current++;
            }
            else{
                System.out.println("Else; File: " +file);
                InputStream stream = new ByteArrayInputStream(file.getBytes(StandardCharsets.UTF_8.name()));

                putFile(stream, "test", "MOCK_COPY"+i+".csv");

                i++;
                file = "";
                current = 0;
                file+=line+"\n";
            }
        }
        if(!file.isEmpty()){
            System.out.println("End File: " +file);
            InputStream is2 = IOUtils.toInputStream(file, "UTF-8");

            putFile(is2, "test", "MOCK_COPY"+i+".csv");

        }
    } catch (IOException e){
        e.printStackTrace();
    } finally {
        try{
            reader.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }


    return true;
}

public Boolean putFile(InputStream inputStream, final String filePath, final String fileName) throws IOException {
    Long ts = System.currentTimeMillis();
    String remoteFile = filePath + "/" + fileName;

    System.out.println("Start uploading file");

//        ftpClient.changeWorkingDirectory("test");
////this overwrites the existing file
//        ftpClient.storeFile(remoteFile, inputStream);
//inputStream.close();

    OutputStream outputStream = ftpClient.storeFileStream(remoteFile);
    byte[] bytesIn = new byte[4096];
    int read = 0;
    while ((read = inputStream.read(bytesIn)) != -1) {
        outputStream.write(bytesIn, 0, read);
    }
    inputStream.close();
    outputStream.close();

    System.out.println("The file is uploaded successfully in " + (System.currentTimeMillis() - ts) / 1000 + "s");
    return true;
}

Connection:

public void connectToServer(String server, String user, String pass, Integer port) 
        throws WebApplicationException {
    port = port == null ? 21 : null;
    server = server.replace("ftp://","");



    try {
        ftpClient = new FTPClient();
        ftpClient.connect(server, port);
        ftpClient.login(user, pass);
        ftpClient.enterLocalPassiveMode();
        ftpClient.setFileType(FTP.BINARY_FILE_TYPE);
        System.out.println("Connected to ftp");
    }
    catch (IOException e) {
        e.printStackTrace();
        throw new WebApplicationException(Response.status(Response.Status.NOT_FOUND)
                .entity("FTP error: " + e.getMessage()).type(MediaType.TEXT_PLAIN).build());
    }
}

I'm getting null pointer exception from OutputStream (but ftpCLient and remotefile is not null)

I observed that if I only try to upload file using that method everything is going well.

Anyone have any idea whats happening?

Answer 1

With FTP protocol, you cannot download and upload at the same time over the same connection.

• Either separate the download and upload process.

• Or open two connections, one for download and other for upload.

---

I'm not claiming that this is the root cause of the exception you are getting. But it's quite possible. And even if not, your code cannot work anyway, so there's no point trying to fix it in its current state.