Multi-threaded read from file and execute runnable

Question

WHAT?

---

I am trying to build a tool that will reads a text file and publishes the text, after doing some string transformation.

HOW?

---

The tool reads the file line by line and populates a LinkedBlockingQueue. At the same time I initiate multiple threads that will then take a message each from the LBQ, do some processing and publish them.

Main

private static LinkedBlockingQueue<String> lbQueue = new LinkedBlockingQueue<>();
private static Boolean keepPublisherActive = Boolean.TRUE;
public static void main(String[] args) {
    try {
        tool.initMessagePublish();
        tool.searchUsingScanner();
    } catch (Exception ex) {
        logger.error("Exception in Tool Main() " + ex.toString());
        throw ex;
    }
}

File Reader

private void searchUsingScanner() {
        Scanner scanner = null;
        try {
            scanner = new Scanner(new File(LOG_FILE_PATH));
            while (scanner.hasNextLine()) {
                String line = scanner.nextLine().trim();
                if (StringUtils.isNotBlank(line)) {
                    lbQueue.offer(line);
                }
            }
        } catch (Exception e) {
            logger.error("Error while processing file: " + e.toString());
        } finally {
            try {
                if (scanner != null) {
                    scanner.close();
                }
                // end thread execution
                keepPublisherActive = false;
            } catch (Exception e) {
                logger.error("Exception while closing file scanner " + e.toString());
                throw e;
            }
        }
    }

Multi-threaded Publisher

    private void initMessagePublish() throws InterruptedException {
        ExecutorService service = Executors.newFixedThreadPool(6);
    try {
        while (keepPublisherActive || lbQueue.getSize() > 0) {
            service.execute(messagePublisher); // messagePublisher implements Runnable
        }
    } catch (Exception ex) {
        logger.error("Multi threaded message publish failed " + ex.toString());
        throw ex;
    } finally {
        service.shutdown();
    }
}

THE PROBLEM

---

The intention behind calling initMessagePublish() fist is that the publisher need not wait for all lines to be read from the file before starting to publish. It should start publishing as soon as something becomes available in the LBQ.

But with the current implementation, the control never comes out of the initMessagePublish and start searchUsingScanner. How do I solve this? Basically, the two methods should execute parallely.

Answer 1

Just start messagePublisher in a new Thread (Line no #5 in Main class):

new Thread(()->tool.initMessagePublish()).start();

It should solve your problem.