I want to create a Quartz job which reads .csv files and moves them when file is processed. I tried this:
@Override
public void execute(JobExecutionContext context) {
File directoryPath = new File("C:\\csv\\nov");
// Create a new subfolder called "processed" into source directory
try {
Files.createDirectory(Path.of(directoryPath.getAbsolutePath() + "/processed"));
} catch (IOException e) {
throw new RuntimeException(e);
}
FilenameFilter textFileFilter = (dir, name) -> {
String lowercaseName = name.toLowerCase();
if (lowercaseName.endsWith(".csv")) {
return true;
} else {
return false;
}
};
// List of all the csv files
File filesList[] = directoryPath.listFiles(textFileFilter);
System.out.println("List of the text files in the specified directory:");
Optional<File> csvFile = Arrays.stream(filesList).findFirst();
File file = csvFile.get();
for(File file : filesList) {
try {
List<CsvLine> beans = new CsvToBeanBuilder(new FileReader(file.getAbsolutePath(), StandardCharsets.UTF_16))
.....
.build()
.parse();
for(CsvLine item: beans){
....... sql queries
Optional<ProcessedWords> isFound = processedWordsService.findByKeyword(item.getKeyword());
......................................
}
} catch (Exception e){
e.printStackTrace();
}
// Move here file into new subdirectory when file processing is finished
Path copied = Paths.get(file.getAbsolutePath() + "/processed");
Path originalPath = file.toPath();
try {
Files.move(originalPath, copied, StandardCopyOption.REPLACE_EXISTING);
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}
Folder processed is created when the job is started but I get exception:
2022-11-17 23:12:51.470 ERROR 16512 --- [cessor_Worker-4] org.quartz.core.JobRunShell : Job DEFAULT.keywordPostJobDetail threw an unhandled Exception:
java.lang.RuntimeException: java.nio.file.FileSystemException: C:\csv\nov\11_42_33.csv -> C:\csv\nov\processed\11_42_33.csv: The process cannot access the file because it is being used by another process
at com.wordscore.engine.processor.ImportCsvFilePostJob.execute(ImportCsvFilePostJob.java:127) ~[main/:na]
at org.quartz.core.JobRunShell.run(JobRunShell.java:202) ~[quartz-2.3.2.jar:na]
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573) ~[quartz-2.3.2.jar:na]
Caused by: java.nio.file.FileSystemException: C:\csv\nov\11_42_33.csv -> C:\csv\nov\processed\11_42_33.csv: The process cannot access the file because it is being used by another process
at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:92) ~[na:na]
at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103) ~[na:na]
at java.base/sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:403) ~[na:na]
at java.base/sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:293) ~[na:na]
at java.base/java.nio.file.Files.move(Files.java:1432) ~[na:na]
at com.wordscore.engine.processor.ImportCsvFilePostJob.execute(ImportCsvFilePostJob.java:125) ~[main/:na]
... 2 common frames omitted
Do you know how I can release the file and move it into a sub directory?
EDIT: Update code with try-catch
@Override
public void execute(JobExecutionContext context) {
File directoryPath = new File("C:\\csv\\nov");
// Create a new subfolder called "processed" into source directory
try {
Path path = Path.of(directoryPath.getAbsolutePath() + "/processed");
if (!Files.exists(path) || !Files.isDirectory(path)) {
Files.createDirectory(path);
}
} catch (IOException e) {
throw new RuntimeException(e);
}
FilenameFilter textFileFilter = (dir, name) -> {
String lowercaseName = name.toLowerCase();
if (lowercaseName.endsWith(".csv")) {
return true;
} else {
return false;
}
};
// List of all the csv files
File filesList[] = directoryPath.listFiles(textFileFilter);
System.out.println("List of the text files in the specified directory:");
Optional<File> csvFile = Arrays.stream(filesList).findFirst();
File file = csvFile.get();
for(File file : filesList) {
try {
try (var br = new FileReader(file.getAbsolutePath(), StandardCharsets.UTF_16)){
List<CsvLine> beans = new CsvToBeanBuilder(br)
......
.build()
.parse();
for (CsvLine item : beans) {
.....
if (isFound.isPresent()) {
.........
}}
} catch (Exception e){
e.printStackTrace();
}
// Move here file into new subdirectory when file processing is finished
Path copied = Paths.get(file.getAbsolutePath() + "/processed");
Path originalPath = file.toPath();
try {
Files.move(originalPath, copied, StandardCopyOption.REPLACE_EXISTING);
} catch (IOException e) {
throw new RuntimeException(e);
}
}
Quartz config:
@Configuration
public class SchedulerConfig {
private static final Logger LOG = LoggerFactory.getLogger(SchedulerConfig.class);
private ApplicationContext applicationContext;
@Autowired
public SchedulerConfig(ApplicationContext applicationContext) {
this.applicationContext = applicationContext;
}
@Bean
public JobFactory jobFactory() {
AutowiringSpringBeanJobFactory jobFactory = new AutowiringSpringBeanJobFactory();
jobFactory.setApplicationContext(applicationContext);
return jobFactory;
}
@Bean
public SchedulerFactoryBean schedulerFactoryBean(Trigger simpleJobTrigger) throws IOException {
SchedulerFactoryBean schedulerFactory = new SchedulerFactoryBean();
schedulerFactory.setQuartzProperties(quartzProperties());
schedulerFactory.setWaitForJobsToCompleteOnShutdown(true);
schedulerFactory.setAutoStartup(true);
schedulerFactory.setTriggers(simpleJobTrigger);
schedulerFactory.setJobFactory(jobFactory());
return schedulerFactory;
}
@Bean
public SimpleTriggerFactoryBean simpleJobTrigger(@Qualifier("keywordPostJobDetail") JobDetail jobDetail,
@Value("${simplejob.frequency}") long frequency) {
LOG.info("simpleJobTrigger");
SimpleTriggerFactoryBean factoryBean = new SimpleTriggerFactoryBean();
factoryBean.setJobDetail(jobDetail);
factoryBean.setStartDelay(1000);
factoryBean.setRepeatInterval(frequency);
factoryBean.setRepeatCount(4); // factoryBean.setRepeatCount(SimpleTrigger.REPEAT_INDEFINITELY);
return factoryBean;
}
@Bean
public JobDetailFactoryBean keywordPostJobDetail() {
JobDetailFactoryBean factoryBean = new JobDetailFactoryBean();
factoryBean.setJobClass(ImportCsvFilePostJob.class);
factoryBean.setDurability(true);
return factoryBean;
}
public Properties quartzProperties() throws IOException {
PropertiesFactoryBean propertiesFactoryBean = new PropertiesFactoryBean();
propertiesFactoryBean.setLocation(new ClassPathResource("/quartz.properties"));
propertiesFactoryBean.afterPropertiesSet();
return propertiesFactoryBean.getObject();
}
}
Quartz config:
org.quartz.scheduler.instanceName=wordscore-processor
org.quartz.scheduler.instanceId=AUTO
org.quartz.threadPool.threadCount=5
org.quartz.jobStore.class=org.quartz.simpl.RAMJobStore
As you can see I wan to have 5 threads in order to execute 5 parallel jobs. Do you know how I can process the files without this exception?
Assuming we have File file = new File("c:/test.txt"), and print the the following paths:
Path copied = Paths.get(file.getAbsolutePath() + "/processed");
Path originalPath = file.toPath();
We will get the result:
copied: C:\test.txt\processed
originalPath: C:\test.txt
So its incorrect. You should try to get the parent path plus the processed folder plus the file name.
Path copied = Paths.get(file.getParentFile().getAbsolutePath() + "/processed/" + file.getName());
Path originalPath = file.toPath();