PDI - Collecting File From FTP Older Than N Day
I have a job that will Collect Data from FTP using Get a file with FTP and I want it's only collect yesterday file or older than n day or base on specific date.
How do that? Is any way or possible?
What I know is Get a file with FTP only copy file directly from FTP to destination folder. So, I can't use any field and assign it into JavaScript variable to create condition.
My requirement is moving only yesterday or ... file from FTP into Location I need, not all of them because I have a lot of file about 30K-40K with various file size and it will took a lot of time if I do that. Below is the pic what I have design.
By using the 'Get File Names' step in a transformation, you can access your FTP files (via VFS) and their atributes, namely the 'lastmodifiedtime'.
With this information you can do a simple filter by dates, and only download the files which are older than N days, or any other filter you require. With that in hand you can move, download or any other file related action you desire.
- Access to previous row in Kettle
- Use a field from a previous hop as a parameter in a Table input using Pentaho Kettle
- Pentaho kettle Can't run transformation due to plugin missing
- Waiting for Transformations in a Job
- Pentaho Kettle replace cell value if value exists in other two columns
- How to find where KETTLE HOME is defined
- Get row number from stream to perform concat
- How to delete rows which were deleted in source table (Pentaho data integration)
- Pentaho metastore location when connecting to repository
- How to pass variables between steps inside a transformation in Pentaho
- How to loop through elements in XML to shred and load to database
- Can't change Application Type RESTClient Pentaho Data Integration
- Send email when transformation didn't work with Pentaho
- PDI 8.0 S3 CSV Input
- Pentaho Data Integrator - MongoDB Input - JSONParseException
- How to get json keys in Pentaho?
- Not able to run spoon.bat or any other batch file in Pentaho Data Integration (Kettle)
- Date format conversion from "yyyy-mm-dd" to "dd/mm/yyyy" in pentaho using javascript
- Pentaho Data Integration - Merge rows (diff) step not executing with Table Input "Replace variables in script"
- Unzip file issue in Pentaho-spoon
- Pentaho 7.1 Community Edition Cluster Unified Repository
- Mismatch on the day of week and week name in Pentaho
- How to connect to Oracle database with multiple-address-line TNS in Pentaho Data Integration/Kettle ETL
- Replace Complete String - Pentaho Data Integration
- Delete last two character of a string - Pentaho Data Integration
- Pentaho/Kettle - Split Fields into two columns
- Unable to find ID Kettle
- Pentaho DI default logging database for kitchen.sh
- Pentaho (kettle) download files from ChEBI FTP server
- Carte Server unable to connect to repository in Slave Server