Search code examples
hdfsbigdatasqoop

Using Append and last modified in sqoop import statement

We are using sqoop to extract data from Oracle database to HDFS. People used to update and add new rows regularly to the SQL table. I am aware of sqoop incremental imports --append and --last modified options.

My question is whether we can use both options in same import statement?

For example,

sqoop --incremental import --lastmodified --append --check-column 'lastmodified' --check-column 'id' --lastvalue '2017-09-22-123456' --lastvalue 100

Or we need to use separately? Or any other better approach?

Solution

  • I found the answer myself.

    lastmodified can take care of both updates and newly added rows based on the last modified column. No need of using Append in this scenario.