I need to load data from sybase(production database) to HDFS. By using sqoop it is taking very long time and frequently hit the production database. So, I am thinking to create data files from sybase dump and after that copy the data files to hdfs. Is there any tool(open source) is available to create required data files(flat files) from sybase dump.
Thanks,
The iq_bcp command line utility is designed to do this on a per table basis. You just need to generate a list of tables, and you can iterate through the list.
iq_bcp [ [ database_name. ] owner. ] table_name { in | out } datafile
iq_bcp MyDB..MyTable out MyTable.csv -c -t#$#
-c specifies a character (plaintext) output
-t allows you to customize the column delimiter. You will want to use a character or series of characters that do not appear in your extact e.g. if you have a text column that contains text with a comma, a csv will be tricky to import without additional work.