what would be my options to make a script from command where I need to put my sudo password in?
Im exporting a fsimage and would like to do it on regular basis. It could be run from my accout but ideally, I would like to create a user dedicated to make these exports.
I would like to stay away from using root cron and use a more secure way of doing this
Entire command looks like this:
sudo ssh czmorchr 'hdfs oiv -p Delimited -i $(ls -t /dfs/nn/current/fsimage_* | grep -v md5 |
head -1) -o /dev/stdout 2>/dev/null' | grep -v "/.Trash/" |sed -e 's/\r/\\r/g' | awk 'BEGIN
{ FS="\t"; OFS="\t" } $0 ! ~ /_impala_insert_staging/ && ($0 ~ /^\/user\/hive\/warehouse\/cz_prd/ ||
$0 ~ /^\/user\/hive\/warehouse\/cz_tst/) { split($1,a,"/"); db=a[5]; table=a[6]; gsub(".db$", "", table); }
db && $10 ~ /^d/ {par=""; for(i=7;i<=length(a);i++) pa r=par"/"a[i] } db && $10 !~ /^d/
{ par=""; for(i=7;i<=length(a) - 1;i++) par=par"/"a[i]; file=a[length(a)] } NR > 1 { print db,table, par, file, $0 }' |
hadoop fs -put -f - /user/hive/warehouse/cz_prd_mon_ma.db/hive_warehouse_files/fsim age.tsv
I was able to resolve this issue using setfacl command. (Context:) I setup another folder in HDFS where standby node should dump its fsimages. Then I used this command and after that, I was able to run the script above without sudo and in a crontab.
setfacl -m u:hdfs:rwx /home/user_name/fsimage-dump/namenode