Unix - Shell Script - Connect to SFTP from script

Description:

This post cover various ways to connect to SFTP from shell script. Passing the password in the command prompt is easier and secure as user is explicitly typing it down when prompted. But in shell script, password needs to be passed automatically and securely.

Approach-01: using sshpass command

Resource-01 covers detail about the command and how to install in various version on unix.  

SFTP_USER=<user>
SSHPASS=<password>

TARGET_SERVER=<host>
sshpass -e sftp -oBatchMode=no -b - $SFTP_USER@$TARGET_SERVER <<EOF
    ls -lrt
    cd <target path>
    put <source path/file name>
    bye
EOF

Approach-02: using lftp command

SFTP_USER=<user>

LFTP_PASSWORD=<PASSWORD>

TARGET_SERVER=<host>

lftp --env-password sftp://$SFTP_USER@$TARGET_SERVER <<EOF
    ls -lrt
    cd <target path>
    put <source path/file name>
    bye
EOF

Resources:

1. Detail about 'sshpass' command. How to install the command, various ways to pass password to connect to SFTP etc. 
   
   https://www.cyberciti.biz/faq/noninteractive-shell-script-ssh-password-provider/

Unix - Shell Script - Find files and execute a command on matches

Description:

Often time we need to perform additional tasks on file/folder found in a certain path. Instead of fetching the list and looping through the items, it's easier to combine the search and additional task in to one command, more like chaining commands. "find" unix command allows executing additional command to be executed on matched items. This article provides few examples to demonstrate how to do so with explanation.

Example-01: Delete all the log files older than 14 days

find . -maxdepth 1 -type f -mtime +14 -exec rm -fr {} \;

Explanation:

• Scan current folder path for only files (find .)
• Skip scanning sub folder(s) (-maxdepth 1)
• Consider only files, exclude directory (-type f)
• Match files which are older than 14 days (-mtime +14)
• Perform delete on matched items (-exec). -exec allows to execute a command on items
• Forcefully delete any match found without any confirmation message (rm -fr)
• Use {} (curly braces) as a placeholder for each file that the find command locates
• Need to add command terminator (\;). This is a must if -exec attribute is used in find command

Example-02: Copy all the matching files 

find . -maxdepth 1 -type f -exec cp {} ~/ \;

Explanation:

• Copy all matched items in home directory (-exec cp {} ~/). 
• Other parts are explained in example-01

Example-03: Change permission of matched files

find . -maxdepth 1 -type f -exec chmod 744 {} \;

Explanation:

• Change the file permission to 744 (-exec chmod 744). 
• Other parts are explained in example-01

Example-04: Execute multiple commands. Change permission and move all files to a folder 

find . -maxdepth 1 -type f -exec chmod 744 {} \; -exec mv {} ~/test/ \;

Explanation:

• Chains multiple commands using (-exec) multiple times. Each command must be ended with (\;)
• First command changes the permission and second command moves the files to test folder under home path.
• Other parts are explained in example-01

References:

1. Detail about 'find' command - https://kb.iu.edu/d/admm