1. Overview
In this document, we will explore a shell script named sftp_file_transfer.sh that automates the process of transferring files from one server to another.
2. Technologies and Tools Used
The following technologies has been used to achieve the expected output.
- Oracle PLSQL
- UNIX/Linux.
3. Use Case
Use the script to automate the replication of data between servers, ensuring data consistency and availability.
Schedule the script to run at specific intervals to transfer backup files securely between servers.
Leverage the script as part of a larger data integration workflow. Execute database procedures before or after the file transfer process.
Transfer log files securely from one server to another for centralized monitoring and analysis.
Implement the script for securely exchanging sensitive data between servers, crucial in environments with strict security requirements.
4. Architecture
Let’s look into a shell script named sftp_file_transfer.sh that automates the process of transferring file from one server to another server.
Shell script:
#############################################################################
## Script Name : sftp_file_transfer.sh
## Comments : Transferring file from one server to another using sftp process
#############################################################################
echo “==========================================================================”
echo “$0 Started at : ” $(date)
local_dir=’/home/’
scr_running=$(cat $local_dir/sftp_file_transfer.chk | wc -l)
chk_flag=$(expr $scr_running)
if [ $chk_flag -ne 0 ]; then
echo “Another instance of the script is already running”
exit;
fi
echo “RUNNING” > $local_dir/sftp_file_transfer.chk
sqlplus -s username/pwd@DB << EOF
exec Procedure_name;
exit;
EOF
cd $local_dir
sftp user1@servername1 <<EOF
cd /path/
mget filename*.dat
rm filename*.dat
bye
EOF
cnt=$(ls -lrt *.dat | wc -l)
if [ $cnt -ne 0 ]; then
sftp user2@server2 1>$local_dir/sftp_file_transfer.log 2>&1 << EOF
lcd $local_dir/
cd /pathtoplace
mput filename*.dat
ls -l
quit
EOF
fi
>$local_dir/sftp_file_transfer.chk
echo “Script Completed”
Understanding the Script
SFTP Configuration:
Verify that SFTP is properly configured on both servername1 and server2. Ensure that the user accounts used for SFTP have the necessary permissions and can perform the required file operations.
SSH Key Authentication:
Consider setting up SSH key authentication to avoid password prompts during the SFTP transfers. This requires generating and exchanging SSH keys between the machines involved.
Concurrency Control:
The script uses a check file (sftp_file_transfer.chk) to prevent multiple instances from running concurrently. This ensures data integrity during file transfers.
Database Interaction:
The script interacts with an Oracle Database using sqlplus. It executes a stored procedure (Procedure_name) as part of the workflow.
SFTP Transfer – Server 1:
Utilizes the sftp command to connect to servername1. The script changes to a specified directory, retrieves files matching a pattern (filename*.dat), removes them from the remote server, and exits.
File Check:
Checks if any files were retrieved from Server 1 before proceeding with the next transfer.
SFTP Transfer – Server 2:
If files are present, connects to server2, transfers the files from the local directory, and lists the files on the remote server.
Cleanup:
Clears the check file to mark the completion of the script.
Logging:
Redirects the output of the second sftp command to a log file (sftp_file_transfer.log) for monitoring and debugging.
5. Conclusion
The sftp_file_transfer.sh script provides a versatile and secure solution for automating file transfers between servers. Its integration with Oracle Database procedures enhances its capabilities, making it a valuable tool for various data management scenarios. Customize the script according to your specific needs and leverage its potential to streamline your data transfer workflows.
