the fact of real shit

Copy large amount of file to remote server using nohup tar and ssh

This command will copy large amount of file to remote server by compressing and decompressing on the fly. It saves time and bandwidth. It will execute in background, so you can detach current login session.

nohup sh -c “tar -c /any/directory/at/source/server/ | gzip -2 | ssh server-alias ‘cat | tar xz -C /target/directory/of/target/server/'” > /dev/null 2>&1 &

here nohup output sent to /dev/null that means i don’t want any nohup output. you can adjust its behavior.

I use the command for millions of file that occupied more than 500 GB.

Posted in linuxTagged , , ,

Install php 7.2 ssh2 in ubuntu 16x

recently i need to install ssh2 connection through my php code to connect remote server by sftp for file transfer. i’m using php 7.2 and face a “sigment fault” issue for normal installation. i solved this issue by install required demon. hope it may help someone who face same issue.

first you need to install/upgrade some basic program

apt-get install gcc make autoconf libc-dev pkg-config

then install base library

apt-get install libssh2-1-dev

now install required php modules

apt-get install php7.2-dev php-pear

now install ssh by pecl (the most important part of installation)

— pecl channel-update pecl.php.net
pear config-show
— pear config-set php_ini /etc/php/7.2/apache2/php.ini
— pear config-set temp_dir /etc/php/temp/pear
pecl install ssh2-1.1.2

ignore commented line OR use if you understand by yourself

now you need to enable ssh2 extension into your php cli installation

echo “extension=ssh2.so” > /etc/php/7.2/mods-available/ssh2.ini
ln -s /etc/php/7.2/mods-available/ssh2.ini /etc/php/7.2/cli/conf.d/30-ssh2.ini

please check you php installation/configuration path. set priority on your own. i set here 30 without proper understanding 😀

now another important part is your PHP code. when we use ssh2 in fopen wraper, in other version of ssh2 connection we need to open a connection and we can use resource id. but with the above change you must use user, password, port i.e. full access information every time we need to connect to server. here is my sample code –

$fh = @fopen(‘ssh2.sftp://’ . $this->user.’:’.$this->pass.’@’.$this->ip.’:’.(intval($this->port)>0?intval($this->port):22) . $pRemoteLocation, $pMode);

all other code like directory creation or any other command execution may be same as before.

Posted in php, ubuntuTagged , , , , , , ,