www.kingmaxyang.cn

不能不说的秘密

Archive for August, 2011

服务器间大文件传输

转自:http://www.sar4.com/2010/11/21/Linux%E4%B8%8B%E5%A4%A7%E6%96%87%E4%BB%B6%E5%A4%8D%E5%88%B6.html

1,gzip+scp+gunzip
server1压缩并传送:

server1$ gzip -c /backup/mydb/mytable.MYD > mytable.MYD.gz
server1$ scp mytable.MYD.gz root@server2:/var/lib/myql/mydb/

server2解压:

server2$ gunzip /var/lib/mysql/mydb/mytable.MYD.gz

2,通道+ssh
server1压缩通过通道ssh传送到server2并在server2上解压

server1$ gzip -c /backup/mydb/mytable.MYD | ssh root@server2 "gunzip -c - > /var/lib/mysql/mydb/mytable.MYD"

3,通道+netcat
server2监听12345端口:

server2$ nc -l -p 12345 | tar xvzf -

server1传送数据:

server1$ tar cvzf - /var/lib/mysql/mydb/mytable.MYD | nc -q 1 server2 12345

4,rsync

server1$ rsync -ave ssh server2:/var/lib/mysql/mydb/ /home/mydb/

速度对比:
100 Mb/S Ethernet LAN.文件大小为738 MB可压缩至100 MB

Method 				Time (seconds)
rsync without compression 	71
scp without compression 	68
nc without compression 		67
rsync with compression (-z) 	63
gzip, scp, and gunzip 		60(44 + 10 + 6)
ssh with compression 		44
nc with compression 		42

 

posted by kingmaxyang in Tech and have No Comments