Month: October 2014

How to encrypt files, directories and partitions in Linux (using eCryptFS and Cryptsetup)

  • ecryptfs/ECryptfs@wiki/ecryptfs@man is a filesystem-level encryption, kernel-native stacked cryptographic filesystem for Linux (not FUSE-based). Only certain files or directories are encrypted selectively. EncFS is a filesystem in user-space/FUSE.
# install
$ sudo apt-get install ecryptfs-utils | yum install ecryptfs-utils | sudo pacman -S ecryptfs-utils
$ sudo modprobe ecryptfs

# creates encrypted ~/.Private and decrypted ~/Private; PAM-module will at login automatically decrypted ~/.Private into ~/Private, and at logout unmounted ~/Private and encrypted ~/Private back into the ~/.Private
$ ecryptfs-setup-private

# mount/encrypt /srv w/ layover where lower directory is same as mount-point
$ mount -t ecryptfs /srv /srv

# unmount to hide data
$ unmount /srv

# automatically mounting
$ cat /mnt/usb/passwd_file.txt
passphrase_passwd=[secrets]
# ecryptfs_sig must be same as /root/.ecryptfs/sig-cache.txt
$ cat /root/.ecryptfsrc
key=passphrase:passphrase_passwd_file=/mnt/usb/passwd_file.txt
ecryptfs_sig=5826dd62cf81c615
ecryptfs_cipher=aes
ecryptfs_key_bytes=16
ecryptfs_passthrough=n
ecryptfs_enable_filename_crypto=n
$ cat /etc/fstab
/dev/sdb1       /mnt/usb        ext3     ro       0 0
/srv            /srv            ecryptfs defaults 0 0

From How to encrypt files and directories with eCryptFS on Linux and How To Encrypt Directories/Partitions With eCryptfs On Debian Squeeze

  • DMCrypt+LUKS is full-disk encryption solutions which means that the entire disk is encrypted. CipherShed is an TrueCrypt-fork opensource alternative.
# install
$ sudo apt-get install cryptsetup | sudo yum install cryptsetup-luks

# encrypt /dev/xvdc; asked for passphrase
$ cryptsetup -y -v luksFormat /dev/xvdc
# initialize volume
$ cryptsetup luksOpen /dev/xvdc backup2
$ ls -l /dev/mapper/backup2
$ cryptsetup -v status backup2

# zero-out, create filesystem and mount LUKS partition
$ pv -tpreb /dev/zero | dd of=/dev/mapper/backup2 bs=128M
$ mkfs.ext4 /dev/mapper/backup2
$ mkdir /backup2 ; mount /dev/mapper/backup2 /backup2

# unmount and secure data
$ umount /backup2
$ cryptsetup luksClose backup2

# remount encrypted partition
$ cryptsetup luksOpen /dev/xvdc backup2
$ mount /dev/mapper/backup2 /backup2

# change LUKS passphrase
$ cryptsetup luksDump /dev/xvdc
$ cryptsetup luksAddKey /dev/xvdc
$ cryptsetup luksRemoveKey /dev/xvdc

From Linux Hard Disk Encryption With DMCrypt+LUKS

How to benchmark Linux machines (using Phoronix Test Suite and OpenBenchmarking.org)

Phoronix Test Suite is an open-source benchmarking platform which provides an extensible list of test profiles and test suites. Results can be uploaded and compared openbenchmarking.org.

# install
$ sudo yum install phoronix-test-suite (EPEL) | sudo apt-get install phoronix-test-suite
# if behind proxy then set initial proxy configuration
$ phoronix-test-suite network-setup

# list available tests profiles and suites
$ phoronix-test-suite list-tests (http://openbenchmarking.org/tests/pts)
$ phoronix-test-suite list-suites (http://openbenchmarking.org/suites/pts)

# run a test suite; may ask to install additional packages; take a coffee
$ phoronix-test-suites benchmark sqlite (~40mins)
$ phoronix-test-suites benchmark aio-stress (~9mins)
# note: benchmark = install+run

# list locally saved results
$ phoronix-test-suite list-results
$ phoronix-test-suite show-result|remove-result [test result]
$ ls ~/.phoronix-test-suite/test-results

# run a openbenchmarking id test and compare
$ phoronix-test-suite benchmark 1410317-SO-TEST2784377

# cleanup
$ rm -rf ~/.phoronix-test-suite/

From How to benchmark virtual machines
See PTS documentation in pdf

How to mount a remote directory over ssh on Linux

sshfs is a filesystem client based on ssh, with a FUSE implementation.

# install
$ sudo apt-get install sshfs | sudo yum install sshfs (EPEL)
$ sudo usermod -a -G fuse <user_name>

# mount remote directory, asking for user and password
$ sshfs my_user@remote_host:/path/to/directory <local_mount_point>
# unmount
$ fusermount -u <local_mount_point>

# mount on boot
$ cat /etc/fstab
sshfs#my_user@remote_host:/path/to/directory <local_mount_point> fuse user 0 0

from How to mount a remote directory over ssh on Linux

How to count number of source code lines (using cloc)

cloc counts, and compute differences of, lines of source code and comments.

# install
$ sudo yum install cloc (Fedora Rawhide) | sudo apt-get install cloc

cloc [optins] <FILE|DIR> ...
$ find . -name "*.py" | xargs cloc
$ cloc package.tar.gz

# proprocessor options: '--diff', '--ignore-whitespace'
$ cloc --diff old.tar.bz  new.tar.bz2

# filter options: '--exclude-dir', '--exclude-lang', '--match-d', '--match-f'
$ cloc --show-lang

# output options: '--quiet', '--csv', '--report-file', '--xml', '--out'

from How to count lines of source code in Linux

How to create a self-extracting archive in Linux (using makeself)

makeself is a small shell script that generates a self-extractable tar.gz archive from a directory and adds a small shell script stub at the beginning of the archive to initiate self-extraction, and guide installation of extracted files.

# install from srpm, for el/centos
$ rpmbuild --rebuild http://dl.fedoraproject.org/pub/fedora/linux/development/rawhide/source/SRPMS/m/makeself-2.2.0-3.fc21.src.rpm
$ rpm -Uv makeself-*.rpm

# install for debian/ubuntu
$ sudo apt-get install makeself

# create a self-extractng archive
$ makeself.sh --notemp ./backup ./backup.run "SFX archive for backup" echo "Extraction done"

# extract
$ ./backup.run

# create a self-extracting installer
$ touch /program/installer.sh ; chmod 755 ./program/install.sh
$ makeself.sh ./program ./program.run "SFX installer for program" ./install.sh

from How to create a self-extracting archive or installer in Linux

How to checkpoint and restore a Linux process (using criu)

Checkpoint/Restore In Userspace – CRIU/CRIU@wiki is a tool to freeze a running application (or part of it) and checkpoint it to persistent storage as a collection of files. One can then use the files to restore and run the application from the point it was frozen at. Needs Linux kernel 3.9.

# install from binary, for debian/ubuntu
$ (debian/ubuntu) sudo apt-get install libprotobuf-dev libprotoc-dev protobuf-c-compiler libprotobuf-c0 libprotobuf-c0-dev
$ wget http://download.openvz.org/criu/criu-1.3.1.tar.bz2 -O - | tar -xj
$ make -C crtools-1.3.1

# install from srpm, for el/centos
$ sudo yum install protobuf-c-devel --enablerepo=epel
$ rpmbuild --rebuild http://dl.fedoraproject.org/pub/fedora/linux/development/rawhide/source/SRPMS/c/criu-1.3.1-1.fc22.src.rpm
$ rpm -Uv criu*.rpm

# dump, checkpoint a running process; process will terminate and its state will be stored in set of files in current directory
$ criu dump -t <PID>

# restore a dumped process to its original running state
$ criu restore -t <PID>

# if process launched from shell use '--shell-job'
$ criu dump -t <PID> --shell-job
$ criu restore -t <PID> --shell-job

# to re-parent restored process to init, use 'restore-detached'
$ criu restore -t <PID> --shell-job --restore-detached

# to checkpoint and restore a process w/ active TCP connections, use '--tcp-established'; uses http://lwn.net/Articles/495304/
$ criu dump -t <PID> --tcp-established
$ criu restore -t <PID> --tcp-established

# live migration application or container between machines, see http://criu.org/Live_migration
$ criu dump --tree <pid> --images-dir <path-to-existing-directory> --leave-stopped
$ scp -r <path-to-images-dir> <dst>:/<path-to-images>
$ criu restore --tree <pid> --images-dir <path-to-images>

from How to checkpoint and restore a Linux process

How to version control /etc in Linux (using etckeeper)

It is a good idea to “version control” everything in /etc directory, so that you can track configuration changes, or recover from a previous configuration state if need be.

etckeeper is a collection of tools for versioning content, specifically in /etc directory. Uses existing revision control systems (e.g., git, bzr, mercurial, or darcs) to store version history in a corresponding backend repository. It integrates with package managers (e.g., apt, yum) to automatically commit any changes made to /etc directory during package installation, upgrade or removal. It tracks file metadata that revison control systems do not normally support, but that is important for /etc, such as the permissions of /etc/shadow.

# install w/ git the default
$ (el/centos) sudo yum install etckeeper git-core | (debian/ubuntu) sudo aptitude install etckeeper git-core
# or w/ bzr
$ (el/centos) sudo yum install etckeeper bzr | (debian/ubuntu) sudo aptitude install etckeeper etckeeper-bzr
$ cat /etc/etckeeper/etckeeper.conf
VCS="bzr"

# setup
$ etckeeper init
$ etckeeper commit "initial commit"

# now can use regular git/bzr commands to handle further changes, or etckeeper vcs
$ etckeeper vcs status | sudo git status
$ etckeeper vcs diff /etc | sudo git diff /etc
$ etckeeper commit "any comment" | sudo git commit -m "any comment"
$ etckeeper vcs log /etc/sysconfig/*
$ etckeeper vcs diff -r1..3
$ etckeeper vcs diff -c3
$ etckeeper vcs revert --revision 2 /etc

# automatic commit changes made to /etc as part of package installation or upgrade
$ yum install httpd
$ etckeeper vcs log | git log --summary -1
$ etckeeper vcs diff -c5

# manually commit changes made to /etc by other commands
$ passwd someuser
$ git status
$ git commit -a -m "changed a password"

# remove/ignore some files
$ git rm --cached printcap # modified by CUPS
$ echo printcap >> .gitignore
$ git commit -a -m "don't track printcap"

# checkout a different /etc branch
$ git checkout april_first_joke_etc
$ etckeeper init

# use clone to backup /etc to a remove server
$ ssh server 'mkdir /etc-clone; cd /etc-clone; chmod 700 .; git init --bare'
$ git remote add backup ssh://server/etc-clone
$ git push backup --all

# multiple machines, start with a etckeeper repository on one machine, then add another machine's etckeeper repository as a git remote and diff/merge them, dont checkout
$ git remote add dodo ssh://dodo/etc
$ git fetch dodo
$ git diff dodo/master group |head

from How to version control /etc directory in Linux

How to detect and disable weak ciphers and SSL 2.0/3.0 in Apache and IIS (PCI Compliance, poodlebleed)

SSLProtocol all -SSLv2 -SSLv3
SSLHonorCipherOrder on
SSLCipherSuite "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH+aRSA+RC4 EECDH EDH+aRSA RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS"
# test for sslv2
$ openssl s_client -connect localhost:443 -ssl3

# disable sslv2 in apache
SSLProtocol -ALL +SSLv3 +TLSv1

# and in iis
[HKEY_LOCAL_MACHINESYSTEMCurrentControlSetControlSecurityProvidersSCHANNELProtocols{PCT 1.0,SSL 2.0}Server]
"Enabled"=dword:00000000

# test for weak ssl ciphers
$ openssl s_client -connect SERVERNAME:443 -cipher LOW:EXP

# disable weak ssl ciphers in apache
$ SSLCipherSuite ALL:!aNULL:!ADH:!eNULL:!LOW:!EXP:RC4+RSA:+HIGH:+MEDIUM

# and in iis
[HKEY_LOCAL_MACHINESYSTEMCurrentControlSetControlSecurityProvidersSCHANNELCiphers{DES 56/56,NULL,RC2 40/128,RC2 56/128,RC4 40/128,RC4 56/128,RC4 64/128}]
"Enabled"=dword:00000000
# either enable all except sslv2/3
SSLProtocol All -SSLv2 -SSLv3

# or disable everything except tlsv1.x
(el6) SSLProtocol -All +TLSv1
(el7) SSLProtocol -All +TLSv1 +TLSv1.1 +TLSv1.2

# and for ngix
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

# and in iis
[HKey_Local_MachineSystemCurrentControlSetControlSecurityProviders SCHANNELProtocolsSSL 3.0Server]
"Enabled"=dword:00000000
$ sudo yum install sslscan (epel) | sudo apt-get install sslscan
$ sslscan <host>

How to add additional repositories to CentOS/RHEL (EPEL, …)

  • Extra Packages for Enterprise Linux EPEL: rebuilds of Fedora packages for EL
## EPEL is included on CentOS extras, so for CentOS just
$ yum install -y epel-release

## for RHEL
$ cat /etc/yum.repos.d/epel-bootstrap.repo
[epel]
name=Bootstrap EPEL
mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=epel-\$releasever&arch=\$basearch
failovermethod=priority
enabled=0
gpgcheck=0
$ yum --enablerepo=epel -y install epel-release
$ rm -f /etc/yum.repos.d/epel-bootstrap.repo

# import GPG key
$ export $V=`rpm -q --queryformat '%{VERSION}' $(rpm -qa '(redhat|sl|centos|raclelinux)-release(|-server|-workstation|-client|-computenode)')`
$ sudo sudo rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-$V

$ yum repolist
  • The Community Enterprise Linux Repository ELRepo: kmod driver packages to enhance hardware support. Newer kernels are also available
$ rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org

$ rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-2.el7.elrepo.noarch.rpm
or
$ rpm -Uvh http://www.elrepo.org/elrepo-release-6-6.el6.elrepo.noarch.rpm
or
$ rpm -Uvh http://www.elrepo.org/elrepo-release-5-5.el5.elrepo.noarch.rpm
$ rpm -qa|grep redhat-|xargs rpm -e

$ rpm --import http://isoredirect.centos.org/centos/7/os/x86_64/RPM-GPG-KEY-CentOS-7
$ rpm -Uvh http://mirror.centos.org/centos/7/os/x86_64/Packages/{centos-release-7-0.1406.el7.centos.2.3.x86_64.rpm,centos-logos-70.0.6-1.el7.centos.noarch.rpm}

How to capture and playback network traffic in Linux (using tcpreplay)

Useful for debugging and testing.

$ sudo apt-get install tcpreplay | sudo yum install tcpreplay (needs EPEL)
# set network interface in promiscuous mode, http://en.wikipedia.org/wiki/Promiscuous_mode
net-tools: $ ifconfig eth0 promisc | ifconfig eth0 -promisc
iproute2: $ ip link set dev eth0 promisc on | ip link set dev eth0 promisc off | ip link show dev eth0

# setup promiscuous mode permanently in rhel/centos
$ cat /etc/sysconfig/network-scripts/ifcfg-eth0
  PROMISC=yes
$ systemctl restart network
(and in debian/ubuntu)
$ cat /etc/network/interfaces ... 
 auto eth0
 iface eth0 inet manual
 up ifconfig eth0 promisc up
 down ifconfig eth0 promisc down

# capture live network traffic, and dump it to a pcap file
$ sudo tcpdump -w dump.pcap -i eth0
# Rewrite any destination IP address and MAC address
$ tcprewrite --infile=dump.pcap --outfile=temp1.pcap --dstipmap=0.0.0.0/0:192.168.1.20 --enet-dmac=E0:DB:55:CC:13:F1

# Rewrite any source IP address and MAC address
$ tcprewrite --infile=temp1.pcap --outfile=temp2.pcap --srcipmap=0.0.0.0/0:192.168.1.10 --enet-smac=84:A5:C8:BB:58:1A

# Rewriting TCP/UDP Ports
$ tcprewrite --infile=example.pcap --outfile=new.pcap --portmap=80:8080

# Update the checksum of every packet
$ tcprewrite --infile=temp2.pcap --outfile=final.pcap --fixcsum

# Replay
$ sudo tcpreplay --intf1=eth0 final.pcap

# Replay looping 100 times
$ sudo tcpreplay --loop=100 --intf1=eth0 final.pcap

# Replay 5x faster then original
$ sudo tcpreplay --multiplier=5.0 --intf1=eth0 final.pcap

# Replay at 10Mbps
$ sudo tcpreplay --mbps=10.0 --intf1=eth0 final.pcap

# Replay at 100 packets/sec
$ sudo tcpreplay --pps=100 --intf1=eth0 final.pcap

# Infinite replay
$ sudo tcpreplay --loop=0 --intf1=eth0 final.pcap

from tcpreplay@xmodulo