“People got OO all wrong, its not about classes or objects. The big idea is messaging” (from Allen Kay)

Just a gentle reminder that I took some pains at the last OOPSLA to try to remind everyone that Smalltalk is not only NOT its syntax or the class library, it is not even about classes. I’m sorry that I long ago coined the term “objects” for this topic because it gets many people to focus on the lesser idea.

The big idea is “messaging” — that is what the kernal of Smalltalk/Squeak is all about (and it’s something that was never quite completed in our
Xerox PARC phase).

from Kay, Allen. “prototypes vs classes was: Re: Sun’s HotSpot”

Kay is one of the fathers of the idea of object-oriented programming, inventor of Smalltalk.

How to navigate/browse shell history in Linux (using hstr/hh)

hstr@github is a ncurses used to view, navigate, select and search command history from bash and zsh. Its an improved history|grep or ctrl-r.

$ wget https://github.com/dvorka/hstr/releases/download/1.16/hh-1.16-bin-64b.tgz -O - | tar -xz

# ncurses list of history filtered by PATTERN
# use options
$ HH_CONFIG=hicolor,regexp hh

from hstr: history the easy way

How to monitoring/show the progress of CLI tools in Linux (using cv and pv)

cv/coreutils viewer@github supports all basic utilities in coreutils package. It is written in C and shows the progress as percentage. Works by scanning /proc for commands that it supports, checks directories fd and fdinfo for opened files and seek positions and finally reports the progress for the largest file.

$ apt-get|yum|yaourt cv

'-m,--monitor' loop while monitored processes are still running
'-w,--wait' estimate I/O throughput and ETA
'-c,--command command' monitor only this command

# see all current and upcoming coreutils commands
$ watch cv -q
# see downloads progress
$ watch cv -wc firefox
# launch and monitor any heavy command using '$!'
$ cp bigfile newfile & cv -mp $!

from cv: progress bar for cp, mv, rm, dd…

pv/pipe viewer/pv@man is a terminal-based tool for monitoring the progress of data through a pipeline.

$ apt-get|yum|pacman install pv

command1 | pv | command2
pv input.file | command1 | pv > output.file

# display options
'-p,--progress/-t,--timer/-e,--eta/-b,--byte' show progress/timer/ETA/bytes
# output modifiers
'-N,--name NAME' prefix the output information
'-c,--cursor' use cursor positioning escape sequences
'-l,--line-mode' instead of counting bytes, count lines
'-s,--size SIZE' assume the total amount of data to be transferred is SIZE
# data transfer modifiers
'-L,--rate-limit RATE' limite transfer rate by bytes/sec
'-B,--buffer-size BYTES'

# watch how quickly a file is transferred using nc
$ pv file | nc -w 1 host 3000
# see progress of both pipes
$ pv -cN rawlogfile file.log | gzip | pv -cN gziplogfile > file.log.gz
# with ncurses's dialog
$ (pv -n backup.tar.gz | tar xzf - -C path/to/data ) 2>&1 | dialog --gauge "Running tar, please wait..." 10 70 0
# rsync and pv
$ rsync options source dest | pv -lpes Number-Of-Files

from pv@nixcraft and rsync and pv@nixcraft

How to lock your Linux terminal (using vlock and physlock)

vlock is a program to lock one or more sessions on the Linux console. This is especially useful for Linux machines which have multiple users with access to the console. One user may lock his or her session(s) while still allowing other users to use the system on other virtual consoles.

$(el) yum install kbd
$(deb) apt-get install 
$(arch) pacman -Sy kdb

'-c,--current' lock only this virtual console, allowing user to switch to other virtual consoles
'-a,--all' lock all virtual consoles by preventing other users from switching virtual consoles
'-n,--new' switch to a new virtual console before locking all console sessions, needs to be compiled with plugin

# lock current console
$ vlock
# lock all consoles (including virtual console); must run from virtual console
$ vlock -a
# lock whole system, switches to a virtual console and then locks as 'vlock -a'
$ sudo vlock -n

from vlock: lock your terminal (or system)

physlock@github lightweight alternative to vlock, equivalent to `vlock -an’. It is written because vlock blocks some linux kernel mechanisms like hibernate and suspend and can therefore only be used with some limitations.

$ git clone https://github.com/muennich/physlock
$ cd physlock ; make ; sudo make install

'-d' fork and detach, useful for suspend/hibernate scripts
'-l' only lock console switching
'-L' only enable console switching
'-u USER'  allow the given user to unlock the computer

$ /usr/local/bin/physlock

Using pacapt universal package manager (pacman for all)

pacapt is a shell script that wraps native package managers in a pacman@arch syntax. Intended for Arch users having to temporarily deal with another distribution.

It supports pacman by arch, dpkg/apt-get by debian/ubuntu, homebrew by OSX, yum/rpm by redhat/centos/fedora, portage by gentoo, zypper by opensuse, pkgng by freebsd, pkg_tools by openbsd. See pacman_rosetta@arch

## install
$ wget -O /usr/local/bin/pacapt https://github.com/icy/pacapt/raw/ng/pacapt
$ chmod 755 /usr/local/bin/pacapt

'-Q' list packages installed
'-Qq FILE' query which package provides file
'-Ql PACKAGE' list package files
'-S PACKAGE' install package
'-Ss PACKAGE' search package
'-Suy' upgrade system and package db
'-R PACKAGE' remove package
'-Scc' delete all downloaded packages

How to remote syslog in Linux (using rsyslog)

rsyslog is an open-source implementation of syslog protocol / rfc3164 and extends it with content-based filtering, rich filtering capabilities, flexible configuration options and adds features such as using TCP for transport. Its used prior to migration to systemd-journald.

  • Facility level is type of processes to monitor: auth, cron, daemon, kernel, local0..local7
  • Severity/Priority level is type of log message: emerg/0, alert/1_, crit/2, err/3, warn/4, notice/5, info/6, debug/7
  • Destination is either local file or remote rsyslog server @ip:port

As a rsyslog client it can filter and sends internal log messages to either local file system or a remote rsyslog server. As rsyslog server it collects logs from other hosts and sends them into internal log messages. See syslogserver@windows.

$ yum install rsyslog | apt-get install rsyslog | pacman -S rsyslog

##(server) enable listener
$(host1) vi /etc/rsyslog.conf
# udp
$ModLoad imudp 
$UDPServerRun 514
# tcp (slower but more reliable)
$ModLoad imtcp 
$InputTCPServerRun 514 

##(server) create template to log to filesystem
# see http://linux.die.net/man/5/rsyslog.conf
$(host1) vi /etc/rsyslog.d/remote_host
# log everything to 'host/progname.log'
$template RemoteLogs,"/var/log/%HOSTNAME%/%PROGRAMNAME%.log" *
# format it '[facility-level].[severity-level] ?RemoteLogs'
*.* ?RemoteLogs 
# stop processing messages
& ~

# same but using ip
$ vi /etc/rsyslog.d/remote_ip
$template IpTemplate,"/var/log/%FROMHOST-IP%.log" 
*.*  ?IpTemplate 
& ~

##(client) route all messages to remote server
$(host2) vi /etc/rsyslog.d/route_all
*.*  @host1:514 
# same but using tcp instead
#*.*  @@host1:514
# same but only for some kernel facility
kern.* @

$(both) service rsyslog restart | systemctl restart rsyslog

from rsyslog server@xmodulo and rsyslog client@xmodulo

syslog(3) is the syscall used to send messages to system logger. There are wrappers in all languages, including shells

## from shell
# see http://linux.die.net/man/1/logger
$ logger -p local0.info -t PROGNAME MESSAGE

## forward journald to local syslog daemon
# see http://www.freedesktop.org/software/systemd/man/journald.conf.html
$ vi {/etc,/run,/usr/lib}/systemd/journald.conf.d/*.conf
# same as kernel command line option 'systemd.journald.forward_to_syslog=True'

How to USB flash a bootable Linux installation media (using dd/ddrescue and testdrive)

dd@man/dd@wiki is a command-line utility for Unix and Unix-like operating systems whose primary purpose is to convert and copy files.

$ yum|apt-get|pacman coreutils

# find device name
$ lsblk
# unmount it
$ sudo umount /dev/sdx

# write/data transfer
$ sudo dd bs=4M if=/path/to/linux.iso of=/dev/sdx && sync

from usb flash@arch

gnu ddrescue@man combines both dd_rescue’s ability to read big blocks and then shift gears, with dd_rhelp’s ability to remember what parts of the disk have been looked at already. It’s written in C++ and it’s small and fast.

$(deb) apt-get install gddrescue 
$(el) yum install ddrescue 
$(arch) pacman -Su ddrescue

'-d,--direct' direct disc access for input file
'-D,--synchronous' use synchronous writes for output file

$ sudo ddrescue -d -D --force /path/to/linux.iso /dev/sdx

from disk drive recovery: ddrescue, dd_rescue, dd_rhelp

testdisk@arch can be used to recover if you dd/ddrescue to the wrong device

How to write a Linux kernel module

Kernel modules are pieces of code that can be loaded and unloaded into the kernel upon demand. They extend the functionality of the kernel without the need to reboot the system.

## obtaining information
# list loaded modules
$ lsmod
# show module info
$ modinfo MODULENAME
# list dependencies
$ modprobe --show-depends MODULENAME

## automatic module load
# configure udev/systemd-modules to what modules to load at boot, see 'man modules-load.d'
$ vi {/etc,/run,/usr/lib}/modules-load.d/PROGRAM.conf

## manual module load
# load by name
$ modprobe MODULENAME
# load by filename from '/lib/modules/$(uname -r)/'
$ insmod FILENAME [ARGS]
# unload module
$ modprobe -r MODULENAME
# same

## passing parameters to module
# either from '/etc/modprobe.d'
$ vi /etc/modprobe.d/FILENAME.conf
options MODULENAME parametername=parametervalue
# or from kernel command line

## blacklisting: prevent the kernel module from loading
# either from '/etc/modprobe.d'
$ vi /etc/modprobe.d/FILENAME.conf
blacklist MODULENAME
# or from kernel command line

from kernel modules@arch

You can write your own modules, see the linux kernel module programming guide.

# install build dependencies (kernel source)
$(deb) apt-get install build-essential linux-headers-$(uname -r)
$(el) yum install yum install gcc gcc-c++ make kernel-headers
$(arch) pacman -Syu base-devel linux-headers

# write a hello world module
$ vi hello.c
#include <linux/module.h> // all kernel modules
#include <linux/kernel.h> // KERN_EMERG, KERN_ALERT, KERN_CRIT, ... 
#include <linux/init.h>   // __init and __exit macros
MODULE_DESCRIPTION("A Simple Hello World module");
static int __init hello_init(void) {
    printk(KERN_NOTICE "Hello world!n");
    return 0; // non-0 means init_module failed
static void __exit hello_cleanup(void) {
    printk(KERN_NOTICE "Cleaning up module.n");

$ Makefile
obj-m += hello.o
    make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
    make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

# testing
$ make ; sudo insmod hello.ko
$ dmesg|grep -i hello
$ sudo rmmod hello.ko

from how to write your own linux kernel module

Using homebrew and cask package managers in OSX

Homebrew is a package management system for OSX. It installs packages in its own cellar directory and symblinkes to /usr/local.

Homebrew terminology: Formula is the package definition, keg is formula installation prefix, cellar is where all kegs are installed, tap is optional formula repositories, bottle is pre-built binary keg that can be unpacked.

## install
$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
$ brew --version

## using
# update formulas
$ brew update
# install/uninstall/upgrade/info/search packages
$ brew install|uninstall|upgrade|info|search "package"
# list installed
$ brew list

## contributing
# forked https://github.com/Homebrew/homebrew and clone
$ git clone https://github.com/<username>/homebrew.git
# create new formula
$ brew create https://foo.com/bar-1.0.tgz
$ vi $HOMEBREW_REPOSITORY/Library/Formula/foo.rb
$ git checkout -b foo
$ git commit Library/Formula/foo.rb && git push
# open pull request

## using tap, optional repository git formulas
$ brew tap homebrew/science
$ brew install "formula"

from homebrew, formula cookbook, brew tap and interesting taps and branches

Homebrew cask extends homebrew and brings its elegance, simplicity, and speed to OS X applications and large binaries alike. Applications are kept in their Caskroom under /opt and symblinked to $HOME/Applications.

## install
$ brew install caskroom/cask/brew-cask

## using
$ brew cask install google-chrome
$ open ~/Applications/"Google Chrome.app"

## contributing
# forked https://github.com/caskroom/homebrew-cask/fork and clone
$ git clone https://github.com//homebrew-cask.git
# add official repo as remote
$ git remote add upstream https://github.com/caskroom/homebrew-cask.git
# symb link to use brew cask in your private repo
$ $HOME/homebrew-cask/developer/bin/develop_brew_cask
# switch back to official repo to run 'brew update'
$ $HOME/homebrew-cask/developer/bin/production_brew_cask

from homebrew cask and hacking on homebrew-cask

How to setup a GNU/Linux-like environment in Windows (using cygwin, mingw, gow or msys2)

cygwin/cygwin@wiki is a Unix-like environment and command-line interface for Windows. Cygwin consists of two parts: a dynamic-link library (DLL) as an API compatibility layer providing a substantial part of the POSIX API functionality, and an extensive collection of software tools and applications that provide a Unix-like look and feel.

%comspec% cinst cygwin -y (or https://cygwin.com/install.html)

# open bash terminal
%comspec% %CYGWINPATH%/bin bash.exe --login -i
# open a terminal emulator
%comspec% %CYGWINPATH%/bin mintty.exe

from MinTTY Gives Cygwin a Native Windows Interface

mingw@fedora/mingw-w64 brings free software toolchains to Windows. It hosts a vibrant community which builds and debugs software for Windows while providing development environment for everyone to use.

$ vi hello.c
#include <stdio.h>
int main () { printf ("Hello world!n"); return 0; }

## build using 'gcc', dependent on 'cygwin1.dll' 3.2Mb
# open cygwinsetup.exe and install 'gcc' 
$ gcc hello.c -o hello-gcc.exe

## builds using 'mingw64', dependent on 'msvcrt.dll' / native
# open cygwinsetup.exe and install 'mingw64-x86_64' or 'mingw64-i686'
# note: http://www.delorie.com/howto/cygwin/mno-cygwin-howto.html
$ x86_64-w64-mingw32-gcc hello.c -o hello-mingw64.exe
# or ./configure --host=x86_64-w64-mingw32 ...

gow@github (Gnu On Windows) is the lightweight alternative to cygwin. It uses a convenient Windows installer that installs about 130 extremely useful open source UNIX applications compiled as native win32 binaries.

%comspec% cinst gow -y (or https://github.com/bmatzelle/gow/releases)
# note: it adds gowbin to PATH

# list available commands
%comspec% gow.bat -l

# execute bash shell script
%comspec% bash.exe script.sh [script options]

from gow@tuxdiary

msys2 (Minimal SYStem 2) is a fork of cygwin focus on Windows interop dropping the Posix, using MinGW-w64 toolchains. Also ported Arch’s Pacman for easy package management.

# see http://sourceforge.net/p/msys2/wiki/MSYS2%20installation/

# open a shell
%comspec% %MSYS64PATH%/msys2_shell.bat

# install new package
$ pacman -Suy PACKAGE
# search package
$ pacman -Ss PATTERN
# list packages installed
$ pacman -Q

# build using 'mingw64' or 'gcc', both depend in 'msys-2.0.dll' 3.2Mb
$ x86_64-pc-msys-gcc hello.c -o hello-mingw64.exe
$ pacman -Syu gcc
$ gcc hello.c -o hello-msys2.exe

from msys2@tuxdiary