[off topic] [rants] GNU Make - missing features

1. For all "declaration" nature of make, it is still very very hard to build your own rule out of predefined ones found in make database. e.g. making a rule for new file extension to call existing rule for older file extension is only possible with dumb copy-and-pasting. Now imaging project with hundreds makefiles - all at different times doing such copy-and-pasting: one hell to do maintenance.
   
2. Arithmetics. I was really surprised to find that make doesn't have any internal facility for arithmetic expression calculation. Even more so I was surprised when I learned that GNU project doesn't even have such library. Best call - is to call external optional program 'dc' or hope that bash is installed and do arithmetics through it. That bars many accessibility features like progress counter for long build processes. Also obviously renders makefiles unportable.
   
3. Clean mechanism to overwrite rules. That's actually what makes proper dependency check implementation impossible. E.g. if you have dependency "a: b" and "a: c" that would really mean "a: b c" - and there is no way to cancel the "a: b" dependency.
4. List and string routines. Again, mere convenience, yet has performance impact. To put it simply: it is very stupid when to transform file name I have to call several external programs. String manipulation is needed for file name transformations. List routines are needed for proper management of list of targets scheduled to build.
5. Recursiveness. The dead horse was beaten many times before. Lack of proper recursiveness support bars use of make on very large projects. Or leads to arcane make files understood by few thus having very very low accessibility. One can't require everybody on project to be a make specialist.

Looking at all the rants, my mind instantly points out that the list is precisely difference between make and cons. Last update to cons - pcons - was in 2001 and (thanks to that fact that it was written in Perl) it still works perfectly in all situations I have been to.

I had very ridiculous accident: people using pcons were told to switch to make ("Of course there is no reasons to it - it is just our corporate policy"). Case looked bleak: dumbing down build system back to stone age of make wasn't problem - problem was the lost functionality (the list above). I have advised (and it actually worked) simple solution: put pcons right along with sources and create makefiles with all targets calling cons instead. Worked beautifully - thanks to slim nature of cons/pcons which is written in Perl and has little/no external dependencies.

[off topic] One-liner to make GNU Info usable

Something like this should be shipped along with GNU Info documentation system - to make it any useful:

$ cat ~/usr/bin/iinfo
#!/bin/bash
info --subnodes --output=- "$@" 2>/dev/null | less
$

Option "--subnodes" tells info to dump everything it has - w/o frigging inhuman document structure. Option "--output=-" tells to dump that all on stdout. The bit "2>/dev/null" is to shut up the stupid useless stuff info spits on stderr. And '| less' bit tells it to behave like proper "man".

Rants. Ironically, even the one-liner cannot make GNU tools any better: they show their age, were not updates in years and do not support most modern things one expects. Searching internet generally brings more and better results than digging outdated GNU documentation. (N.B. GCC of all GNU tools is kind of exception.)

[off topic] message logging severities

I can recall only five levels of severities:

MAJOR: Alarm. Critical unrecoverable error. Results in certain termination of application.
MINOR: Recoverable error. Application can go on, but something is fishy.
NOTICE: Not an error, but shouldn't have happened.
INFO: Run-time information statistics, "anomalies" a.k.a. unsolicited messages. Must produce passable amount of information since might be used for monitoring of real system.
TRACE: Used to trace what's happening in the system: functions called and their arguments. Produces insane amount of output.
DEBUG: Debugging info. Developer's corner.

Production systems run on MINOR level, so that all errors are displayed. MAJOR level normally has abort() built in so that OS would dump core of application for further investigations.

Systems in testing run with INFO level. Often the output is saved and required to match in repeated tests. E.g. if we have fed application with 1000 bytes of info, we would expect INFO message to reflect that 1000 bytes where handled. Not 999, not 1001, not 500 + 500.

Edit1: Added "TRACE" level.

Off-Topic :: Wait-Free/Lock-less synchronization crash course

Well, results are not that good as I have hoped. (In depth info at Wikipedia. Read the first reference on the page written by Maurice Herlihy in 1993.)

1. Rule of thumb: lock-less can only be data structures which synchronize using single datum.
2. Compare-And-Swap( ptr, old, new ) can be used for implementation of atomic counters. The operation is supported by all high performance platforms (e.g. PowerPC & IA-32).
   
3. One cannot make lock-less FIFO for N:M (N readers/M writers) nor for 1:M (1 reader/M writers) configurations. Seems that 1:1 (1 read and 1 writer) only possible configuration and in fact it doesn't require any special support at CPU level. (Reference to rule of thumb: FIFO requires two data items for implementation - head/tail (or next/prev).)
   
4. Stack (LIFO) can be implemented wait-free/lock-less using Compare-And-Swap() op, since it needs only single pointer for implementation - head - pointing to first element to remove/last element inserted.

Conclusion:

1. Unixes were 100% right putting pipes at foundation of I/O: pipe can be implemented as couple of FIFOs and as long as pipe isn't shared between more than 2 applications, it can be completely lock free.
   
2. Stack (along with Compare-And-Swap op) allows to implement memory management (for static block size) very efficiently. (Edit1. In preemptive SMP OS, that might lead to situation when Compare-And-Swap would succeed despite fact that state of queue already changed. Proper implementation for multiple CPUs would use several (at minimum 2) stacks. Allocations are made from one stack, freeing is done on next stack. Allocation function rotates stacks when detects that current stack is empty. Allocation is failed when all stacks were queried at least once and all are empty. Alternatively, one can introduce additional counter: generation counter. That way Compare-And-Swap needs to be able to write double word. When element is allocated, generation counter is incremented. When element is freed, we put on stack pair - pointer and generation counter - which would be with high probability unique.)
   
3. Message passing systems suck big time because (i) 1:1 FIFOs are inapplicable and (ii) it is impossible to optimize memory allocation with lock-less stack since it is impossible to tell where the message would end up. (The later can only be implemented in worst case scenario of shared memory.)
   

vim 7.0 :: tabs / tabbed editing

Not that I fun of all the GUI niceties, but some people cannot live without mouse and think :b#/:bn/:bp are bore. So vim7 has added a way for mouse fans (*) to work with multiple files.

For tab fans, Linux.com published intro into vim7' tabs: Vim tips: Using tabs.

(*) How people edit text files with mouse is beyond me. Do they use charmap or what? [/sarcasm]

Sample Ex/Vim Ranges

Here we go.

:%s/[a-z]/\u&/g

Substitute in all lines (the range defined by '%') all lower case latin letters ('[a-z]') with upper case letters ('[A-Z]'). Substitute in all lines (the range defined by '%') every lower case latin letter ('[a-z]') with upper-case letter ('\u&', see for more info :h s/\u).

:<'>' w new_file_from_selection.txt

Write ('w') all selected lines ("<'>'") into new_file_from_selection.txt. (You do not need to type that "<'>'" thing. Select lines in visual mode (entered by Shift-V) and then press ':' - vim will put the <'>' in prompt line for you)

:5,+99 w new_file.txt

Write ('w') to the file "new_file.txt" lines: 5th and next 99 ones included, total of 100 lines. (Provided that the actual file has minimum 1054 lines).

:0,$ ! tac

Filter ('!') all lines ('%' is simple shortcut for '0,$') through the external application 'tac'. The filter command would then replace all affected lines with the output of the application - in that case the lines would reappear in inverse order, thanks to the 'tac' command. ('tac' is of course opposite of 'cat'. man tac.).

:<'>' ! wc -l

That would replace all selected (in visual mode) lines with the count of selected lines, as counted by 'wc -l'.

And to conclude, the most useless (unless you're vim scripting addict) range example:

:.=

Print current line (range '.') number (command '=').

Edit1 Fix the uppercase command, as per anon comment. It should have been some real weird booze I had that day. N.B. more optimal is of course this: :%s/.\+/\U&/.

ZQ

[Linux.com] Vim tips: Working with external commands

Vim tips: Working with external commands

Off Topic :: GNU Arch (TLA) Commands Quick Reference

User identification

$ tla my-id  "John Doe <user@aol.com>"
$ tla my-id                       # to print currently set id

Create Archive

$ mkdir  ~/{archives}
$ tla make-archive  user@aol.com--2006-quickref ~/{archives}/2006-quickref

Find Archives

$ tla archives                    # to print all available archives
$ tla whereis-archive user@aol.com--2006-quickref

Default Archive

$ tla  my-default-archive  user@aol.com--2006-quickref
$ tla  my-default-archive         # to print it
$ tla  my-default-archive -d      # to delete the preference

Provisioning Archive Space

$ tla archive-setup hello-world--mainline--1.0
$ tla categories                  # prints hello-world
$ tla branches hello-world        # prints hello-world--mainline
$ tla versions hello-world--mainline        # prints hello-world--mainline--1.0

Initializing before Import

$ cd ~/wd/hello-world
$ tla init-tree hello-world--mainline--1.0
$ tla tree-root                   # prints /home/jdoe/wd/hello-world
$ tla tree-version                # prints user@aol.com--2006-quickref/hello-world--mainline--1.0
$ tla log-versions                # prints user@aol.com--2006-quickref/hello-world--mainline--1.0
$ tla set-tree-version [new-version-name]
$ tla add-log-version [new-log-name]
$ tla remove-log-version [existing-log-name]

Checking Inventory

$ tla inventory --names --source  # to see what arch classifies as sources
$ tla inventory --source --ids    # to see ids given by arch to sources
$ tla inventory --names --backups # to see what arch classifies as backups
$ tla inventory --junk | xargs rm # remove all what arch think is junk

Adding/Removing Files

$ tla add hw.c main.c
$ tla delete hw.c
$ mv hw.c hello.c
$ tla move hw.c hello.c
$ tla tree-lint   # to see if all clean, tla won't import/ci from unclean directory

Import (Initial check in)

$ tla make-log                    # create log file and print its name
$ vim `tla make-log`              # edit log in vim
$ tla import                      # copy working copy into archive
$ tla revisions hello-world--mainline--1.0  # prints base-0
$ tla revisions --summary --creator --date hello-world--mainline--1.0
$ tla log-revisions
$ tla logs hello-world--mainline--0.1       # prints base-0
$ tla logs --summary --creator --date hello-world--mainline--1.0
$ tla cat-log hello-world--mainline--1.0--base-0

Committing Changes

$ tla changes --diffs             # or (in tla 1.3) just "tla diff"
$ tla make-log                    # create log, required
$ tla commit                      # do it, creates patch-1
$ tla commit -s "Summary" -L "Log Message"            # commit with log auto generated
$ tla revisions --summary hello-world--mainline--1.0  # last revision info
$ tla logs --summary hello-world--mainline--1.0       # last log entry info

Checking Out

$ tla my-default-archive          # check where is default archive is
$ tla whereis-archive user@aol.com--2006-example      # display local/net path
$ tla categories        # lists available project in default archive "hello-world"
$ tla branches hello-world        # lists available branches of the project "*--mainline"
$ tla versions hello-world--mainline        # lists available version of the branch "*--0.1"
$ tla revisions hello-world--mainline--0.1  # lists available revisions base-0 & patch-1
$ tla get hello-world--mainline--0.1 hello-world                # check out most recent revision of into "hello-world" directory, or
$ tla get hello-world--mainline--0.1--base-0 hello-world        # check out specific revision
$ ls                    # should list "hello-world"

Update (Replay)

$ tla undo              # back off local changes
$ tla replay            # get latest patches from repository
$ tla redo              # reapply local changes
$ tla update            # all of three steps above with one command.

Networked Archives

$ tla register-archive lord@emf.net--2003b          
   http://regexps.srparish.net/{archives}/lord@emf.net--2003b    # HTTP, R/O
$ tla register-archive lord@regexps.com--2002
   ftp://ftp.regexps.com/{archives}/lord@regexps.com--2002       # FTP, R/W
$ tla register-archive user@aol.com--2006
   sftp://random.homeunix.org/home/jdoe/{archives}/2006-quickref # SFTP, R/W
$ tla my-default-archive [archive-name]      # and the rest is just like described above

Mirroring (Locally) Network Archives

$ tla register-archive lord@emf.net--2003b-SOURCE $remote_location   # once
$ tla make-archive --mirror-from lord@emf.net--2003b-SOURCE $local_location # once
$ tla archive-mirror lord@emf.net--2003b    # repeat to pull latest changes

Mirroring (Remotely) Local Archives

$ tla make-archive --mirror mine@somewhere.com $remote_location      # once
$ tla archive-mirror mine@somewhere.com     # repeat to push latest changes

P.S. Made after this tutorial.

More from that Oualline person

Top 10 things Vi user need to know about Vim.

Very short and clear.

Vim Cookbook by Steve Oualline

Vim Cookbook by Steve Oualline. Interesting reading. Lots of little tips.

P.S. Delves a bit into ranges supported in ex commands. I still cannot get myself to write entry on vim ranges.