[link] Vi Lovers Home Page

Vi Lovers Home Page - terrific resource for everybody. Piles of links - tutorials and FAQs.

Must read.

[link] Gentoo's VI guide

Learning vi

Pretty good for beginners - just like about any other document I have seen from Gentoo.

[off topic] Divide and Conquer (DaC) vs. Model-View-Controller (MVC)

Thinking about team work organization, I was constantly humming "Divide and Conquer" (DaC) principle. That's pretty obvious since it is guiding principle how big teams are taking on large problems - by splitting them into many smaller ones. (Small enough for single person to solve.)

Turning back to work, my eyes have met again with several of my constructs loosely based on MVC model.

Sudden enlightenment: MVC is another application of more general DaC.

How DaC works. A (simple) problem is split into three (primitive) problems:

• 2 smaller problems of original bigger problem and (one more piece everybody forgets about)

• problem of integration the two smaller solutions into solution for bigger problem.

The third part - often forgotten line connecting two blocks - is very important and often play crucial development organization role. Recurrently applying DaC we can slice problems until they reach manageable size.

How MVC works. A problem is split into three problems:

• stateful part containing all data/state (model),

• stateless part containing all utilities needed to work with model (view) and

• controller part which best exemplified by UI to manage the data of model using the utility of view.

MVC generally drawn as three blocks, with model being independent, view depending solely on model and controller depending on both model and view.

Similarities are obvious.

On high level, MVC's controller is the interface connecting the two pieces of problem - model (state) and view (functions). (Bigger problem split into two smaller problems - model and view - with controller playing role of integration problem.)

On lower level, MVC looks like DaC splitting original problem into three smaller problems and plus three more integration problems (view to model, controller to view and to model).

Clever tricks from VIM's Bram Moolenaar

The page where I found tip for the annoying PuTTY problem, also had great tip:

:help usr_12.txt

There you will find clever tricks you can do with VIM to accomplish simple yet sometimes needed tasks. And overall page has lots of simple hints. Very good reading.

VIM vs. PuTTY keypad

The problem was annoying me for very long time: under PuTTY, applications (e.g. less, vim) using tek window (alternative terminal screen) had totally screwed keypad. Solution was found here.
Apparently, PuTTY tries to do something smart. And fails.
Workaround:

1. Change function key/keypad mapping from standard "ESC[n~" to "Linux".
2. Disable application keypad mode

Apparently only second is required, but first is also recommended.
The options, as usual to all options of Windows applications, change their location in settings dialog periodically so I cannot give their precise locations. Go through PuTTY setting and you will find them.

[off topic] Driving Principles of Programming

I often asked on how software development works. To most people unfamiliar with all stinky innards of development process, figuring out how to get anything out of developers is very very tricky. Sometimes people see logic behind what I do - but often they are lost when trying to deal with me.

So as an insight into developer's head I would explain shortly here two of my software development principles. "Shortly" and "two" - because I have managed to date to formulate only one (second) of them and first was well know before I was even born.

1. This is grand principle which regulates lion share of decisions made by software developer. LAZINESS. Some think of it as of bad habit, but in as old programmers' proverb goes to find simplest solution to a problem, assign the problem to laziest programmer you have. In development and programming, laziness is source of all simple decisions which are often by outsiders are seen as "original" or even "genius". To programmers - it's just N days of work saved.

1a. Corollary. In development team with no lazy people, one would hardly see any innovation - or properly working program. After all, non-lazy people can make even pigs flying.

2. It just dawned on me recently and I have managed to formulate my second most used programming principle. EGOISM. (I would claim nothing and rather expect that somebody already did research that before me.) The principle is very simple: make something what would be useful to you and most likely others would find it also useful. Check the Unix made of thousands of small and big utilities - most of the utilities were initially developed for particular purpose of its author himself. But later were also found by others to be useful. Well, Unix itself was made for very particular purpose - and none of its developers even thought that Unix would catch up. Yet it did. Why? Because first and foremost it was useful to its creators. They didn't cared much what others did with it - how egoistic of them!? ;)

2a. Corollary. Excessive non-egoistic programming results in piles of abstract interfaces made to abstract other interfaces which abstract other interfaces. They do not do anything in particular - they are written often because people are told to write something. It is hard to make something useful - if you are just a little gear which needs only to mesh with other such gears. And to mesh better - we need more interfaces, abstraction layers and level of indirection. It also happens to be a generic sickness of many successful projects which in beginning "just work", but authors along the way get bored and start adding needless minor features (by request of vocal user minority) which in the end obstruct core functionality. Program was started as something working - thus became popular. Later on to satisfy needs of few others (here ended the egoism!) features added were not needed by authors themselves (nor (as usually!) were properly communicated by end users to developers) thus ended up being implemented poorly.

That's it for today.

[off topic] [rants] GNU Make - missing features

1. For all "declaration" nature of make, it is still very very hard to build your own rule out of predefined ones found in make database. e.g. making a rule for new file extension to call existing rule for older file extension is only possible with dumb copy-and-pasting. Now imaging project with hundreds makefiles - all at different times doing such copy-and-pasting: one hell to do maintenance.
   
2. Arithmetics. I was really surprised to find that make doesn't have any internal facility for arithmetic expression calculation. Even more so I was surprised when I learned that GNU project doesn't even have such library. Best call - is to call external optional program 'dc' or hope that bash is installed and do arithmetics through it. That bars many accessibility features like progress counter for long build processes. Also obviously renders makefiles unportable.
   
3. Clean mechanism to overwrite rules. That's actually what makes proper dependency check implementation impossible. E.g. if you have dependency "a: b" and "a: c" that would really mean "a: b c" - and there is no way to cancel the "a: b" dependency.
4. List and string routines. Again, mere convenience, yet has performance impact. To put it simply: it is very stupid when to transform file name I have to call several external programs. String manipulation is needed for file name transformations. List routines are needed for proper management of list of targets scheduled to build.
5. Recursiveness. The dead horse was beaten many times before. Lack of proper recursiveness support bars use of make on very large projects. Or leads to arcane make files understood by few thus having very very low accessibility. One can't require everybody on project to be a make specialist.

Looking at all the rants, my mind instantly points out that the list is precisely difference between make and cons. Last update to cons - pcons - was in 2001 and (thanks to that fact that it was written in Perl) it still works perfectly in all situations I have been to.

I had very ridiculous accident: people using pcons were told to switch to make ("Of course there is no reasons to it - it is just our corporate policy"). Case looked bleak: dumbing down build system back to stone age of make wasn't problem - problem was the lost functionality (the list above). I have advised (and it actually worked) simple solution: put pcons right along with sources and create makefiles with all targets calling cons instead. Worked beautifully - thanks to slim nature of cons/pcons which is written in Perl and has little/no external dependencies.

[off topic] One-liner to make GNU Info usable

Something like this should be shipped along with GNU Info documentation system - to make it any useful:

$ cat ~/usr/bin/iinfo
#!/bin/bash
info --subnodes --output=- "$@" 2>/dev/null | less
$

Option "--subnodes" tells info to dump everything it has - w/o frigging inhuman document structure. Option "--output=-" tells to dump that all on stdout. The bit "2>/dev/null" is to shut up the stupid useless stuff info spits on stderr. And '| less' bit tells it to behave like proper "man".

Rants. Ironically, even the one-liner cannot make GNU tools any better: they show their age, were not updates in years and do not support most modern things one expects. Searching internet generally brings more and better results than digging outdated GNU documentation. (N.B. GCC of all GNU tools is kind of exception.)

[off topic] message logging severities

I can recall only five levels of severities:

MAJOR: Alarm. Critical unrecoverable error. Results in certain termination of application.
MINOR: Recoverable error. Application can go on, but something is fishy.
NOTICE: Not an error, but shouldn't have happened.
INFO: Run-time information statistics, "anomalies" a.k.a. unsolicited messages. Must produce passable amount of information since might be used for monitoring of real system.
TRACE: Used to trace what's happening in the system: functions called and their arguments. Produces insane amount of output.
DEBUG: Debugging info. Developer's corner.

Production systems run on MINOR level, so that all errors are displayed. MAJOR level normally has abort() built in so that OS would dump core of application for further investigations.

Systems in testing run with INFO level. Often the output is saved and required to match in repeated tests. E.g. if we have fed application with 1000 bytes of info, we would expect INFO message to reflect that 1000 bytes where handled. Not 999, not 1001, not 500 + 500.

Edit1: Added "TRACE" level.

Off-Topic :: Wait-Free/Lock-less synchronization crash course

Well, results are not that good as I have hoped. (In depth info at Wikipedia. Read the first reference on the page written by Maurice Herlihy in 1993.)

1. Rule of thumb: lock-less can only be data structures which synchronize using single datum.
2. Compare-And-Swap( ptr, old, new ) can be used for implementation of atomic counters. The operation is supported by all high performance platforms (e.g. PowerPC & IA-32).
   
3. One cannot make lock-less FIFO for N:M (N readers/M writers) nor for 1:M (1 reader/M writers) configurations. Seems that 1:1 (1 read and 1 writer) only possible configuration and in fact it doesn't require any special support at CPU level. (Reference to rule of thumb: FIFO requires two data items for implementation - head/tail (or next/prev).)
   
4. Stack (LIFO) can be implemented wait-free/lock-less using Compare-And-Swap() op, since it needs only single pointer for implementation - head - pointing to first element to remove/last element inserted.

Conclusion:

1. Unixes were 100% right putting pipes at foundation of I/O: pipe can be implemented as couple of FIFOs and as long as pipe isn't shared between more than 2 applications, it can be completely lock free.
   
2. Stack (along with Compare-And-Swap op) allows to implement memory management (for static block size) very efficiently. (Edit1. In preemptive SMP OS, that might lead to situation when Compare-And-Swap would succeed despite fact that state of queue already changed. Proper implementation for multiple CPUs would use several (at minimum 2) stacks. Allocations are made from one stack, freeing is done on next stack. Allocation function rotates stacks when detects that current stack is empty. Allocation is failed when all stacks were queried at least once and all are empty. Alternatively, one can introduce additional counter: generation counter. That way Compare-And-Swap needs to be able to write double word. When element is allocated, generation counter is incremented. When element is freed, we put on stack pair - pointer and generation counter - which would be with high probability unique.)
   
3. Message passing systems suck big time because (i) 1:1 FIFOs are inapplicable and (ii) it is impossible to optimize memory allocation with lock-less stack since it is impossible to tell where the message would end up. (The later can only be implemented in worst case scenario of shared memory.)