Fixed point arithmetic in pure shell

One of the many prerequisite for writing a raytracer in pure shell is to be able to work with floats, which are not supported by the language. There are only two “types”: strings and integers.

Fixing this problem is actually very easy and is named fixed-point arithmetic. It is in use since the down of computing when FPU were not available or when performances and exact precisions were needed.

The concept is very simple: change the internal representation of a number compared to its displayed representation. For example if a scaling factor of 1000 is chosen then the number 42 is internally represented as 42000, 5.2 is 5200, 0.6 is 600, etc…

The scaling factor should be chosen with care as the size of integers in shell can be surprising for anyone coming from more evolved languages. Here is an excerpt from the section § 2.6.4 Arithmetic Expansion of the The Open Group Base Specifications Issue 7:

Only signed long integer arithmetic is required
As an extension, the shell may recognize arithmetic expressions beyond those listed. The shell may use a signed integer type with a rank larger than the rank of signed long.

In practice only yash implements such an extension while also providing true floating point arithmetic. On this last point it is not alone as zsh does it too:

% echo $(( 5.0 / 2.0 ))
2.5

Be aware that overflow behaviours can be quite surprising.

Operations

Converting numbers from the displayed representation (which can be considered as factor=1 in my case) and the internal representation is quite easy: multiplying by $factor is all it takes. In the other direction things can be a bit more complicated but allows to choose the rounding with great precision. In my case I just don’t care about displaying my scaled numbers in any particular way.

Additions and subtractions with scaled numbers don’t change and the only hazard is watching for overflow and underflow.

$ factor=1000
$ a=4120 # -> 4.120
$ b=812  # -> 0.812
$ echo $(( a + b ))
4932     # -> 4.932
$ echo $(( a - b ))
3308     # -> 3.308

Multiplications and divisions are another thing though.

For the former the scaling factor of the product is the square of the scaling factor of both integers, resulting in bigger numbers than expected and possibly leading to overflows. It is thus necessary to divide the product by the initial factor.

$ factor=1000
$ a=4120 # -> 4.120
$ b=812  # -> 0.812
$ echo $(( a * b / factor ))
3345     # -> 3.345

For division the scaling factor if nullified (ie: 4120 / 812 is the same as 4.120 / 0.812) so the quotient should be multiplied by the scaling factor. However rounding happens during this division so the multiplication should be applied before:

$ factor=1000
$ a=4120 # -> 4.120
$ b=812  # -> 0.812
$ echo $(( a * factor / b ))
5073     # -> 5.073

That’s all.

Changelog