An old benckmark

Estimating the performance of a modern computer is a difficult task and given results are often deceiving: not only clock speed matters; the modern CPUs have multiple cores, some sort of parallelism implemented in each core with sophisticated pipelines, many levels of built-in caches and use all sort of tricks to be faster, as: branch prediction, speculative and out-of-order execution etc.. Another key factor to consider is memory access, and sometimes also the network is important.

Performances can be very different for different applications; to evaluate a computer for your usage you have to test with something enought similar to your application.

In the nineties Alex Bielajew proposed a simple benchmark for EGS4, a Monte Carlo program simulating the transport of radiation in matter. This program was very popular in the nineties among the medical physics groups, and some enhanced versions of this program are in use also today, mainly for simulation and design of medical equipments (EGS5, EGSnrc). The original EGS4 is no more used and the Bielajew benchmark has been forgotten; nevertheless I continued to run that benchmark program on every computer I used from the eighties until 2020.

This benchmark program doesn't have a graphical interface, and doesn't use the parallelism of modern multi-core, multi-threads systems, doesn't use the network or a massive amount of data; it is something similar to many of the programs I run; it was an enought good test for me and can give an idea of the evolution of computer power in all these years.

The following picture summarizes the results. The speed values are relative to the time needed to run the benchmark on a popular mini computer of the old times: the Vax-11/780 (about five hours). The VAX-11/780, commercialized by Digital Equipment in 1977, was one of the first mini-computers available, it had a great success, for its affordable price and good performances; it had a 5MHz clock, and some megabytes of RAM.

EGS4 Benchmark results for my computers, from 1985 up to 2020

The first green points on the left are the Microvax minicomputers I used around 1990. I had a Microvax-II in the eighties that had about the same performance of the Vax-11/780 (not shown in this plot).

The IBM RISC workstations of the nineties (red dots) had a bit better performances, but the Dec-alpha workstations (the little blue dots around 1997) where the faster choiche I had in that period.

Intel CPU became faster in the new century with the Pentium, with a computation speed increased by thousands, compared with the mini computer of ten years before.

The blue circles between 2005 and 2010 are some workstation with Intel Xeon CPU. These are not much faster than common PC of the same time; Xeon, intended for servers, with bigger cache and more cores, are more suited for serving many users and programs, than for running this little benchmark, which is more sensitive to clock speed.

The red triangles are Xeon CPU used for the ENEA CRESCO supercomputer. In the old times of the CRAY systems, supercomputers had special CPU built for the specific supercomputer, with the best technologies available, other systems had cheaper and much less powerful CPU. But today most supercomputer are a grid of hundreds or thousands of commercial CPU, with many cores, connected by a hight performance network: to exploit their power you have to separate your computation into independent parts, to be executed on the different CPU at the same time, in parallel. Not all the problems are suited for parallelization and programming a parallel computer is more an art than an exact science: you have to minimize the data transfer between the CPU, accounting for memory access time, the cache sizes of the specific system and, as in CRESCO, you have to consider that CPU are connected by a network shared among many running programs.
For a simple program, which doesn't use parallelization, a simple PC is a better choice, much cheaper, easy to manage and faster... this was not understood by many of my colleagues...

The yearly increase of CPU speed was greater before 2005: the Moore law, (more an advertising slogan than a real law), telling that the speed of CPU was doubling each year, couldn't cope with thermal problem and current leakage, so maximum clock speed settled a bit below 4 GHz, and performance increase for this benchmark flattened; but we still have an increase in recent years due to better CPU design. The green triangles at the top are the PC and the notebook I bought in 2020, when retired.

An interesting point is the pink dot labeled "ARM" at the bottom right: this is my old cellphone, an Huawey P6 II Pro, with a Kirin 620 ARM CPU; it rates 460 in this performance scale; a supercomputer of the eighties could rate between 20 and 50, but had no graphical capabilities, no wifi, no bluethoot and much less memory.

Today everyone has in his pocket a computer with a computing performance ten or twenty times that of the more powerful million-dollar supercomputers of the eighties. Most of this power is used to operate a graphical interface of extreme complexity, which allow us to send photos of our pet with only some clicks.

The following image is a log-scale plot of the same data; the fast increase in performance in the nineties and the nearly linear increase in performance after 2005 is more visible.

EGS4 Benchmark from 1985 up to 2020, log scale.

All the data of the benchmark are shown in the table below, my data are not uniform, different compilers and operating system can give slightly different results, and some details on computers and compilers used has been forgotten and are not reported in the table.

The first column is the computer nickname, then we have the computer or CPU type; the operating system and compiler (Debian, Centos, Suse are all Linux systems); clock speed; year and the performance, relative to the Vax-11/780 (18510 sec., about five hours).