Characterizing the performance of graphics hardware is much harder than
that of standard processors. They are pretty starightforward: one command
in, some processing, one result out. They're not quite as simple anymore,
with caches, out-of-order execution, speculative branching etc., but they
still have relatively short pipelines in the order of 10-20 stages.
Graphics hardware, on the other hand, is much more complex. Pipelines
with hundreds of stages and buffers of different sizes at various
interconnects are commonplace. In addition to this,
manufacturers are much less forthcoming with details about their internal
pipelines due to competitive pressure in the market. There is very little
information about the best way of doing things, and the relative
performance of different ways of achieving the same result (which are
usually are quite a lot).
Therefore every developer needs to treat the graphics hardware as a big
black box that needs exploring by many little benchmarking programs.
GLScry is a framework to simplify
writing of these bechmarking programs.
It uses python as a scripting language that defines the data to test and
runs it through a C++-based optimized testing harness. Results are
output as datafiles and gnuplot scripts to generate graphs like the ones
shown above.
We're currently working on adding a graphical user interface that will
simplify selecting different datasets and comparing results from
different machines to each other.
Download it and give it a try!