Comparison of MATLAB Versions for CLI and GUI Performance

Banner - Comparison of MATLAB Versions for CLI and GUI Performance

Although the MATLAB core has essentially stayed the same throughout the years, the MATLAB language and runtime has with every release been extended with more functionality as well as featured significant improvements to the interpreter and Just-In-Time (JIT) compilation engine. As developer of the FEATool Multiphysics simulation toolbox which features 10-years of both graphical user interface (GUI) and command line interface (CLI) backwards compatibility with MATLAB (back to R2009b), it would be very useful to be able to advise users which MATLAB version(s) to recommend in different situations for performance and/or other reasons.

With this in mind, and as no official or comprehensive benchmarks of MATLAB versions are available, both GUI and CLI script performance is compared for MATLAB versions R2011b (the subjectively snappiest, most responsive, and currently preferred version in the developers opinion) and R2019a (the latest version at the time of writing). Timings and tests are performed with the FEATool Multiphysics toolbox on a typical laptop with the Microsoft Windows 10 operating system and Intel CPU. The FEATool software is used for benchmarking as the toolbox features a large number of pre-defined and built-in validation test cases and fully automated (GUI) tutorials which can be used to automatically and consistently measure performance.

FEATool Multiphysics MATLAB Simulation Toolbox

MATLAB GUI Performance

In the first step, the GUI model test suite is used for timing runs. This test suite runs about 40 pre-defined multi-physics simulation models (the test suite can be run by starting FEATool with the featool testt command). The examples are completely automated so that all GUI callbacks are automatically and consistently run just as a user typically would do manually. The timing averages from five runs can be seen in the table below.

MATLAB GUI Tutorial Tests

R2019aR2011b
3142 s2486 s

From the results it is clear that MATLAB R2011b runs significantly faster (around 25%) than the latest version R2019a even with the improved JIT optimization as the subsequent MATLAB versions have. Note that the UI layout toolkit used by FEATool has been specifically developed and written to run fast as possible using procedural code. (Note that due to backwards compatibility the FEATool GUI toolkit only uses get(h,'prop') and set(h,'prop',val) calls, and not struct call options introduced with handle graphics hg2. Also these results do not apply to the newer web graphics backend introduced in the last year.)

The Mathworks own GUI Layout Toolbox was also tested in development and at the time shown to have significant overhead and cost due to the object oriented design, and just not performant enough for even moderately complex GUIs such as the ones use by FEATool.

MATLAB CLI Performance

As these tests include everything from running the GUI to CAD geometry and model pre-processing, mesh generation, linear equation solving, plotting and visualization, we can not from the tests above conclusively tell what is slower in newer versions of MATLAB. As FEATool also is fully scriptable (one GUI action maps to an equivalent MATLAB function, and FEATool allows saving the modeling process in both binary and script formats) we can also run the tests without the GUI for comparison. The corresponding averaged results for five runs are shown below (starting FEATool with featool testm runs all the available script models in the FEATool examples directory).

MATLAB CLI Examples Tests

R2019aR2011b
812 s976 s

Without using any GUI functionality the results are now reversed with MATLAB R2019a being somewhat faster. As the FEATool code base is both highly vectorized and optimized, and since Finite Element Analysis (FEA) simulations makes significant use of pre-compiled numerical libraries and sparse linear solvers such as BLAS, LAPACK, Umfpack/SuiteSparse etc. which are not really influenced by JIT improvements, a quite modest 10% speedup is maybe what one can expect from any JIT improvements (instead of a major increase in speed).

Summary

With the automated GUI and CLI simulation tests we have been able to establish that current (R2019a) MATLAB JIT optimizations have resulted in a modest speed increase on average, however older versions of MATLAB are significantly more responsive for using GUI functionality. To FEATool users we can therefore recommend using version 2011b when using the GUI. As FEATool also supports saving and exporting models to standard MATLAB m-script files, subsequent simulations can be run on the command line in a newer version of MATLAB if compute speed is critical. Also if a larger study involving many simulation runs is to be carried out it is recommended to use a xargs batch script to simulations them in parallel.

Originally published at https://www.precisesimulation.com.

Category: news

Tags: benchmark matlab performance

SHARE ON

Twitter LinkedIn Facebook
comments powered by Disqus