Matlab vs Octave vs Armadillo

Out of curiosity, I just conducted a toy comparison between Matlab, Octave and Armadillo, to convince myself that it is going to be paid off by translating some Matlab programs to Armadillo C++. Here are the details:
1. Matlab (2010b) running on Intel(TM) core i7-2600 (4 cores, 8 threads), Windows 7, 8 GB RAM;
2. Matlab (2011a) running on AMD(TM) Phenom II (4 cores, 4 threads), Window 7, 6 GB RAM;
3. GNU Octave, version 3.2.4, running on AMD Athlon(tm) 64 X2 Dual Core Processor 5000+, Ubuntu 11.04, 4 GB RAM;
4. Aramdillo C++ 2.3.91, running on AMD Athlon(tm) 64 X2 Dual Core Processor 5000+, Ubuntu 11.04, 4 GB RAM;
For C++, I compiled using “g++ testArmadillo.cpp -o testArmadillo -larmadillo -O2”

The test program on Matlab/Octave is

tic;A=randn(5000);B=randn(5000);C=AB;toc

The test program on Aramdillo C++ is

#include 
#include 

using namespace std;
using namespace arma;

int main(int argc, char** argv)
  {
  wall_clock timer;
  timer.tic();
  mat A = randu(5000,5000);
  mat B = randu(5000,5000);
  mat C;

  //cout << A*trans(B) << endl;
  C=solve(A,B);
  cout<<"Elapsed time is "<

Test results
1: Elapsed time is 11.648498 seconds.
2: Elapsed time is 12.639551 seconds.
3: Elapsed time is 42.7492 seconds.
4: Elapsed time is 40.4321 seconds.

Assuming 4 core CPU speeds up twice as the 2 core CPU, 3 and 4 shall be 21 seconds and 20 seconds, respectively. Does C++ really run faster than Matlab/Octave running on the same machine? I am not convinced yet. Possibly, the benefit of translating Matlab script to C++ is the potential of parallelization of the C++ program so that it can run on multiple workstations or a cluster.

7 thoughts on “Matlab vs Octave vs Armadillo

  1. Pingback: Build Trilinos 10.8.x on Ubuntu 11.04 using cmake | junweblog

  2. You’re not doing a good comparison here. There are at least two problems.

    1. Why are you timing how long it takes to generate random numbers?

    2. Another issue is that Armadillo is in this case acting like a wrapper for the underlying Lapack library — the solve() function calls Lapack. This could be either the original “slow” Lapack (whatever comes with Ubuntu), or a semi-optimised version like ATLAS, if it was installed. Either way, the obtained speed has almost nothing to do with C++, but with the quality of the Lapack implementation. Matlab is very likely to have its own optimised implementation of Lapack. End result is that you’re comparing apples and oranges.

    Armadillo can actually use highly optimised Lapack versions. For example MKL or ACML. This might be a better start.

  3. Thanks Mark for seriously reading my weblog.
    1. it is true that to compare the matrix inversion only, I should exclude the process of generating random numbers. So my result is actually a lump sum comparison. One can argue that Armadillo is slowed down by random number generations. It is easy to check by alternating the code a bit. I can show it in a later poster.
    2. On my ubuntu 11.04 I have ACML before installing Armadillo and giving the path of ACML to Armadillo during the installation. So according to the user guide, Armadillo should be able to call the ACML version of LAPACK (correct me if I am too optimistic here). So Armadillo and matlab both are calling highly optimized Lapack. It is not comparing apples and oranges. The weak point of this comparison is that the testing environment is not consistent. I will post the comparison result of running Matlab and Armadillo on the same machine and only comparing the matrix inversion part. Your comments are very well appreciated.
    3. I am not comparing C++ with Matlab. C++ in general should be faster than Matlab for doing loops. I am comparing the performance of matrix inversion of Armadillo with Matlab/Octave.

  4. Although not pertaining to your core issue of code speed, a lot of us are equally or more interesting in coding speed; to that end, I think your two implementations of the same functionality in Matlab and Armadillo, respectively, speak for themselves.

  5. Why are you using different machines and operative systems to run Matlab and Armadillo? Wath you want to do would be extremelly usefull, but you cannot make a sensitive comparison at all if you use such different hardware. Unless I am missing something in your post.
    I am a very heavy user of Matlab and C++, and have created a bunch of classes to comunicate both of them and combine the benefits of each one of them, and I can recognise this issue is of great interest.
    Best

    Alejo

  6. I’ve finished to reiplement my program to C++ (using Armadillo) from a Matlab-code prototype a few days ago. And Armadillo version runs more than 25 times faster. Also a did some simple tests similar to yours, and I found significant acceleration with Armadillo while working with small (smaller than 50×50) matrices. With bigger matrices (such 1000×1000)I didn’t found any significant difference in speed. I achieved great acceleration in my application cause it operates on small matrices most of time.
    P. S. Armadillo was configured to use Intel MKL or ATLAS. For my tasks there is no difference. OS – Ubuntu 12.04 running on Core 2 Duo processor.

Leave a Reply

Your email address will not be published. Required fields are marked *