Hello everyone,

I am trying to run a robustness check on a weighted index for 11 countries. The framework is composed of 5 dimensions and 14 indicators. So I created nested loops on specific ranges of weights (for dimensions and indicators), and generated 102,400 models (each model has 11 observations so the total is slightly higher than ~1M observations). [ I can attach the .dta file if needed]

What I intend to do is:
  1. Rank countries by Model
    Code:
     egen Rank= rank(MPI), by(Model)
  2. Set the desired matrix size:
    Code:
           //create a matrix of 1472x1472
        set more off
        mata: st_matrix("Ed", J(102400,102400,.))
  3. Compute the Euclidean Distances between pairs of models and store the sum of the difference in the relevant entry of the matrix (102400x102400). To this end, I generate 2 temporary variables for each pair of models, in a nested loop (code committed for simplicity, if more details needed please do refer to the attached .do file).
    Code:
    if (`a'!=`b'){
        gen temp = (Model`a'-Model`b')^2
        sum temp
        mat Ed[`a',`b']=r(sum)^0.5
        drop temp }
  4. Take the sum of rows of the matrix, to get a metric that gives an idea about the distance of that model and the remaining 102399 models, reducing my figures from (102400x102400) to a column of 102400 entries
    Code:
           mata : st_matrix("S_Ed", rowsum(st_matrix("Ed")))
        mat list S_Ed // this is the sum of rows of Ed
  5. The desired conclusion is to select the model that has the minimum figure.
    Code:
           mata : st_matrix("Min_Ed", colmin(st_matrix("S_Ed"))) // get the minimum of Eds
        mat list Min_Ed
This worked perfectly for a smaller matrix size. But when I try to generate a matrix of size 102400x102400, at the beginning of the .do file :
Code:
mata: st_matrix("Ed", J(102400,102400,.))
I am getting an r(3900) error:
Code:
 J(): 3900 unable to allocate real <tmp>[102400,102400]  <istmt>: - function returned error
Please note that I am using Windows 10 on a multi-core processor machine (64-bit operating system, x64 based processor) with 8.00 GB of installed RAM (7.88 GB useable, when I checked).
Also, I am using Stata/MP 15.0 for Windows (64-bit x86-64)


Finally, when I run memory after the error, I get the following:
Memory usage
used allocated
data 175,718,400 268,435,456
strLs 0 0
data & strLs 175,718,400 268,435,456
data & strLs 175,718,400 268,435,456
var. names, %fmts, 2,309 35,021
overhead 3,178,544 3,179,288
Stata matrices 0 0
ado-files 11,451 11,451
stored results 0 0
Mata matrices 1,904 1,904
Mata functions 34,272 34,272
set maxvar usage 5,281,738 5,281,738
other 37,549 37,549
grand total 184,232,883 277,016,679
Any advice on how to overcome this r(3900) error and get going with my code?

Thx!