Normalized MSE plot comparing the shrinkage M-estimators

EST1-EST4 are available at toolbox ShrinkM and were proposed in the reference [1]. Using this script you are able to reproduce the results for the simulation study of Gaussian data in the aforementioned paper.

In the set-up, the data is generated from a $p$ -variate Gaussian distribution with $p=40$ and the covariance matrix model is AR(1) with correlation parameter equal to $\rho=0.6$ .

Estimators in this study are:

EST1 := RSCM-Ell1 estimator (using Gaussian weight )
EST2 := RHub-Ell1 estimator (using Huber's weight function with )
EST3 := RMVT-Ell1 estimator (using MVT-weight with estimated d.o.f. $\nu$ )
EST4 := RSCM-CV estimator (using cross-validation scheme of [1])
EST5 := RSCM-LW estimator (proposed by Ledoit and Wolf (2004)).

To compute EST5 (Ledoit-Wolf estimator) you need to istall the RegularizedSCM toolbox from: http://users.spa.aalto.fi/esollila/regscm/ or use this direct link: http://users.spa.aalto.fi/esollila/regscm/RegularizedSCM.mltbx

REFERENCE

Esa Ollila, Daniel P. Palomar, and Frederic Pascal, "Shrinking the eigenvalues of M-estimators of covariance matrix", Arxiv, 2020.

Author: Esa Ollila, May 2020, Aalto University.

Initalize
Generate the AR(1) covariance matrix
Start the simulation
Plot of NMSE values
Plot of $\beta$ values

Initalize

clearvars;
p = 40;
rho = 0.6;
q = 0.7; % determines the threshold for Huber's function

NRsamples = 2000;
NRmethods = 5;
nlist = (p+p/2):p/2:7*p
beta_ave = zeros(length(nlist),NRmethods);
NMSE_ave = zeros(length(nlist),NRmethods);

betas = cell(length(nlist),1);
NMSE = cell(length(nlist),1);

nlist =

    60    80   100   120   140   160   180   200   220   240   260   280

Generate the AR(1) covariance matrix

tmp = abs(repmat(1:p,p,1) - repmat(1:p,p,1)');
M = 10*rho.^tmp; %  covariance matrix
V = p*M/trace(M);
Msqrt   = sqrtm(M);
normMsq = trace(M^2);

Start the simulation

for ii = 1:length(nlist)

    n = nlist(ii);
    NMSE{ii} = zeros(NRsamples,NRmethods);
    betas{ii} = zeros(NRsamples,NRmethods);
    rng('default'); % set random seed

    for jj=1:NRsamples

        % data generation
        X0 = randn(n,p);
        X  = X0*Msqrt;

        % EST1: RSCM-Ell1
        [EST1,be1,~,stat1] = RSCM(X);
        NMSE{ii}(jj,1) = norm(EST1 - M,'fro')^2/normMsq;

        % EST2: RHub-Ell1
        [EST2,be2,~,stat2,invC] = RHubM(X,'q',q,'gamma',stat1.gamest);
        NMSE{ii}(jj,2)  = norm(EST2 - M,'fro')^2/normMsq;

        % EST3: RMVT-Ell1
        [EST3,be3,~,stat3] = RMVT(X,'invC',invC,'scaling','gaussian','gamma',stat1.gamest);
        NMSE{ii}(jj,3) = norm(EST3 - M,'fro')^2/normMsq;

        % EST4: RSCM-CV
        [EST4,be4,~,stat4] = RSCM(X,'approach','cv','cvloss','MSE');
        NMSE{ii}(jj,4) = norm(EST4 - M,'fro')^2/normMsq;

        % EST5: RSCM-LW
        [EST5,~,stat5] = regscm(X,'approach','lw','verbose','off');
        NMSE{ii}(jj,5) = norm(EST5 - M,'fro')^2/normMsq;

        betas{ii}(jj,:)= [be1 be2 be3 be4 stat5.beta];

    end
    fprintf('\n');
    beta_ave(ii,:) = mean(betas{ii});
    NMSE_ave(ii,:) = mean(NMSE{ii});

    fprintf('.');

end

.
.
.
.
.
.
.
.
.
.
.
.

fignro=1;
set(0, 'DefaultLineLineWidth', 2);
mycol4=[0.5 0.5 0.5]; % gray
mycol5=[0.142 0.672 0.303];

Plot of NMSE values

figure(fignro);
clf
hold on;
plot(nlist,NMSE_ave(:,3),'rv-','MarkerSize',12); %
plot(nlist,NMSE_ave(:,2),'bo-','MarkerSize',12); % = o
plot(nlist,NMSE_ave(:,1),'kx-','MarkerSize',12); % = x
plot(nlist,NMSE_ave(:,5),'s-','Color',mycol5,'MarkerSize',12);
plot(nlist,NMSE_ave(:,4),'*-','Color',mycol4,'MarkerSize',12);
grid on;
box on;
xlim([nlist(1),nlist(end)]);

set(gca,'FontSize',20,'LineWidth',1.5,'FontName','Helvetica');
xlabel('$n$','Interpreter','Latex','FontSize',24)
yl='$\|\hat \Sigma - \Sigma_0\|_{\mathrm{F}}^2/\| \Sigma_0 \|_{\mathrm{F}}^2$';
ylabel(yl,'Interpreter','Latex','FontSize',24)
grid on; xlim([nlist(1) nlist(end)]);
hleg=legend('RMVT','RHub','RSCM','LW','CV','Orientation','horizontal');
legend boxoff;
set(hleg,'FontSize',14); axis tight;

%  subplot that plots the subfigure
hax = axes('Position',[.5 .43 .37 .37]);
hold on
plot(nlist(3:5),NMSE_ave(3:5,3),'rv-','MarkerSize',12);
plot(nlist(3:5),NMSE_ave(3:5,2),'bo-','MarkerSize',12);
plot(nlist(3:5),NMSE_ave(3:5,1),'kx-','MarkerSize',12);
plot(nlist(3:5),NMSE_ave(3:5,5),'s-','Color',mycol5,'MarkerSize',12);
plot(nlist(3:5),NMSE_ave(3:5,4),'*-','Color',mycol4,'MarkerSize',12);
xlim([119 121]);
ylim([0.1273 0.1305]);
set(gca,'XTick',[119 120 121],'FontSize',12)
set(gca,'YTick',[0.128 0.129 0.130],'FontSize',12)
box on;

Plot of $\beta$ values

figure(fignro+1); clf;
hold on;
plot(nlist,beta_ave(:,3),'rv-','MarkerSize',12);
plot(nlist,beta_ave(:,2),'bo-','MarkerSize',12);
plot(nlist,beta_ave(:,1),'kx-','MarkerSize',12);
plot(nlist,beta_ave(:,5),'s-','Color',mycol5,'MarkerSize',12);
plot(nlist,beta_ave(:,4),'*-','Color',mycol4,'MarkerSize',12);

set(gca,'FontSize',20,'LineWidth',1.5,'FontName','Helvetica');
xlabel('$n$','Interpreter','Latex','FontSize',26)
ylabel('$\beta$','Interpreter','Latex','FontSize',26)
grid on; box on; xlim([nlist(1) nlist(end)]);
hleg=legend('RMVT','RHub','RSCM','LW','RBLW','CV', ...
    'Orientation','vertical','Location','southeast');
set(hleg,'FontSize',18); axis tight;
legend boxoff;

Warning: Ignoring extra legend entries.