Release notes

This table provides quick access to what's new in each version.

FSDA follows typical MATLAB timetable in the sense that there are two releases per year. The first typically in May/June and the second around October/November.

All up to date files are in the git public repository https://github.com/UniprJRC/FSDA

Version (Release) New Features and Changes Version Compatibility Considerations Fixed Bugs and Known Problems Release dates
V 8.7.6.2
2024b 

MODEL SELECTION

New routine univariatems which enables to exclude  variables which are are surely not significant. The step is  preliminary to (robust) variable selection.



ROBUST CENSORED REGRESSION

New set of of routines for censored regression (including transformations). All these routines contain inside file name the word "Cens". The tobit model is a particular case of censored regression. More specifically.
 
New function regressCens for censored regression

New function FSRedaCens to monitor the residuals in censored regression

New function regressCensTra which computes the mle of transformation parameter and signed sqrt likelihood ratio test in the censored regression model

New function FSRfanCens which monitors the signed sqrt root likelihood ratio test

DATASETS

New regression datasets, affairs and Esselunga

TRANSFORMATION

New option bsb in functions normYJ and normBoxCox

DISTRIBUTIONS

New functions tobitcdf, tobitpdf, tobitrnd and tobitinv which compute the cdf, pdf, random numnber geneation and quantiles in the generalized tobit distribution

REGRESSION
New option exactR2 inside simulateLM

GRAPHICS
New option h to fanplotFS in order to send the output figure to subplots

DATASETS
New dataset inttrade

ANALYSIS OF CONTINGENCY TABLES
New indexes added to corrNominal

STATISTICAL UTILITIES
New function grpstatsFS which extends function grpstat and shows the output in much better way.

UTILITIES
New function rows2varsFS which extends function rows2vars of MATLAB to reorient table or timetable.

New function pdfprotect which protects pdf files against printing and copying content and can add a watermark diagonally on all pages of the manuscript.

Oldest version of MATLAB which is supported is R2019b

Arguments blocks are introduced to validate the input

Given that the behaviour of function lgend has changed in 2024b function clikablemultileged which enables to hide/show the groups interactively has been rewritten

In MATLAB 2024b when you access the documentation from an installed version of MATLAB, the documentation now opens in your system web browser. Previously, the documentation opened in the Help browser. In the previous versions of MATLAB we were using a script which auomatically redirected to the Internet documentation of FSDA which is located at http://rosa.unipr.it/FSDA. Now this automatic redirection does not work anymore. October 2024
V 8.7.5.1
release  2024a 

GRAPHICS

Function fanplotFS.m now accepts in input not only the output of FSReda and FSRaddt.  but also the output of FSReda, Sregeda and MMregeda. fanplotFS can be used also to monitor the t-statistics coming from FS or S or MM regression.


New option corres inside resfwdplot. If corres is true a 3 panel plot showing the correlations of adjacent residuals is added to the monitoring residual plot.


New options  colorBackground in spmplot in order to have a background color of each scatter which depends on the value of the correlation coefficient.

New option typespm which enables the user to control how the  lower (upper) part of the scatter plot matrix is shown. It is possible to repalce the scatter with the correlation coefficients given as   "circle", "square" "number". Finally it is also possible to suppress the scatter.

avasmsplot now returns the handle to the big (invisible) axes framing the subaxes of the plot.

New optional arguments in aceplot. Now it is possible to show just selected plots with or without the plot of the transformation for the explanatory variables.

New optional arguments addxline, flabstep and multiPanel in fanplotFS

New optional argument msg in FSRaddt
 
REGRESSION

Functions Sregeda, MMregeda, now also report the values of the t statistics of the regression coefficients. 

UTILITIES STAT

New functions  pivotCoord and pivotCoordInv to transform the data inro compute isometric logratio coordinates and viceversa.
New function logfactorial to compute log(x!) with high precision where x is not necessarily an integer.

DATASETS

New regression datasets, D1, D2, D3, inttrade1, inttrade2, inttrade3, cement, air_pollution, valueadded and nci60, added.

DISTRIBUTION

New functions WNChygepdf, WNChygecdf, WNChygeinv, WNChygernd, FNChygepdf, FNChygecdf, FNChygeinv, FNChygernd for the extended hypergeometric distribution. All the functions which start with WNC refer to the Wallenius non central hypergeometric distribution, while all the functions which start with FNC refer to the Fisher non central hypergeometric distribution.
New functions mWNChygepdf, mWNChygerndmFNChygepdf, mFNChygernd to compute the density and to generate random number from the multivariate Wallenius and multivariate Fisher non central  hypergeometirc distribution. All these functions are a translation from C++ of the routines of Fog (2008), library BiasedUrn.

VOLATILITY

A collection of routines 1) to compute the integrated variance from a diffusion process via the Fourier estimator using Dirichlet or Fejer kernel; 2) to computes the integrated varaince, quarticity, leverage of a diffusion process via the Fourier-Malliavin estimator (routines written by S. Sanfelici and Toscano, G. (2024) .

MULTIVARIATE

Functions for robust correspondence analysis completely redesigned mcdCorAna (mcd in correspondece analysis), FSCorAna (automatic outlier detection based on the Forward Search) and FSCorAnaenv (envelope creation of minimum Mahalanobis distance and inertia explained), FSCorAnaeda (FS in correspondence analysis with exploratory data analysis purposes).

APPS

CorAnaAPP which enables an interactive robust data analysis of the contingency tables


 
Oldest version of MATLAB which is supported is R2019a

Funcion fanplot.m in order to avoid conflicts with function fanplot of the Financial Toolbox has been renamed fanplotFS.m. Old function fanplot.m has been left in the folder but it will be removed in a future release

Function playbackdemoFS.m has been removed

Thanks to MathWorks support now the FSDA tree has a new structure which improves the toolbox's polish, ease of maintenance, and developer/user experience. More specifically, subfolder +aux has been created in order to host the routines which do no have to be called directly by the user. Subfolders private have been added to each folder in order to host routines which are called just by the corresponding parent folder.
  April 2024
V 8.7.2.4
release  2023b 

REGRESSION

New option DataVars inside FSRaddt which enables to select the variables for which t deletion tests have to be computed.

New option DataVars inside aceplot which enables to select the variables for which g(X) again X has to be computed.

New function forecastH which enables to plot the confidence bands for the hetereskedastic regression model.

TRANSFORMATIONS

New option highlight in functions fanplot  which enables to highlight the step of entry of selected units.


GRAPHICS

Now fanplot also accepts as input the output of FSRaddt

Improved labelling inside add2yXplot. Now the labels are left or right aligned depending on their y coordinate.

REGRESSION

In functions Sreg, MMreg and MMregcore, Sregeda, MMregeda, Mscale and Taureg it is possible to specificy in option 'family' 'AS' (that is Andrew's sine link).

STATISTICAL UTILITIES

New function distribspec which extends MATLAB funtion normspec to any distribution.

New Andrew's sine link for robust regression. The new functions which have been added are ASrho, ASpsi, ASpsider, ASpsix, ASbdp, ASeff and ASc

DATASETS

Added dataset about consumer loyalty in retail ConsLoyaltyRet

CLUSTERING

New function txmerge which performs a (hierarchical) merging of the inflated number of components found by tkmeans or tclust.
Oldest version of MATLAB which is supported is R2018b DOCUMENTATION BUG FIXED

If inside Home|Preferences the option "Documentation Location" is set to "Web", the FSDA help which is called is the one inside http://rosa.unipr.it/FSDA.

If inside Home|Preferences the option "Documentation Location" is set to "Installed Locally", the FSDA help which is called is the one installed locally (during the first access to FSDA documentation html files are automatically copied inside docroot folder).
This is for MATLAB versions 2021A-2023A. For previous MATLAB versions the documention which is called is the local one.
April 2023- October 2023
V 8.7.0.0
R 2023a 

REGRESSION

Now function LTSts also accepts missing values in the response

GRAPHICS
New option Varnames in aceplot.
New option showBars in avasms

New function avasmsplot which enables: to display the solutions found using function avasms (model selection using robust avas). Using option databrush it is possible to see after interactive brushing to see the details of each solution.

New options colorBackground and typespm in spmplot.

MULTIVARIATE ANALYSIS

New output in pcaFS: orthogonal distance from the principal component subspace, and score distance. A new plot which shows the orthogonal distance vs the score distance is also autoamtically produced.

 New input in pcaFS: using options bsb or bdp it is possible to have a robust estimate of the covariance matrix. New sliders snd buttons in APP biplotAPP to show how the biplot changes varying breakdown point or subset size.


HTML HELP DOCUMENTATION

The new category "Probability distribution" has been created.
This section contains the FSDA routines for  correlation coefficient, inverse gamma, mixture of non central chi-square, Tweedie, Vervaat  and Wallenius distribution

DATASETS

The description of the datasets (clustering, multivariate or regression) has been added inside the table. Now when a dataset in table format is loaded with DataSetName.Properties.Description it is possibile to see the description of the dataset. For example after:
load geyser.mat
the instruction
geyser.Properties.Description
enables to see the description.

New datasets BigTradeEvents (section clsutering) and clothes33 (section multivariate) and leafpine (section regression)

UTILITIES

Function tabledisp in order to properly display a table or an array from the command line (for example with prespecified number of decimals)

Function crosstab2datamatrix now produces in output also a table.

json function signature definitions added to subfolder clustering which allow contextual help when calling FSDA functions

GUIs
New option inferential in GUIregress.m which shows the detailed calculations to compute t-statistics and confidence intervals.

New option DiscreteData in GUImad.m

New function GUIautocorr which shows the calculations to obtain the autocorrelation.


Oldest version of MATLAB which is supported is R2018a
Independently on the documentation location  inside Home|Preferences  the FSDA help which is called is the one inside http://rosa.unipr.it/FSDA.

 
December 2022-April 2023
V 8.6.3
R 2022b
CLUSTERING DATASETS

New datasets spam,  crabs and USArrest added

MULTIVARIATE DATASETS

New dataset clothes added

REGRESSION  DATASETS

New datasets P12119085full and P17049075full added


REGRESSION

New function  LTStsLSmult to detect multiple level shifts added

New field ARtentout in struct model inside LTSts added in order to replace lagged values with estimated values in presence of an autoregressive component.

.
Oldest version of MATLAB which is supported is R2017b
In order to visualize properly the HTML FSDA help files, it is necessary inside Preferences to select as Documentation Location "Installed Locally" in MATLAB Help Preferences.
December 2022
V 8.6.0
R 2022b
DATASETS

New datasets kidney, melody and wholesale, added to the section clustering datasets. Description added to datasets inside the table format

UTILITIES STAT

New functions Mlocation and Mlocsca added. Mlocation performs M estimate of location in univariate samples. Mlocsca computes simultaneously estimate of location and scale. New function medcouple added. It is used to compute a robust skewness estimator. New functions corrpdf and corrcdf to compute the pdf and cdf of the sample correlation coefficient

REGRESSION CLUSTERING

tclustreg returns in output the value of the objective function for each subset after the concentration steps.
forecastTS improved to cope with autoregressive component. Decomposition of forecasts into the different underlying components when input option plots is equal to 2.
LTStsVarSel new option firstTestLS.

GUIs

New function GUIcorr added to show the calculations to compute the correlation coefficient
All functions (OPTrho, OPTpsi, OPTpsider, OPTwei, OPTpsix, have been rewritten using as input the intervals, [0 c2/3], [(2/3)c c], >c instead of the intervals [0 2c], [2c 3c], >3c

Oldest version of MATLAB which is supported is R2017b

In order to visualize properly the HTML FSDA help files, it is necessary inside Preferences to select as Documentation Location "Installed Locally" in MATLAB Help Preferences.
November 2022
V >8.5.26
R 2022b
DATASETS

New datasets kidney, melody and wholesale, added to the section clustering datasets.

UTILITIES STAT

New functions Mlocation and Mlocsca added.
Mlocation performs M estimate of location in univariate samples. Mlocsca computes simultaneously estimate of location and scale.

REGRESSION CLUSTERING

tclustreg returns in output the value of the objective function for each subset after the concentration steps.
All functions (OPTrho, OPTpsi, OPTpsider, OPTwei, OPTpsix, have been rewritten using as input the intervals, [0 c2/3], [(2/3)c c], >c instead of the intervals [0 2c], [2c 3c], >3c

In order to visualize properly the HTML FSDA help files, it is necessary inside Preferences to select as Documentation Location "Installed Locally" in MATLAB Help Preferences.
October 2022
V 8.5.26
R 2022a

GRAPHICS

New option PlotGroup in scatterboxplot.m

Improved mdpattern with syncronized axes.

New options plots.ColorMapLabelRows and plots.ColorMapLabelRows  in CorAnaplot in order to have a colorbar or row or column labels depending on a particular variable such as the corresponding mass or the communaility.

New function add2boxplot which enables to add outlier labels to an existing figure containing one or more boxplot

 

REGRESSION

Added options rob, tyinitial, trapezoid, orderR2 and scail to avas.

New function avasms, to automatically choose the best AVAS model. Added a series of options to avasms. A new graphical display to show the importance of the different options is shown in the output.

DATASETS

New datasets fish and Marketing_Data added to regression datasets.

UTILITIES STAT

New option standardize in function Powertra

New functions quickselectFS_demo and quickselectFSw_demo which illustrate graphically with an animation the  functioning of quickselectFS and quickselectFSw.

 

UTILITIES HELP

Updated and improved function CreateFSDAtoolboxFile which automatizes all steps needed to create programmatically a toolbox (mltbx) file and github release

New function setToolboxStartEnd, which can set release compatibility in ToolboxPackagingConfiguration.prj file.

 

CLUSTERING

Improved version of ctlcurves with ordered solutions based on LR test

New options  startv1true1unitCentroid and priorSol added to tclust

 

GRAPHICS

New options oneplot added to aceplot.m

 

In order to visualize properly the HTML FSDA help files, it is necessary inside Preferences to select as Documentation Location "Installed Locally" in MATLAB Help Preferences.
April 2022
V8.5.10
R 2021b 

ROBUST CLUSTERING

Likelihood ratio test inside ctlcurves.m and extended output which automatically detects a set of candidate solutions.

 

GRAPHICS

New function ctlcurvesplot.m to plot the output of the trimmed likelihood curves

New function moonplot to show the output of correspondence analysis

New function balloonplot to visualize large categorical data

DATASETS

New datasets oliveoil,  wine and flea added to cluster analysis section.

New datasets csdPerceptions and mobilephone added to the multivariate analysis section

MULTIVARIATE ANALYSIS

New function mcdeda which monitors the output of mcd for a sequence of values of breakdown point

New option conflimMethodCramerV in  function corrNominal to compute the confidence interval of Cramer V coefficient.

New function mdpattern which finds and plots missing data patterns.

REGRESSION

New options inside AVAS to make it robust to the presence of outliers

Sreg and MMreg with hyperbolic tangent estimator made faster.

New options inside LTSts and simulateTS that allow a customized definition of the autoregressive component

 

UTILITIES

Added weights in repDupValWithMean.m

New function setdiffFS in case input arguments just contain positive integer values.

In a lot of functions the instruction coder.target('MATLAB') appears in order to cope with the MATLAB C coder requirements.

We stop supporting releases before 2016b.

Functions balloonplot and mdpattern use bubblechart routine and therefore only work with MATLAB 2020b or greater.



October 2021
V8.5.4 R2021a

MULTIVARIATE ANALYSIS

New function pcaFS which enables to visualize both the percentage of variable explained by the different components, both the factor loadings and interactive dynmic biplot. New function biplotFS which enables to call the app biplotAPP which creates an interactive dynamic biplot.

 

Improvements in functions spmplot (now input can also be a table) and CorAna (now input can also be a table when optional option datamatrix is true)

CORRESPONDENCE ANALYSIS

A new suite of functions for robust correspondence analysis. Functions mcdCorAna, FSCorAnaeda, FSCorAnaenvmmd and mahalCorAna, extend functions mcdFSMeda, FSMenvmmd and mahalFS to the context of contingency tables. The plots produced by functions malindexplot and malfwdplot have been modified to incorporate the output of these new functions.

 

GRAPHICS

New function barVariableWidth which  produces a bar plot with different widths and colors for each bar

In functions mmdplot, mdrplot, mmdrsplot and mdrrsplot added the possibility of showing in output arguments the list of units brushed in each brushing operation

Option tag added to FSR, FSRmdr, FSM, FSMmmd, FSRts, FSRB, FSRr and FSRH

Option label added to malindexplot

 

ROBUST CLUSTERING

newoption warmup in tclustICgpcm

new option commonslope in tclustreg and tclustregeda and tclustregIC

 

DATASETS

New dataset citiesItaly added to multivariate analysis section.

 

REGRESSION

New options for VIOM and MSOM contamination inside function simulateLM

 

STATISTICAL UTILITIES

Added a series of routines for the estimation of integrated and instantaneous variance of a diffusion process via Fourier analysis [M.E. Mancino, M.C. Recchioni, S. Sanfelici, ``Fourier-Malliavin Volatility Estimation. Theory and Practice'', 2017, Springer - New York]

 FE_int_vol,  FE_int_vol_Fejer, FE_splot_vol, FE_spot_vol_FFT,  CEVmodel, OptimalCuttingFrequency

 

GUIs

A new set of functions which enable to monitor  all the necessary steps to be done to obtain the main indexes of descriptve statistics. More in detail:

GUIconcentration, GUImad.html, GUIpowermean.html, GUIquantile.html, GUIskewness.html, GUIstd.html, GUItrimmean.html, GUIvar.html, GUIcov.html, GUIregress.html

 

Routines pcaFS  and biplotFS assume that at least MATLAB 2019b is installed.

We stop supporting releases before 2016a.


Starting from MATLAB 2021a in order to view the FSDA html documentation it is necessary
on the Home tab, in the Environment section, click Preferences. Select MATLAB > Help and change the Documentation Location.
In this case select 

 option "installed locally".
March 2021
V8.3.0 (R2020b)

TRANSFORMATIONS IN REGRESSION

We have enriched the properties of the data transformations of the Yeo and Johnson (2000) for negative and positive responses, which we introduced in R2020a. More specifically, we intervened on the smoothness condition that the second derivative of zYJ(lambda) with respect to y be smooth at y = 0, along Atkinson et al (2019) and (2020), to allow two values of the transformations parameter: lambdaN for negative observations and lambdaP for non-negative ones. Now, function ScoreYJall computes:

1) a global t test associated with the constructed variable for lambda=lambdaP=lambdaN.

2) a t test for positive observations.

3) a t test for negative observations.

4) a F test for the joint presence of the two constructed variables described in points 2) and 3.

5) the F test based on the maximum liklihood estimate of lambdaP and lambdaN

New function ScoreYJall which computes the score tests described in points 1)-5) above.

New function scoreYJmle  which computes, in the case of extended Yeo and Johnson transformation, the likelihood ratio test of H0: lambdaP=lambdaP0 and lambdaN=lambdaNeg0.

Added option usefmin in funtion  boxcoxR. This option uses the solver (fminsearch or fminunc) to find MLE of the two transformation parameters for extended Yeo and Johnson family (Atkinson et al. 2020).

New function fanBIC which takes in input the output of FSRfan and using BIC and smoothness index enables to automatically choose in an efficient and robust way, the best value of the transformation parameter.

New function fanBICpn which enables to automatically choose the best values of the transformation parameters for positive and negative observations

New function normYJpn which extends the companion functions normBoxCox and normYJ to the case of extended Yeo and Johnson transformation.

TIME SERIES

New function SETARX which implements Threshold autoregressive models with two regimes

ROBUST CLUSTERING

New tools for dealing with the 14 Gaussian parsimonious clustering models (GPCM).

In function genSigmaGPCM new option pa.exactrestriction has been added. If pa.exactrestriction is true the covariance matrices are  generated with the exact values of the restrictions specified in pa.cdet, pa.shw and pa.swb. In function MixSim optional input structure sph now can be called with field  sph.exactrestriction

In function tclust the fourth input restrfactor can be a structure which can contain the type of Gaussian Parsimonious Clustering Model - GPCM (restrfactor.pars), the
scalars in the interval [1 Inf) which specifies the
the restriction which have to be applied to the determinants (restrfactor.cdet), to the elements of the shape matrices inside each group (restrfactor.shw) and across groups (restrfactor.shb).

New functions tclustICgpcm, tclustICsolGPCM, tclustICplotGPCM, and  carbikeplotGPCM which extend functions tclustIC, tclustICsol, tclustICplot and carbikeplot to the case of the 14 GPCM.

GRAPHICS

New plots waterfallchart (which implements the waterfall chart (see https://en.wikipedia.org/wiki/Waterfall_chart) and new function funnelchart which implements the funnel chart (see
https://en.wikipedia.org/wiki/Funnel_chart)

new function scatterboxplot (which creates scatter diagram with marginal boxplots).

Improvements to functions

Now spmplot accepts as input a table. In the case the names of the tables are automatically added at the margins. Similarly, in function corrNominal when option datamatrix is true it is possible to supply as first argument a table.

 

 

DATASETS

New datasets balancesheets and facemasks added in the datasets regression section and datasets clustering section respectively

We stop supporting releases before 2015b.


Functions FSRfan, ScoreYJ and ScoreYJall made faster
Mathworks search engine in our HTML web pages in version 2020b works again.

Routines fanBICpn, tclustICgpcm and tclustICplot gpcm use routine heatmap introduced in MATLAB  2017a.

Routine tclustregeda to display the results uses function parallelplot introduced in Matlab 2019a.


October 18th  2020
V8.0 (R2020a)

 

ROBUST REGRESSION

New set of routines for minimum density power divergence estimators (mdpd, mdpdR, mdpdReda, PDrho, PDpsi, PDwei, PDpsider, PDpsix, PDbdp, PDeff, PDc ), discussed in https://www.mdpi.com/1099-4300/22/4/399)

New function simulateLM to simulate linear regression data with prespecified value of R2, prespecified correlation among the explanatory variables and type of distribution.

New function VIOM which computes weights estimates under a Variance-Inflation Outlier Model using MLE or Restricted MLE (REMLE).

ROBUST CLUSTERING

New function tclustregeda  which enables to monitor the regression clustering classification for different levels of trimming.

Improved function tclustregIC which enables to compute the BIC (and other information criteria) for different values of restriction factors and different number of groups, for classification or mixture likelihood and regression clustering.

Modified function tclustICsol now accepts input from tclustregIC to show the yXplot of the best solutions.

New function ctlcurves to select the appropriate number of groups in robust clustering.

New function mdrrsplot which plots the random starts trajectories and enables to brush them. The companion function mmdrsplot referred to multivariate analysis has been improved.

 

TRANSFORMATIONS IN REGRESSION

New function boxcoxR which computes the profile log Likelihood for a range of values of the transforamtion parameter (lambda) and computes the MLE of lambda in the
supplied range. Supported families Box Cox, Yeo and Johnson and extended Yeo and Johnson (Atkinson et al. 2020).

DOCUMENTATION

Improved menu for the automatic installation of the FSDA html help files.

UTILITIES

New function exactcdf for finding the exact cdf of each element in a vector x with respect to the empirical distribution, represented by another vector.

New functions twdpdf and twdrnd to compute the pdf of the Tweedie distribution and generate random numbers from it.

 

We stop supporting releases before 2015a.

Functions PoolPrepare.m and PoolClose.m removed (not necessary with the latest releases of parallel computing toolbox).

Cleaner code for tclust and tclultreg functions now available; the core part which computes the EM algorithm for each subset which is extracted, is now in tclustregcore.m and tcluscore.m; these two functions are internal and therefore undocumented.

From this release we support older releases up to 2015a (however about 85% of the FSDA still runs with older versions of MATLAB up to 2012a).
Mathworks search engine in our HTML web pages in version 2020a does not work anylonger.
To search for FSDA function it is necessary to use the supplemental software search engine.

Corrected small bug in LTSts in case of level shift with very small series.

April  2020
V7.1 (R2019b)

This is the first release which is distributed from Mathworks marketplace and from github platform.

TRANSFORMATION IN REGRESSION

New function tBothSides  which enables to transform both sides of a (nonlinear) regression model.

New function boxcoxR which finds MLE of lambda in linear regression (and confidence interval) using Box Cox or Yeo and Johnson family.

ROBUST TIME SERIES ANALYSIS

New functions  LTStsVarSel.m which enables to perform variable selection in the robust time series model LTSts.m. In functions LTSts.m, simulateTS.m and forecastTS.m it is now possible to add an autoreressive component.

UTILITIES

New function existFS which checks whether a file exists and puts the answer in a cached persistent variable

DOCUMENTATION

Added file getting_started.mlx in subfolder doc of the main root of FSDA for packaging the FSDA toolbox,

 

From this release we support old releases up to 2014b. Now functions tclust.m and logmvnpdfFS.m do not need anymore the presence of MEX files. This modification has been made necessary because at present Mathworks toolbox packaging does not support mex files.
Fixed a bug in tclutICplot
New function startup.m has been added to the main folder of FSDA which copies all HTML documentation files in subfolder (FSDA root)/helpfiles/FSDA into (MATLAB docroot/help). The files can be copied in Windows system just if the user has administrator privileges.

October  2019
V7.0 (R2019a)

CLUSTER ANALYSIS

Function tclustreg has been considerably enhanced. Now the function includes: (i) robust BIC, (ii) possibility of constraining the determinants of the covariance matrices of the explanatory variables, (iii) options for treating datasets with concentrated noise, making use of concentration steps appropriately modified using observation weighting and thinning methods.

New function tclustregIC which (if present) uses the Parallel Computing toolbox to compute robust BIC  for mixture and classification likeilhood for different values of k (number of groups) and different values of c (restriction factor for the variances of the residuals), for a prespecified level of trimming.

New function  for constraining the determinants  restrdeter. This function has its own interest but is called in every  concentration step of function tclust in case determinant restriction is needed.

Routines for constraining the determinants (restrdeterGPCM), the shape matrices (restrshapeGPCM) and to impose common rotation matrices (common principal components) in presense of equal shape (cpcE.m) or varying shape (cpcV.m) and a general routine to impose constraints in the family of the 14 Gaussian Parsimonious Clustering Models (restrSigmaGPCM).

Routine to generate data based on the 14 Gaussian Parsimonious Clustering Models (genSigmaGPCM). This routine can be called directly from function MixSim in order to generate each of the 14 Gaussian Parsimonious Clustering Models with a prespecified level of overlap (see option sph inside MixSim).

Routine GowerIndex to compute matrix of similarity indexes using Gower metric.

DATASETS

New datasets added to the collection:

animals, P12119085, P17049075, fondi_large, JohnDraper data, gasoline data, ms212. See pages datasets_reg and datasets_mult for a description of these datasets.

GRAPHICS

Possibility of brushing using rownames. Rownames also appear in the associated scatter plot matrix, both for regression and multivariate analysis: se new examples in resfwdplot and malfwdplot.

New function aceplot to visualize the results of the output produced by functions ace and avas.

Option RowNamesLabels has been added to add2spm and to add2yX to label the units.

MULTIVARITATE

Funciton FSMeda is now much faster; the original function FSMeda has been kept, renamed FSMedaeasy, because the algorithm is much easier to follow.

REGRESSION

New functions: (i) ace which implements the alternating conditional expectations algorithm to find the transformations of y and X that maximise the proportion of variation in y explained by X and (ii) avas which uses a (nonparametric) variance-stabilizing transformation for the response variable.

New function smothr to smooth values imposing variour constraints (e.g. monotonicity, circularity,..). This function calls the supersmoother routine of Friedman.

New function rlssmo to compute a running line smoother with global cross validation.

New function supsmu to smooth scatterplots using Friedman's supersmoother algorithm.

Function RobCov now includes the estimator covrobc (a corrected version of the covariance matrix of  robust beta coefficients). A new motivating example  shows a case why covrobc should be always used.

UTILITIES

New function repDupValWithMean that enable to replace values of y including non unique elements in vector x with local means.

UTILITIES HELP

Function publishFS is fourthly improved. This function automatically transforms structuerd .m files into MATLAB pure style files. In the HTML help files now the right click of the mouse (similarly to pure Mathworks pages) enables to execute, select or find help (F1 key) for all the versions of MATLAB starting from 2017a.

 

STATISTICAL UTITLIES

New function genr8 to generate random numbers which are coherent across different software platforms.

New function exactcdf to find exact cdf values of each element of an input vector x with respect to an empirical distribution.

New function wthin which thins a uni/bi-dimensional dataset.

New function ctsub which computes numerical integrarion from x(1) to z(i) of y=f(x)

New functions (i) vervaatsim (to simulate precisely from a Vervaat perpetuity
distribution)  (ii) vervaarxdf (to obtain the pdf or the cdf of a Vervaat perpetuity distribution) and (iii) vervaatrnd (to simulate random variates from the Vervaat perpetuity distribution).

 

 

This is the last release where we  we support old releases up to 2012a.
Fixed a bug in mtR when the user wanted to continue the simulation using a negative seed

May  2019
V6.1 (R2018b)

(1) New function qqplotFS that enables to create a qqplot of residuals with confidence bands

(2) New function  mtR which generates the same random numbers produced by R software with Mersenne Twister mt19937ar

(3) New functions associated with Rocke biweght estimator. See for example RKrho, RKpsi, RKpsider, RKwei, RKbdp, RKeff.

(4) Routines FSR, FSRmdr, FSRbsb extended to time series (see new functions FSRts, FSRtsmdr, FSRtsbsb and regressts)

(5) New function verlessthanFS.  It is a faster version of MATLAB function verlessthan.

(6) New datasets added to the collection.

(7) New routine publishBibliography to create in a automatic way the bibliography from the citations present inside the .m files.


Fixed a problem in the brushing from spmplot and on the diagonal there are the boxplots.  See for more details the additional examples in yXplot and spmplot.

Improved option for thinning units inside tclustreg

Solved minor bug in FSM when it was called with option 'bonflev'
September  2018
V6.0 (R2018a)

(1) New function tclusteda that helps choosing the best tclust model. It computes tclust for different values of the trimming factor and produces plots that allow to find the optimal level of trimming. This function uses the parallel processing toolbox, if available.

(2) Extension of the score test. New function ScoreYJpn  that computes the score test for Yeo Johnson transformation separately for positive and negative observations. FSRfan now accepts the new option family "YJpn" and it is possible to monitor the score test for both positive and negative observations (output arguments out.Scorep and out.Scoren).

(4) New functions for time series analysis. simulateTS simulates a time series with trend (up to third order), seasonality (constant or of varying amplitude) with a different number of  harmonics and a level shift. forecastTS produces forecasts with confidence bands for a time series estimated with function LTSts.  

(4) CorAna has an improved display of results. New function CorAnaplot draws a rich Correspondence Analysis graph with different types of confidence ellipses for selected rows and columns.

(5) New function verlessthanFS.  It is a faster version of MATLAB function verlessthan.

(6) Documentation of yXplot considerably improved. New options added (xlimx, ylimy, namey, nameX).

(7) MixSimreg extended to account for multiple parameter distribution (betadistrib option)

(8) histFS has a new optional argument (weights) for plotting a weighted histogram.

(9) options labenv has been added to mmdrsplot.

(10) option axesellipse added to ellipse

(11) New output argument idxMapping used in function ClusterRelabel, to track the indexes permutations used to rearch a desired cluster labelling.

From release 2018a we support old releases up to 2012a.

From 2018a the use of subfunctions tinvFS, finvFS, tcdfFS, fpdfFS, fcdfFS which compute inverse, pdf, cdf of the T and F distribution are not supported anymore.

Following the feedback provided by our users, Function UnitsSameCluster (which was introduced in R2017a) has been renamed (for better readability) ClusterRelabel
Fixed a series of problems associated to MATLAB 2018a. For example now empty variables which will contain numbers have been initialized with [], while empty variables which contain characters are initialized with ' '. See for example functions yXplot and spmplot. May  2018
V5.1 (R2017b)

FSDA has introduced two new categories of tools, one for (robust) time series analysis; another for analyzing categorical data and contingency tables. More precisely:

 

(1) Function LTSts  extends LTS estimator to time series. A related new graphical plot associated to a time series, wedgeplot, provides information on the presence of outliers and level shifts.

(2) CorAna performs correspondence analysis; SparseTableTest computes independence test for large and sparse contingency tables; CressieRead computes the power divergence family of tests, to check the discrepancy/distance between observed and expected frequencies in a contingency table; rcontFS generates a random two-way table with given marginal totals; barnardtest computes the Barnard test,  corrNominal measures strength of association between two unordered (nominal) categorical variables. Similarly for ordinal data with corrOrdinal. crosstab2datamatrix recreates the original data matrix X from contingency table N. This group of functions is complemented by file examples_categorical.m as in style of FSDA.

 

The two categories of functions will be progressively enriched.

 

Other new functions which are included are boxtest (test of equality of covariance matrices used for example in tkmeans), GYfilt (Gervini and Yohai, univariate outlier identifier), mmdrsplot (interactive plot of the trajectories of minimum Mahalanobis distances from different starting points), overlapmap to plot the ordered pairwise overlap values between components, dempk to perform a merging of components found by tkmeans, ncpci to compute a non centrality parameter confidence interval. 

Finally, spmplot has been enriched to superimpose ellipses, density and contour functions to data and extract single panels from the scatter matrix.

  Various inconsistencies in the browsing of the documentation pages and the bibliography have been fixed. November 2017
V5.0 (R2017a) CLUSTERING
Function tclustreg now includes trimmed Cluster Weighted Restricted Models.

New function tclustIC for the automatic selection of the best number of groups.

New function tlclustICsol to extract a set of relevant solutions (and associated tclustICplot).

New function UnitsSameCluster to to control the labels of the clusters which contain predefined units.

New function to compare two partitions (Fowlkes and Mallows index)

Updated routine simdatasetreg to generate new outlier patterns

STATISTICAL UTILITIES

New routines for density estimation and thinning, for univariate and bivariate data (used in tclustreg).

bwe, rthin, wthin, WNChygepdf

UTILITIES
New functions wraptextFS, removeextraspacesLF

MONITORING ROBUST ESTIMATORS
New functions mveeda, MMmulteda
Smulteda, Sregeda

TRANSFORMATIONS IN REGRESSION
 New function ScoreYJ which implements the score test for the Yeo and Johnson transformation. This new transformation has also been embedded inside function FSRfan.

SAMPLING AND COMBINATORIAL updated function randsampleFS and subsets.


NEW mlx files
.mlx files introduced for examples_MixSim

GRAPHICS
New function to create the car-bike plot to find the most relevant solutions.

Functions resfwdplot, malfwdplot generalized in order to take as input the output of procedures which monitor robust estimators.

  Control version added in many graphical routines to take account the modifications in MATLAB 2017a

From MATLAB R2012a callback functions started to use new internal data structures. Interactive plots that use such callbacks (e.g. those in GUI brushRES) may produce errors in releases older than R2012a. The issue should be now fixed. Please report to us any problem you may experience.

May 2017
V4.1 (R2016b) New functions for  bivariate density estimation and random thinning (kdebiv.m, rthin.m) used to extend tclustreg.m features.

The FSDA help folder now contains XML files associated to the functions documentation. This is in view of generating/updating automatically or using a GUI the functions documentation, in html as well as in the function head.

New html documentation generated with publishFS.
We are monitoring  possible compatibility issues that may emerge from  changes in the tagging policy of the graphical objects in MATLAB.
add2spm modified to take into account a change in the property name of the legend object introduced in MATLAB R2016b
(LegendPeerHandle is now called LayoutPeers).

A change in MATLAB R2016b  function legend.m was affecting FSDA function add2yX (the legend was plotted twice). Bug fixed.

Bug affecting FSMeda only in the univariate case fixed.
October 2016
V4.0 (R2016a) Major release.

New function, publishFS, introduced to generate documentation pages directly from the .m files.

New function, makecontentsfileFS, introduced to create a the list of files present in a FSDA folder and/or subfolders. It extends MATLAB function makecontentfilesFS.

.mlx files introduced for examples_multivariate and examples_regression.

New features added to the tclust function, including determinant restriction and new adjusted BIC criterion for the estimation of the number of groups.

Added functions for reweighting FSR and FSRB (FSRr and FSRBr).

Functions FSR, FSRB and FSRH redesigned; a routing implementing the core of the Forward Search algorithm (FSRcore) introduced to avoid code redundancies.

New function, winsor, to winsor data.

New function FSMbsb, which will replace FSMbbm.

New function randindexFS, to evaluate the quality of different clusterings.

New routines poolClose and poolPrepare introduced to conveniently open and close a pool of parallel workers.

Several new robust functions to generate, for example, the Tukey Biweigh rho function (HUrho),  the tuning constant associated to a certain efficiency (HUeff), the psi functions (HUpsi), its derivative (HUpsider), etc. For a full list, see functions under utilities_stats folder.


Documentation made compatible with the new MATLAB help navigation and browser-style, which was redesigned in release R2015b.


Many small bugs fixed. Functions affected include tclust, tclustReg, position, MixSimReg.

ClickableMultiLegend made compatible with R2016a.

drawnow command introduced in several graphical functions, to solve episodic bugs in the generation of the plots.

May 2016

V3.1 (R2015b)

Major release.

Function simdataset.m modified to allow the user to simulate outliers from different distributions and contamination schemes and/or contaminate existing datasets.

New Bayesian regression analysis routines: FSRB.m, FSRBeda.m, FSRBmdr.m, regressB.m.

In FSReda.m: monitoring of confidence intervals of beta and sigma2.

In FSRBeda.m: monitoring of HPD (highest posterior density regions) of beta and sigma2.

New functions for inverse gamma computation: inversegampdf.m, inversegamcdf.m, inversegaminv.m.

Added functions to monitor units forming subset in heterosckedastic and Bayesian regression: FSRHbsb.m, FSRBbsb.m.

Added new datasets for Bayesian examples.

FSMtra.m: added option for the robust transformation in the Yeo-Johnson family.

addFSDA2path.m: modified for compatibility with unix platforms and to address changes in the folder organization of FSDA functions.

Added routines to compute and visualize robust bivariate boxplot (function boxplotb.m)

New routine for automatic outlier detection in heteroskdastic regression (FSRH.m)


In function spmplot, the multiple histograms (boxplots) on the main diagonal of the scatter plot matrix is now working also with R2015a and R2015b.

The html documentation pages of some new functions are missing. As a consequence, some links may be broken. New complete documentation, produced automatically from the documentation in the head of our m-files, will be released soon.

September 2015

V3.0 (R2015a)

Major release.

Added robust regression (tau) and multivariate estimators (Stahel-Donoho). Functions taureg.m and SDest.m. New weight functions (hyperbolic, Hampel and optimal).
New routine for robust Bayesian regression (i.e. FSRBmdr).
New routine for robust heteroskedastic regression (i.e. FSRHmdr).
Added cluster analysis routines (tclust, tkmeans, MixSim), with related example datasets. New exploratory tools for fininding the number of groups based on random start monitoring of minimum deletion residual (fsrmdrRS.m, first FSDA function which exploits the parallel processing toolbox, if present).
New routines for connecting MATLAB with R.
New function to obtain proper threshold for robust estimators in order to have an empirical size equal to the nominal (function RobRegrSize.m).
Added univariate robust scale estimators Qn and Sn (functions Qn.m and Sn.m).
New utilities to extract elements above diagonal in a vector (function triu2vec.m) and to position figures (function upperfracpos.m).
New function to compute the cdf of the linear combination of non central chi-squared random variables (ncx2mixtcdf).

Documentation made compatible with the new help system introduced with MATLAB R2015a

In function spmplot, the multiple histograms (boxplots) on the main diagonal of the scatter plot matrix do not work with R2015a because of recent changes in gplotmatrix.

February 2015

 

V2.1 (R2013a)

New very fast implementation of the Forward Search for multivariate analysis (FSMmmd). Forward Search in regression modified to deal with the cases in which in a particular step of the search subset is not full rank (FSR and FSRmdr). New scatter plot matrix with multiple groups and multiple boxplots on the main diagonal (spmplot).

Structure modified for compatibility with MATLAB R2012b to R2013a releases. Three APPs introduced in release R2013a.


-

May 2013

 

V2.0 (R2011b)

Major release. Traditional robust estimators added including S, MM, MCD, MVE, (Sreg, Smult, MMreg, MMmult, mcd, mve) and univariate and bivariate analysis (unibiv). New interactive plots and interactive graphics features.

Redesigned the definition of many optional input parameters in terms of structures


-

September 2011

 

V1.1 (R2010b)

Multivariate data analysis routines have been added. Function resfwdplot considerably improved.


-


-

September 2010

 

V1.0 (R2010a)

-

A few functions use ~ to denote unused output parameters as from release 2009b


-

February 2010