This table provides quick access to what's new in each version.
FSDA follows typical MATLAB timetable in the sense that there are two releases per year. The first typically in May/June and the second around October/November.
All up to date files are in the git public repository https://github.com/UniprJRC/FSDA
|Version (Release)||New Features and Changes||Version Compatibility Considerations||Fixed Bugs and Known Problems||Release dates|
|V8.5.8 (prerelease 2021b)||
Likelihood ratio test inside ctlcurves.m and extended output which automatically detects a set of candidate solutions.
New function ctlcurvesplot.m to plot the output of the trimmed likelihood curves
New function moonplot to show the output of correspondence analysis
New function balloonplot to visualize large categorical data
New datasets oliveoil, wine and flea added to cluster analysis section.
New datasets csdPerceptions and mobilephone added to the multivariate analysis section
New function mcdeda which monitors the output of mcd for a sequence of values of breakdown point
New option conflimMethodCramerV in function corrNominal to compute the confidence interval of Cramer V coefficient.
New options inside AVAS to make it robust to the presence of outliers
Sreg and MMreg with hyperbolic tangent estimator made faster.
Added weights in repDupValWithMean.m
New function setdiffFS in case input arguments just contain positive integer values.
lot of functions the instruction coder.target('MATLAB') appears in
order to cope with the MATLAB C coder requirements.
We stop supporting releases before 2016b.
||The offical expected release date is October 2021|
New function pcaFS which enables to visualize both the percentage of variable explained by the different components, both the factor loadings and interactive dynmic biplot. New function biplotFS which enables to call the app biplotAPP which creates an interactive dynamic biplot.
A new suite of functions for robust correspondence analysis. Functions mcdCorAna, FSCorAnaeda, FSCorAnaenvmmd and mahalCorAna, extend functions mcd, FSMeda, FSMenvmmd and mahalFS to the context of contingency tables. The plots produced by functions malindexplot and malfwdplot have been modified to incorporate the output of these new functions.
New function barVariableWidth which produces a bar plot with different widths and colors for each bar
Option tag added to FSR, FSRmdr, FSM, FSMmmd, FSRts, FSRB, FSRr and FSRH
Option label added to malindexplot
newoption warmup in tclustICgpcm
New dataset citiesItaly added to multivariate analysis section.
New options for VIOM and MSOM contamination inside function simulateLM
Added a series of routines for the estimation of integrated and instantaneous variance of a diffusion process via Fourier analysis [M.E. Mancino, M.C. Recchioni, S. Sanfelici, ``Fourier-Malliavin Volatility Estimation. Theory and Practice'', 2017, Springer - New York]
A new set of functions which enable to monitor all the necessary steps to be done to obtain the main indexes of descriptve statistics. More in detail:
and biplotFS assume
that at least MATLAB 2019b is installed.
We stop supporting releases before 2016a.
Starting from MATLAB 2021a in order to view the FSDA html documentation it is necessary
on the Home tab, in the Environment section, click Preferences. Select MATLAB > Help and change the Documentation Location.
In this case select
option "installed locally".
TRANSFORMATIONS IN REGRESSION
We have enriched the properties of the data transformations of the Yeo and Johnson (2000) for negative and positive responses, which we introduced in R2020a. More specifically, we intervened on the smoothness condition that the second derivative of zYJ(lambda) with respect to y be smooth at y = 0, along Atkinson et al (2019) and (2020), to allow two values of the transformations parameter: lambdaN for negative observations and lambdaP for non-negative ones. Now, function ScoreYJall computes:1) a global t test associated with the constructed variable for lambda=lambdaP=lambdaN. 2) a t test for positive observations. 3) a t test for negative observations. 4) a F test for the joint presence of the two constructed variables described in points 2) and 3. 5) the F test based on the maximum liklihood estimate of lambdaP and lambdaN
New function ScoreYJall which computes the score tests described in points 1)-5) above.
New function scoreYJmle which computes, in the case of extended Yeo and Johnson transformation, the likelihood ratio test of H0: lambdaP=lambdaP0 and lambdaN=lambdaNeg0.
Added option usefmin in funtion boxcoxR. This option uses the solver (fminsearch or fminunc) to find MLE of the two transformation parameters for extended Yeo and Johnson family (Atkinson et al. 2020).
New function fanBIC which takes in input the output of FSRfan and using BIC and smoothness index enables to automatically choose in an efficient and robust way, the best value of the transformation parameter.
New function fanBICpn which enables to automatically choose the best values of the transformation parameters for positive and negative observations
New function normYJpn which extends the companion functions normBoxCox and normYJ to the case of extended Yeo and Johnson transformation.
New function SETARX which implements Threshold autoregressive models with two regimes
New tools for dealing with the 14 Gaussian parsimonious clustering models (GPCM).
In function genSigmaGPCM new option pa.exactrestriction has been added. If pa.exactrestriction is true the covariance matrices are generated with the exact values of the restrictions specified in pa.cdet, pa.shw and pa.swb. In function MixSim optional input structure sph now can be called with field sph.exactrestriction
In function tclust the
fourth input restrfactor can be a structure which can contain the
type of Gaussian Parsimonious Clustering Model - GPCM (restrfactor.pars),
New plots waterfallchart
(which implements the waterfall chart (see
https://en.wikipedia.org/wiki/Waterfall_chart) and new function
funnelchart which implements the
funnel chart (see
new function scatterboxplot (which creates scatter diagram with marginal boxplots).
Improvements to functions
Now spmplot accepts as input a table. In the case the names of the tables are automatically added at the margins. Similarly, in function corrNominal when option datamatrix is true it is possible to supply as first argument a table.
|We stop supporting
releases before 2015b.
Functions FSRfan, ScoreYJ and ScoreYJall made faster
search engine in our HTML web pages in version 2020b works again.
Routines fanBICpn, tclustICgpcm and tclustICplot gpcm use routine heatmap introduced in MATLAB 2017a.
Routine tclustregeda to display the results uses function parallelplot introduced in Matlab 2019a.
|October 18th 2020|
New set of routines for minimum density power divergence estimators (mdpd, mdpdR, mdpdReda, PDrho, PDpsi, PDwei, PDpsider, PDpsix, PDbdp, PDeff, PDc ), discussed in https://www.mdpi.com/1099-4300/22/4/399)
New function simulateLM to simulate linear regression data with prespecified value of R2, prespecified correlation among the explanatory variables and type of distribution.
New function VIOM which computes weights estimates under a Variance-Inflation Outlier Model using MLE or Restricted MLE (REMLE).
New function tclustregeda which enables to monitor the regression clustering classification for different levels of trimming.
Improved function tclustregIC which enables to compute the BIC (and other information criteria) for different values of restriction factors and different number of groups, for classification or mixture likelihood and regression clustering.
New function ctlcurves to select the appropriate number of groups in robust clustering.
New function mdrrsplot which plots the random starts trajectories and enables to brush them. The companion function mmdrsplot referred to multivariate analysis has been improved.
TRANSFORMATIONS IN REGRESSION
New function boxcoxR which
computes the profile log Likelihood for a range of values of the
transforamtion parameter (lambda) and computes the MLE of lambda in
Improved menu for the automatic installation of the FSDA html help files.
New function exactcdf for finding the exact cdf of each element in a vector x with respect to the empirical distribution, represented by another vector.
|We stop supporting
releases before 2015a.
Functions PoolPrepare.m and PoolClose.m removed (not necessary with the latest releases of parallel computing toolbox).
Cleaner code for tclust and tclultreg functions now available; the core part which computes the EM algorithm for each subset which is extracted, is now in tclustregcore.m and tcluscore.m; these two functions are internal and therefore undocumented.
From this release we support older releases up to 2015a (however about 85% of the FSDA still runs with older versions of MATLAB up to 2012a).
search engine in our HTML web pages in version 2020a does not work
To search for FSDA function it is necessary to use the supplemental software search engine.
Corrected small bug in LTSts in case of level shift with very small series.
This is the first release which is distributed from Mathworks marketplace and from github platform.
TRANSFORMATION IN REGRESSION
New function tBothSides which enables to transform both sides of a (nonlinear) regression model.
New function boxcoxR which finds MLE of lambda in linear regression (and confidence interval) using Box Cox or Yeo and Johnson family.
ROBUST TIME SERIES ANALYSIS
New functions LTStsVarSel.m which enables to perform variable selection in the robust time series model LTSts.m. In functions LTSts.m, simulateTS.m and forecastTS.m it is now possible to add an autoreressive component.
New function existFS which checks whether a file exists and puts the answer in a cached persistent variable
Added file getting_started.mlx in subfolder doc of the main root of FSDA for packaging the FSDA toolbox,
|From this release we
releases up to 2014b. Now functions tclust.m and logmvnpdfFS.m do
not need anymore the presence of MEX files. This modification has
been made necessary because at present Mathworks toolbox packaging
does not support mex files.
||Fixed a bug in
New function startup.m has been added to the main folder of FSDA which copies all HTML documentation files in subfolder (FSDA root)/helpfiles/FSDA into (MATLAB docroot/help). The files can be copied in Windows system just if the user has administrator privileges.
Function tclustreg has been considerably enhanced. Now the function includes: (i) robust BIC, (ii) possibility of constraining the determinants of the covariance matrices of the explanatory variables, (iii) options for treating datasets with concentrated noise, making use of concentration steps appropriately modified using observation weighting and thinning methods.
New function tclustregIC which (if present) uses the Parallel Computing toolbox to compute robust BIC for mixture and classification likeilhood for different values of k (number of groups) and different values of c (restriction factor for the variances of the residuals), for a prespecified level of trimming.
New function for constraining the determinants restrdeter. This function has its own interest but is called in every concentration step of function tclust in case determinant restriction is needed.
Routines for constraining the determinants (restrdeterGPCM), the shape matrices (restrshapeGPCM) and to impose common rotation matrices (common principal components) in presense of equal shape (cpcE.m) or varying shape (cpcV.m) and a general routine to impose constraints in the family of the 14 Gaussian Parsimonious Clustering Models (restrSigmaGPCM).
Routine to generate data based on the 14 Gaussian Parsimonious Clustering Models (genSigmaGPCM). This routine can be called directly from function MixSim in order to generate each of the 14 Gaussian Parsimonious Clustering Models with a prespecified level of overlap (see option sph inside MixSim).
Routine GowerIndex to compute matrix of similarity indexes using Gower metric.
New datasets added to the collection:
Funciton FSMeda is now much faster; the original function FSMeda has been kept, renamed FSMedaeasy, because the algorithm is much easier to follow.
New functions: (i) ace which implements the alternating conditional expectations algorithm to find the transformations of y and X that maximise the proportion of variation in y explained by X and (ii) avas which uses a (nonparametric) variance-stabilizing transformation for the response variable.
New function smothr to smooth values imposing variour constraints (e.g. monotonicity, circularity,..). This function calls the supersmoother routine of Friedman.
New function rlssmo to compute a running line smoother with global cross validation.
New function supsmu to smooth scatterplots using Friedman's supersmoother algorithm.
Function RobCov now includes the estimator covrobc (a corrected version of the covariance matrix of robust beta coefficients). A new motivating example shows a case why covrobc should be always used.
New function repDupValWithMean that enable to replace values of y including non unique elements in vector x with local means.
Function publishFS is fourthly improved. This function automatically transforms structuerd .m files into MATLAB pure style files. In the HTML help files now the right click of the mouse (similarly to pure Mathworks pages) enables to execute, select or find help (F1 key) for all the versions of MATLAB starting from 2017a.
New function genr8 to generate random numbers which are coherent across different software platforms.
New function exactcdf to find exact cdf values of each element of an input vector x with respect to an empirical distribution.
New function wthin which thins a uni/bi-dimensional dataset.
New function ctsub which computes numerical integrarion from x(1) to z(i) of y=f(x)
New functions (i) vervaatsim
(to simulate precisely from a Vervaat perpetuity
|This is the last
release where we we support old
releases up to 2012a.
||Fixed a bug in
mtR when the user wanted to continue the simulation using a negative
(1) New function qqplotFS that enables to create a qqplot of residuals with confidence bands
(2) New function mtR which generates the same random numbers produced by R software with Mersenne Twister mt19937ar
(5) New function verlessthanFS. It is a faster version of MATLAB function verlessthan.
(6) New datasets added to the collection.
(7) New routine publishBibliography to create in a automatic way the bibliography from the citations present inside the .m files.
||Fixed a problem
in the brushing from spmplot and on the diagonal there are the
boxplots. See for more details the additional examples in yXplot and
Improved option for thinning units inside tclustreg
Solved minor bug in FSM when it was called with option 'bonflev'
(1) New function tclusteda that helps choosing the best tclust model. It computes tclust for different values of the trimming factor and produces plots that allow to find the optimal level of trimming. This function uses the parallel processing toolbox, if available.
(2) Extension of the score test. New function ScoreYJpn that computes the score test for Yeo Johnson transformation separately for positive and negative observations. FSRfan now accepts the new option family "YJpn" and it is possible to monitor the score test for both positive and negative observations (output arguments out.Scorep and out.Scoren).
(4) New functions for time series analysis. simulateTS simulates a time series with trend (up to third order), seasonality (constant or of varying amplitude) with a different number of harmonics and a level shift. forecastTS produces forecasts with confidence bands for a time series estimated with function LTSts.
(5) New function verlessthanFS. It is a faster version of MATLAB function verlessthan.
(6) Documentation of yXplot considerably improved. New options added (xlimx, ylimy, namey, nameX).
(7) MixSimreg extended to account for multiple parameter distribution (betadistrib option)
(8) histFS has a new optional argument (weights) for plotting a weighted histogram.
(9) options labenv has been added to mmdrsplot.
(10) option axesellipse added to ellipse
(11) New output argument idxMapping used in function ClusterRelabel, to track the indexes permutations used to rearch a desired cluster labelling.
|From release 2018a we support old
releases up to 2012a.
From 2018a the use of subfunctions tinvFS, finvFS, tcdfFS, fpdfFS, fcdfFS which compute inverse, pdf, cdf of the T and F distribution are not supported anymore.
Following the feedback provided by our users, Function UnitsSameCluster (which was introduced in R2017a) has been renamed (for better readability) ClusterRelabel
|Fixed a series of problems associated to MATLAB 2018a. For example now empty variables which will contain numbers have been initialized with , while empty variables which contain characters are initialized with ' '. See for example functions yXplot and spmplot.||May 2018|
FSDA has introduced two new categories of tools, one for (robust) time series analysis; another for analyzing categorical data and contingency tables. More precisely:
(2) CorAna performs correspondence analysis; SparseTableTest computes independence test for large and sparse contingency tables; CressieRead computes the power divergence family of tests, to check the discrepancy/distance between observed and expected frequencies in a contingency table; rcontFS generates a random two-way table with given marginal totals; barnardtest computes the Barnard test, corrNominal measures strength of association between two unordered (nominal) categorical variables. Similarly for ordinal data with corrOrdinal. crosstab2datamatrix recreates the original data matrix X from contingency table N. This group of functions is complemented by file examples_categorical.m as in style of FSDA.
The two categories of functions will be progressively enriched.
Other new functions which are included are boxtest (test of equality of covariance matrices used for example in tkmeans), GYfilt (Gervini and Yohai, univariate outlier identifier), mmdrsplot (interactive plot of the trajectories of minimum Mahalanobis distances from different starting points), overlapmap to plot the ordered pairwise overlap values between components, dempk to perform a merging of components found by tkmeans, ncpci to compute a non centrality parameter confidence interval.
Finally, spmplot has been enriched to superimpose ellipses, density and contour functions to data and extract single panels from the scatter matrix.
|Various inconsistencies in the browsing of the documentation pages and the bibliography have been fixed.||November 2017|
Function tclustreg now includes trimmed Cluster Weighted Restricted Models.
New function tclustIC for the automatic
selection of the best number of groups.
New routines for density estimation and thinning, for univariate and bivariate data (used in tclustreg).
|Control version added in many graphical
routines to take account the modifications in MATLAB 2017a
From MATLAB R2012a callback functions started to use new internal data structures. Interactive plots that use such callbacks (e.g. those in GUI brushRES) may produce errors in releases older than R2012a. The issue should be now fixed. Please report to us any problem you may experience.
|V4.1 (R2016b)||New functions for bivariate density
estimation and random thinning (kdebiv.m,
rthin.m) used to extend tclustreg.m features.
The FSDA help folder now contains XML files associated to the functions documentation. This is in view of generating/updating automatically or using a GUI the functions documentation, in html as well as in the function head.
New html documentation generated with publishFS.
|We are monitoring possible compatibility
issues that may emerge from changes in the tagging policy of the graphical
objects in MATLAB.
||add2spm modified to take into account
a change in the property name of the legend object introduced in MATLAB
(LegendPeerHandle is now called LayoutPeers).
A change in MATLAB R2016b function legend.m was affecting FSDA function add2yX (the legend was plotted twice). Bug fixed.
Bug affecting FSMeda only in the univariate case fixed.
|V4.0 (R2016a)||Major release.
New function, publishFS, introduced to generate documentation pages directly from the .m files.
New function, makecontentsfileFS, introduced to create a the list of files present in a FSDA folder and/or subfolders. It extends MATLAB function makecontentfilesFS.
.mlx files introduced for examples_multivariate and examples_regression.
New features added to the tclust function, including determinant restriction and new adjusted BIC criterion for the estimation of the number of groups.
Added functions for reweighting FSR and FSRB (FSRr and FSRBr).
Functions FSR, FSRB and FSRH redesigned; a routing implementing the core of the Forward Search algorithm (FSRcore) introduced to avoid code redundancies.
New function, winsor, to winsor data.
New function FSMbsb, which will replace FSMbbm.
New function randindexFS, to evaluate the quality of different clusterings.
New routines poolClose and poolPrepare introduced to conveniently open and close a pool of parallel workers.
Several new robust functions to generate, for example, the Tukey Biweigh rho function (HUrho), the tuning constant associated to a certain efficiency (HUeff), the psi functions (HUpsi), its derivative (HUpsider), etc. For a full list, see functions under utilities_stats folder.
|Documentation made compatible with
the new MATLAB help navigation and browser-style, which was redesigned
in release R2015b.
||Many small bugs fixed. Functions affected
include tclust, tclustReg, position, MixSimReg.
ClickableMultiLegend made compatible with R2016a.
drawnow command introduced in several graphical functions, to solve episodic bugs in the generation of the plots.
Function simdataset.m modified to allow the user to simulate outliers from different distributions and contamination schemes and/or contaminate existing datasets.
New Bayesian regression analysis routines: FSRB.m, FSRBeda.m, FSRBmdr.m, regressB.m.
In FSReda.m: monitoring of confidence intervals of beta and sigma2.
In FSRBeda.m: monitoring of HPD (highest posterior density regions) of beta and sigma2.
New functions for inverse gamma computation: inversegampdf.m, inversegamcdf.m, inversegaminv.m.
Added functions to monitor units forming subset in heterosckedastic and Bayesian regression: FSRHbsb.m, FSRBbsb.m.
Added new datasets for Bayesian examples.
FSMtra.m: added option for the robust transformation in the Yeo-Johnson family.
addFSDA2path.m: modified for compatibility with unix platforms and to address changes in the folder organization of FSDA functions.
Added routines to compute and visualize robust bivariate boxplot (function boxplotb.m)
New routine for automatic outlier detection in heteroskdastic regression (FSRH.m)
|In function spmplot, the multiple histograms (boxplots) on the main
diagonal of the scatter plot matrix is now working also with R2015a
The html documentation pages of some new functions are missing. As a consequence, some links may be broken. New complete documentation, produced automatically from the documentation in the head of our m-files, will be released soon.
Added robust regression (tau) and multivariate estimators (Stahel-Donoho).
Functions taureg.m and SDest.m. New weight functions (hyperbolic, Hampel
Documentation made compatible with the new help system introduced with MATLAB R2015a
In function spmplot, the multiple histograms (boxplots) on the main diagonal of the scatter plot matrix do not work with R2015a because of recent changes in gplotmatrix.
New very fast implementation of the Forward Search for multivariate
analysis (FSMmmd). Forward Search in regression
modified to deal with the cases in which in a particular step of the
search subset is not full rank (FSR and
FSRmdr). New scatter plot matrix with multiple
groups and multiple boxplots on the main diagonal (spmplot).
Structure modified for compatibility with MATLAB R2012b to R2013a releases. Three APPs introduced in release R2013a.
Major release. Traditional robust estimators added including S, MM,
MCD, MVE, (Sreg, Smult,
mcd, mve) and univariate
and bivariate analysis (unibiv). New interactive
plots and interactive graphics features.
Redesigned the definition of many optional input parameters in terms of structures
Multivariate data analysis routines have been added. Function
resfwdplot considerably improved.
A few functions use ~ to denote unused output parameters as from release 2009b