UNIPRconfigCluster

UNIPRconfigCluster creates a cluster profile for MATLAB Parallel Server

expand all in page

Syntax

  • UNIPRconfigCluster(user)example
  • UNIPRconfigCluster(user,Name,Value)example

Description

The routine is designed to use the High Performance Computing Facilities of the University of Parma (HPCUNIPR), https://www.hpc.unipr.it/dokuwiki/doku.php?id=calcoloscientifico:userguide however, it is included in the general utilities of FSDA, because it can be easily extended to exploit other cloud services. UNIPRconfigCluster creates a cluster profile for MATLAB Parallel Server on HPCUNIPR and enables the current client (Windows) PC to become a member of the remote cluster.

PREREQUISITES: this routine assumes that:

1) The user has an account on the remote server (in this case the HPCUNIPR), has generated a SSH key pairs and has appended his/her public key on the remote server inside the file named "authorized_keys" which is located in the .ssh folder of the user. For more information to have/request an account on the HPCUNIPR cluster, please see the userguide at the web address https://www.hpc.unipr.it/dokuwiki/doku.php?id=calcoloscientifico:userguide 2) MATLAB addon "Parallel Computing Toolbox plugin for MATLAB Parallel Server with Slurm" has been installed on the local computer and that the parallel computing toolbox is also installed.

3) In the local computer you have mapped a network drive to your directory in the remote system. For example in the case of the University of Parma the remote directory (assuming the user is paolo.andrei) is \\sshfs\paolo.andrei@gui.hpc.unipr.it. On windows systems the software which enable to do this mapping are called SSHFS-Win · SSHFS for Windows and can be downloaded from the github address https://github.com/billziss-gh/sshfs-win. The two .msi files to install are called sshfs-win-3.5.20357-x64.msi and sshfs-win-3.5.20357-x86.msi.

Note that the default letter of the network drive is H: but it can be changed using option MapNetworkDrive 4) In the remote system you have created the path ~/parallel/matlab Using the varargin it is possible to specify the typical parameters of the SLURM workload manager (https://slurm.schedmd.com/) that is the requested number of workers, number of threads for CPU, number of tasks per node, the amount of RAM ....

example

UNIPRconfigCluster(user) Use all default options.

example

UNIPRconfigCluster(user, Name, Value) Use varargin to specify SLURM paramters UNIPRconfigCluster('john.doe','tasksxnode', 1,...

Examples

expand all

  • Use all default options.

    • We assume that john.doe is a valid user inside UNIPR HPC. 

    UNIPRconfigCluster('john.doe')

  • Use varargin to specify SLURM paramters UNIPRconfigCluster('john.doe','tasksxnode', 1,...

    • UNIPRconfigCluster('john.doe','tasksxnode', 1,...
    'memxcpu','2G', 'NumWorkers', 2, 'NumThreads', 2)

    Related Examples

    expand all

  • Create or update the MATLAB parallel profile only.

    • In this example just the MATLAB cluster profile is created and routine UNIPRrunCluster is not called. 

    UNIPRconfigCluster('john.doe', 'IPaddress', false)

  • Create or update the MATLAB parallel profile and add local client to the remote cluster.

    • UNIPRconfigCluster('john.doe', 'IPaddress', true)
    % Remark: given that the default option IPaddess is set to true
    % the previous instruction was equivalent to the instruction
    % UNIPRconfigCluster('john.doe')

  • run also the function runclusterUNIPR() with specific IP address.

    • With option IPaddress it is possible to control the IP address with which the user wants to join the remote cluster. 

    UNIPRconfigCluster('john.doe', 'IPaddress', '192.168.1.10')

    Input Arguments

    expand all

    user — user SSH credentials on the remote cluster. Character or string.

    User must be specified as a char or as a string. In the case of UNIPR user must be in the format of name.surname (same format of the e-mail account without the domain part).

    Example - 'paolo.andrei' or "paolo.andrei"

    Data Types: character or string

    Name-Value Pair Arguments

    Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside single quotes (' '). You can specify several name and value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

    Example: 'NumWorkers', 2 , 'NumThreads', 2 , 'tasksxnode', 2 , 'memxcpu','2G' or 'memxcpu',"8G" , 'IPaddress', false , 'MapNetworkDrive', 'Z:'

    NumWorkers —number of workers per CPU.positive integer.

    Number of cores to use to perform the calculations.

    The default value is 2.

    Example: 'NumWorkers', 2

    Data Types: double

    NumThreads —number of threads per CPU.positive integer.

    The number of threads that each core is going to use. The default value is 2.

    Example: 'NumThreads', 2

    Data Types: double

    tasksxnode —number of tasks per node.positive integer.

    The default value is 1. Note that MATLAB is able to use multiple CPU-cores via libraries that have been written using shared-memory parallel programming models like OpenMP, Intel Threading Building Blocks (TBB) or pthreads.

    For pure multithreaded codes, only a single node and single task can be used and the optimal value of cpus-per-task is sought. For more information on tasks per node please see https://researchcomputing.princeton.edu/support/knowledge-base/scaling-analysis

    Example: 'tasksxnode', 2

    Data Types: double

    memxcpu —amount of RAM per CPU expressed in GigaBytes.character | string containing a positive integer followed by G.

    The default value is '2G'.

    Note that once you are logged into the cluster it is possible to use the command sinfo in order to find maximum CPU/memory per node.

    https://support.ceci-hpc.be/doc/_contents/SubmittingJobs/SlurmFAQ.html

    Example: 'memxcpu','2G' or 'memxcpu',"8G"

    Data Types: char or string

    IPaddress —initialize and connect the Windows client to the HPC cluster.logical, | char/string containing the IPaddress which must be added to the remote cluster.

    The default value is true, that is the routine uses the IPaddress of the local windows client. If IPaddress is false only the SLURM parameters are set and in order to to connect the Windows client to the cluster it is necessary to run routine UNIPRruncluster. Note that routine UNIPRConfigCluster can be launched every time it is necessary to change the slurm parameters. If the SLURM parameters do not change, than it is enough to call routine UNIPRruncluster to add the current local machine to the cluster.

    Example: 'IPaddress', false

    Data Types: boolean or character or string containing a valid IP address

    MapNetworkDrive —letter which identifies network drive.character | string.

    Letter followed by : which identifies the mapped network drive to your directory in the remote system. The default value of MapNetworkDrive is 'H:'.

    Example: 'MapNetworkDrive', 'Z:'

    Data Types: character or string

    Output Arguments

    expand all

    See Also

    This page has been automatically generated by our routine publishFS