Generic API

an abstraction of communication between parallel tasks More...

Collaboration diagram for Generic API:

Modules
	Collective API
	I/O operations that involve all tasks that have opened a file.

Typedefs
typedef int(*	sion_create_lcg_cb) (void **local_commgroup, void *global_commgroup, int grank, int gsize, int lrank, int lsize, int filenumber, int numfiles)
	creates data structures for local communication group More...
typedef int(*	sion_free_lcg_cb) (void *local_commgroup)
	free data structure which local_commgroup points on
typedef int(*	sion_barrier_cb) (void *commgroup)
	performs a barrier on all tasks described with commgroup (local or global) More...
typedef int(*	sion_bcastr_cb) (void *data, void *commgroup, int datatype, int nelem, int root)
	performs a broadcast operation from task root to all other tasks described with commgroup (local or global) More...
typedef int(*	sion_gatherr_cb) (void *indata, void *outdata, void *commgroup, int datatype, int nelem, int root)
	performs a gather operation on all tasks described with commgroup (local or global) More...
typedef int(*	sion_gathervr_cb) (void *indata, void *outdata, void *commdata, int datatype, int *count, int nelem, int root)
	performs a gather operation on all tasks described with commdata (local or global) More...
typedef int(*	sion_scatterr_cb) (void *indata, void *outdata, void *commgroup, int datatype, int nelem, int root)
	performs a scatter operation on all tasks described with commgroup (local or global) More...
typedef int(*	sion_scattervr_cb) (void *indata, void *outdata, void *commgroup, int datatype, int *count, int nelem, int root)
	performs a scatter operation on all tasks described with commgroup (local or global) More...
typedef char *(*	sion_get_multi_filename_cb) (const char *, int)
typedef int64_t(*	sion_gather_execute_cb) (const void *, int64_t, void *, int, int, int, int, int64_t process_cb(const void *, int64_t *, int), void *, bool spec_iterator_cb(void *, int64_t *))
typedef int64_t(*	sion_execute_scatter_cb) (void *, int64_t, void *, int, int, int, int, int64_t process_cb(void *, int64_t *, int), void *, bool spec_iterator_cb(void *, int64_t *))
typedef int(*	sion_get_capability_cb) (void *)

Functions
int	sion_generic_create_api (char *name)
	Create new api. More...
int	sion_generic_free_api (int aip)
	free new api More...
int	sion_generic_register_create_local_commgroup_cb (int aid, sion_create_lcg_cb cb)
	register callback to create local communication group
int	sion_generic_register_free_local_commgroup_cb (int aid, sion_free_lcg_cb cb)
	register callback to free local communication group
int	sion_generic_register_barrier_cb (int aid, sion_barrier_cb cb)
	register callback for communication
int	sion_generic_register_bcastr_cb (int aid, sion_bcastr_cb cb)
	register callback for communication
int	sion_generic_register_gatherr_cb (int aid, sion_gatherr_cb cb)
	register callback for communication
int	sion_generic_register_gathervr_cb (int aid, sion_gathervr_cb cb)
	register callback for communication
int	sion_generic_register_scatterr_cb (int aid, sion_scatterr_cb cb)
	register callback for communication
int	sion_generic_register_scattervr_cb (int aid, sion_scattervr_cb cb)
	register callback for communication
int	sion_generic_register_get_multi_filename_cb (int aid, sion_get_multi_filename_cb cb)
int	sion_generic_register_gather_and_execute_cb (int aid, sion_gather_execute_cb cb)
	register callback for communication
int	sion_generic_register_execute_and_scatter_cb (int aid, sion_execute_scatter_cb cb)
	register callback for communication
int	sion_generic_register_get_capability_cb (int aid, sion_get_capability_cb cb)
int	sion_generic_paropen (int aid, const char *fname, const char *file_mode, int64_t *chunksize, int32_t *fsblksize, void *gcommgroup, int grank, int gsize, int *filenumber, int *numfiles, const int *lrank, const int *lsize, FILE **fileptr, char **newfname)
	Open a sion file a generic interface. More...
int	sion_generic_parclose (int sid)
int	sion_generic_paropen_mapped (int aid, char *fname, const char *file_mode, int *numFiles, void *gcommgroup, int grank, int gsize, int *nlocaltasks, int **globalranks, int64_t **chunksizes, int **mapping_filenrs, int **mapping_lranks, int32_t *fsblksize, FILE **fileptr)
int	sion_generic_parclose_mapped (int sid)
int	sion_generic_parreinit (int sid, int64_t chunksize)

Detailed Description

an abstraction of communication between parallel tasks

Introduction

The generic API of SIONlib is designed to implement a user-defined SIONlib parallel API. This new feature of SIONlib enables also SIONlib for those application, not using one of the SIONlib standard communication layers like MPI, OpenMP, or hybrid (MPI+OpenMP).

SIONlib requires only a few communication functions to collect and distributed meta-data information during open- and close-call among the parallel tasks. Most of these communication functions are collective operation like bcast, gather, or scatter.

Communications Groups

According to the existing SIONlib standard communication layer, SIONlib uses two different groups of tasks:

• Global
  • This communication group includes all tasks which will be part of the SIONlib file operation (e.g. MPI_COMM_WORLD).
• Local
  • SIONlib uses this communination group to collect those tasks which accesses same physical file.

A data structure has to be defined and allocated for the global commgroup before sion_open call and a pointer to the data structure has to be passed to sion_open as a parameter.

The local commgroup has to be created during sion file open. Therefore, a callback function fore create and free such a commgroup has to be registered. The callback functions for communication will get a pointer to one of these commgroup data structures afterwards as a parameter.

SIONlib uses both commgroups for communication: the global commgroup for example to collect and distribute task-to-file mapping information. The local commgroup will be used to collect and distribute file meta information (e.g. chunksizes). Callback functions are therefore required to be able to perform the implemented communication operation on both commgroup.

The second, local communication group will be defined by SIONlib according to the parameters filenumber, numfiles, lrank and lsize given by each task to the sion_generic_paropen call. For the definition of the commgroup SIONlib also uses a callback function internally, which has to create the commgroup and store the data into the corresponding data structure.

Remarks:

• The communination groups are not dynamic. This means, the communication groups must not be changed after a file is opened.

Usage

For using the generic SIONlib API the following steps are required:

Definition phase

• Definition of callback functions and commgroups
• Registration of callback functions and commgroups

I/O-operation phase

• Parallel open of SIONlib file
• I/O-operations (read/write, ..)
• Parallel close of SIONlib file

Example

An example can be found in the parallel test section of the SIONlib distribution:

• <topdir>/test/serial/test_genapi_1.c (generic API, only create and link test)
• <topdir>/test/parallel/test_genapi_1.c (generic API)
• <topdir>/test/parallel/test_genapi_2.c (generic mapped API)

The example defines a SIONlib generic API and performs some I/O tests with the new API. Internally the API uses MPI to implement the communication operations.

Typedef Documentation

sion_barrier_cb

typedef int(* sion_barrier_cb) (void *commgroup)

performs a barrier on all tasks described with commgroup (local or global)

Blocks until all processes have reached this routine.

Definition at line 144 of file sion_generic.h.

sion_bcastr_cb

typedef int(* sion_bcastr_cb) (void *data, void *commgroup, int datatype, int nelem, int root)

performs a broadcast operation from task root to all other tasks described with commgroup (local or global)

Replicates data on root at memory position data of size nelem to memory position data of all other tasks.

Definition at line 149 of file sion_generic.h.

sion_create_lcg_cb

typedef int(* sion_create_lcg_cb) (void **local_commgroup, void *global_commgroup, int grank, int gsize, int lrank, int lsize, int filenumber, int numfiles)

creates data structures for local communication group

• only local_commgroup is output parameter: pointer to new data structure
• will be called as soon as all information is available (e.g. after read of mapping during paropen)

Definition at line 135 of file sion_generic.h.

sion_gatherr_cb

typedef int(* sion_gatherr_cb) (void *indata, void *outdata, void *commgroup, int datatype, int nelem, int root)

performs a gather operation on all tasks described with commgroup (local or global)

• collects nelem of data type datatype from each task from memory at indata and stores data of tasks on task root in memory at position outdata
• nelem has same the value on each task, outdata is the of size nelem * ntasks
• on root indata and outdata are not overlapping

Definition at line 157 of file sion_generic.h.

sion_gathervr_cb

typedef int(* sion_gathervr_cb) (void *indata, void *outdata, void *commdata, int datatype, int *count, int nelem, int root)

performs a gather operation on all tasks described with commdata (local or global)

• collects nelem of data type datatype from each tasks from memory at indata and stores data of tasks on task root in memory at position outdata
• nelem can be different among tasks (or zero), size of outdata is sum of nelem on each task
• count is an array containing the number of elems per task, this array will only to be provided on root, otherwise NULL
• on root indata and outdata are not overlapping

Definition at line 166 of file sion_generic.h.

sion_scatterr_cb

typedef int(* sion_scatterr_cb) (void *indata, void *outdata, void *commgroup, int datatype, int nelem, int root)

performs a scatter operation on all tasks described with commgroup (local or global)

• distributes nelem of data type datatype to each tasks from memory at outdata + offset on task root and stores data in memory at position indata
• nelem has same value on each task, outdata is of size nelem * ntasks
• on root indata and outdata are not overlapping

Definition at line 174 of file sion_generic.h.

sion_scattervr_cb

typedef int(* sion_scattervr_cb) (void *indata, void *outdata, void *commgroup, int datatype, int *count, int nelem, int root)

performs a scatter operation on all tasks described with commgroup (local or global)

• distributes nelem of data type datatype to each tasks from memory at outdata + offset on task root and stores data in memory at position indata
• count is an array containing the number of elems per task, this array must only be provided on root, otherwise NULL
• nelem can be different among tasks (or zero), size of outdata is sum of nelem on each task
• on root indata and outdata are not overlapping

Definition at line 183 of file sion_generic.h.

Function Documentation

sion_generic_create_api()

int sion_generic_create_api

(

char *

name

)

Create new api.

Parameters

name	a name for that API which will be used in debugging messages for example

Returns

an integer value containing an API descriptor

Definition at line 31 of file sion_generic.c.

sion_generic_free_api()

int sion_generic_free_api

(

int

aip

)

free new api

Frees internal data structures for this API.

Definition at line 59 of file sion_generic.c.

sion_generic_paropen()

int sion_generic_paropen	(	int	aid,
		const char *	fname,
		const char *	file_mode,
		int64_t *	chunksize,
		int32_t *	fsblksize,
		void *	gcommgroup,
		int	grank,
		int	gsize,
		int *	filenumber,
		int *	numfiles,
		const int *	lrank,
		const int *	lsize,
		FILE **	fileptr,
		char **	newfname
	)

Open a sion file a generic interface.

For a description please see the description of sion_paropen_mpi.

• WRITE-Mode
  • The parameters gcommgroup, grank, gsize, filenumber, numfiles, lrank, lsize are input parameters. However these have to be defined as variables and initialized before call.
• READ-Mode
  • grank and gsize are input parameters
  • filenumber, numfiles, lrank, lsize are output parameters and will be read from SIONlib file meta data
  • The data container of the SIONlib file (chunks) from file will be mapped to tasks according to the order given by the global commgroup.

Parameters

[in]	aid	sion API id
[in]	fname	name of file, should be equal on all tasks
[in]	file_mode	like the type parameter of fopen (currently recognized options: "rb", "wb")
[in,out]	chunksize	maximum size to be written with single write call
[in,out]	fsblksize	file system block size (-1 for automatic)
[in]	gcommgroup	global communicator
[in]	grank	global rank of process
[in]	gsize	size of global communicator
[in,out]	filenumber	file number (for multiple file mode)
[in,out]	numfiles	number of files to use (-1 for automatic choosing from local communicator)
[in]	lrank	local rank of process
[in]	lsize	size of local communicator
[in,out]	fileptr	file pointer (NULL for not using an external file pointer)
[out]	newfname	return value for actual file name if using multiple files

Return values

sid	sion file handle or -1 if error occured

Definition at line 362 of file sion_generic.c.

sion_generic_paropen_mapped()

int sion_generic_paropen_mapped	(	int	aid,
		char *	fname,
		const char *	file_mode,
		int *	numFiles,
		void *	gcommgroup,
		int	grank,
		int	gsize,
		int *	nlocaltasks,
		int **	globalranks,
		int64_t **	chunksizes,
		int **	mapping_filenrs,
		int **	mapping_lranks,
		int32_t *	fsblksize,
		FILE **	fileptr
	)

each task specifies a set of globalranks which should opened on this task

Parameters

aid	SIONlib API id
fname	name of file
file_mode	like the type parameter of fopen
numFiles	number of files to use
gcommgroup	global communicator
grank	global rank of calling task
gsize	size of global communicator
nlocaltasks	gives the number of globalranks to be opened by this task
globalranks	an array containing these globalranks to be opened
chunksizes	an array containing the chunksize for each globalrank which is locally opened
mapping_filenrs	an array containing the filenumber for each globalrank which is locally opened
mapping_lranks	an array containing the local rank in the file for each globalrank which is locally opened
fsblksize	file system block size
fileptr	file pointer

Returns

a SIONlib file handle or -1 if an error occurred

• WRITE-Mode
  • The parameters gcommgroup, grank, gsize, numFiles, nlocaltasks, globalranks, chunksizes, mapping_filenrs, mapping_lranks, fsblksize are input parameters. However these have to be defined as variables and initialized before call.
• READ-Mode
  • The parameters gcommgroup, grank, gsize, numfiles, nlocaltasks, globalranks are input parameter.

Definition at line 620 of file sion_generic.c.