## SIONlib: Scalable I/O library for parallel access to task-local files

### SIONlib: API

Parallel Interface | Serial Interface | Common Interface
--- | --- | ---
sion_paropen_mpi() | sion_open() | sion_ensure_free_space()
sion_parclose_mpi() | sion_open_rank() | sion_feof()
| sion_close() | sion_bytes_avail_in_block()
| sion_get_locations() | sion_seek()
|  | sion_seek_fp()
|  | sion_fwrite()
|  | sion_fread()

#### ---> Parallel open, MPI version: `sion_paropen_mpi()`

      int sion_paropen_mpi(    char          *fname,
                               char*          file_mode,
    			   int           *numFiles,
    			   MPI_Comm       gComm,
    			   MPI_Comm      *lComm,
    			   sion_int64    *chunksize,
    			   sion_int32    *fsblksize,
    			   int           *globalrank,
    			   FILE         **fileptr,
    			   char         **newfname);
      FSION_PAROPEN_MPI( FNAME, FILE_MODE, NUMFILES,
                         GCOMM, LCOMM, CHUNKSIZE, FSBLKSIZE,
                         GLOBALRANK, NEWFNAME, SID)
        CHARACTER*(*) FNAME, FILE_MODE, NEWFNAME
        INTEGER       NUMFILES, FSBLKSIZE, GLOBALRANK, SID
        INTEGER*8     CHUNKSIZE

Description: This function opens a sion file in parallel for writing or reading data. It has to be called from each processor at the same time (like a MPI collective operation). In write-mode SIONlib will create one or more physical files on disk containing data of all tasks (see also file format description). The number has to be defined by the parameters numfiles or lComm. If numfiles is greater than zero the tasks in gComm will be split to numfiles groups. The new communicator will be returned afterwards by parameter lcomm. If numfile is less than zero, the communicator lcomm has to be a local communicators. Tasks which are in the same local communicator will be write to the same physical files. All tasks in gComm has to be also a member of one of the local communicators. The function opens on each task one of the physical files with the POSIX open command in direct access mode and sets the file pointer to a task-specific position. The POSIX file pointer will be returned by the parameter fileptr, which can be used to write or read data from file with the POSIX file operations. Important: all parameters of the sion_open function are always pointer. This is because most of the paramater are input values if the file is opened for write and are output parameters if the file is opened fo read.

Parameters description:
      fname       :  name of file, should equal on all tasks         (input)
      file_mode   :  like the type parameter of fopen:               (input)
                     current recognize option: "rb", "wb"
      numFiles    :  number of physical files to use/used            (in/out)
                     <=0 groups of tasks write to the same file are
                         described by communicator lComm
                     >0  gComm will be split in numfiles local
                         communicators
      gComm       :  MPI communicator, which contains all tasks
                     involved in writing/reading sion file
                     typical: MPI_COMM_WORLD                         (input)
      lComm       :  MPI communicator, which contains all tasks
                     involved in writing/reading to the same
                     physical file
                     typical: MPI_COMM_WORLD                         (in/out)
      chunksize   :  requested space for this task                   (in/out)
      fsblcksize  :  blocksize of filesystem, must be equal on       (in/out)
                     all processors, -1 -> use fs default value
      globalrank  :  any global unique id for this task, will        (in/out)
                     be stored in sion file, usefull if comm is not
                     MPI_COMM_WORLD
                     typical: globalrank= rank in MPI_COMM_WORLD
      fileptr     :  filepointer for this task                       (out)
      newfname    :  pointer name of physical file, used by          (out)
    		 current task, will not returned if newfname
    		 is a NULL pointer, will be freed by sion_par_close()

Return Code:
      sid         :  sion file handle integer (0, ...)
                     -1 if error occured

top of page

#### ---> Parallel close, MPI version: `sion_parclose_mpi()`

      int sion_parclose_mpi(   int      sid );

Description: This function closes the sion in parallel on all tasks which was involved in opening the sion_file. Therefore the local communicator, defined at sion_paropen_mpi, will be used internally to collect all the meta data from each task and to write it at the end of the sion file on task zero of the local communicator. This function has to be called collectively on all tasks of the global communicator given at opening time. There is currently no fault tolerant handling of the meta-data. The varible size meta-data will be held in memory on each task and will written to disk not before calling this function. This means, that the file cannot be read again if there were problem to write the file to end (like touching the disk space limit). One reason for this, is that all operations between parallel open and close are not collective, which allows to allocate additional chunks for task independently.

Parameters description:
      sid         : sion file handle                                (input)

Return Code:
      rc         :  1 if close was ok

top of page

#### ---> Ensure free space for next write: `sion_ensure_free_space()`

      int sion_ensure_free_space(int sid, long numbytes);

Description: This function ensures that there is enough space available for writing numbytes bytes to the sion file. sion_ensure_free_space allocates a new block at the end of the sion file of size chunksize if numbytes cannot be written in the current block. This function is only be needed if the caller function from the user program do not count the number of bytes written. The function is a task local function, which can be called independently from other MPI tasks. The function moves in some cases the filepointer to a new position and flushes also the local filepointer.

Parameters description:
      sid         : sion file handle                                 (input)
      numbytes    : number of bytes which will be written            (input)

Return Code:
      rc          :  1 if space could ensured, there is currently
                     no indicator if a new chunk was allocated

top of page

#### ---> check end of file (chunk): `sion_feof()`

      int sion_feof(int sid);

Description: This function indicates if the end of file is reached for this task. This means that the file pointer is behind of the last bytes of last chunk of this task. If all bytes of the current chunk are already read and there are more chunks available for this task, sion_feof will advance the filepointer to the start position of the next chunk in the sion file. The function is a task local function, which can be called independently from other MPI tasks.

Parameters description:
      sid         : sion file handle                                 (input)

Return Code:
      rc         :  1 if end of last chunk reached
                    0 otherwise

top of page

#### ---> get available bytes in current block (chunk): `sion_bytes_avail_in_block()`

      long sion_bytes_avail_in_block( int sid );

Description: This function returns the number of bytes currently available in the current chunk (bytes not read). This function is a local function, which can be called independently from other MPI tasks.

Parameters description:
      sid         : sion file handle                                 (input)

Return Code:
      rc         :  rc>0 number of bytes
                    rc<=0 file position is after end of block

top of page

#### ---> open one rank of a sion file: `sion_open_rank()`

      int sion_open_rank(    char  *fname,
                             char  *file_mode,
                             long  *chunksize,
                             int   *fsblocksize,
                             int   *rank,
                             FILE **fileptr);

Description: This function opens a sion file for a specific rank. It can be used to open the sion file independently from each task. E.g. if no MPI is available or only a subset of tasks chunks are needed. Using this function the meta data at beginning of the sion file are read from each task instead of read once and distribute (sion_open). sion_open_rank reads only the tasks specific meta data from the meta data block at the end of the sion file. Warning: Only read operations are currently supported.

Parameters description:
      fname       :  name of file, should equal on all tasks         (input)
      file_mode   :  like the type parameter of fopen:               (input)
                     current recognize option: "rb", "wb"
      chunksize   :  chunksize on this task                          (in/out)
      fsblocksize :  blocksize of filesystem, must be equal on       (in/out)
                     all processors
      rank        :  rank number for which the file should be open   (in)
      fileptr     :  filepointer for this task                       (out)

Return Code:
      sid         :  sion file handle integer (0, ...)
                     -1 if error occured

top of page

#### ---> open all ranks of a sion file: `sion_open()`

      int sion_open(    char *fname,
                        char* file_mode,
                        int  *ntasks,
                        int  *nfiles,
    		    sion_int64 **chunksizes,
                        sion_int32  *fsblksize,
                        int  **globalranks,
                        FILE **fileptr);

Description: This function opens all tasks of a sion file in serial mode. The meta-data of all tasks will be loaded in memory, which allows to access different parts of the file by using sion_seek_fp(), for example to switch between ranks. Default position fo the filepointer is before the first byte of the first task. In the case of multi-file sion-files sion_open() open internally each physical file with sion_open recursively. sion_open manage for each physical file an own sion file descriptor and data structure. Warning: multi-file-support is not implemented in sion_open of the current version (1.1p9).

Parameters description:
      fname       :  name of file, should equal on all tasks         (input)
      file_mode   :  like the type parameter of fopen:               (input)
                     current recognize option: "rb", "wb"
      ntasks      :  number of tasks used to write this file         (in/out)
      nfiles      :  number of physical files to be used             (in/out)
      chunksizes  :  array of chunksizes for each task               (in/out)
      fsblcksize  :  blocksize of filesystem-1 ->
                     use fs default value                            (in/out)

      globalranks :  array of global rank number for each task       (in/out)
      fileptr     :  filepointer for this task                       (out)

Return Code:
      sid         :  sion file handle integer (0, ...)
                     -1 if error occured

top of page

#### ---> close serial opened file: `sion_close()`

      int sion_close(int sid);

Description: This function closes a sion file which was opened in serial mode with sion_open or sion_open_rank. In Write mode this function will also all meta data to the meta data block of the sion file at the beginning and the end of the file. The function is a task local function, which can be called independently from other MPI tasks.

Parameters description:
       sid         : sion file handle                                 (input)

Return Code:
       rc          : 1 if ok

top of page

#### ---> get meta dat from a sion file: `sion_get_locations()`

      int sion_get_locations(    int    sid,
                                 int   *size,
                                 int   *chunks,
                                 sion_int64 *globalskip,
                                 sion_int64 *start_of_varheader,
                                 sion_int64 **sion_localsizes,
                                 sion_int64 **sion_globalranks,
                                 sion_int64 **sion_chunkcount,
                                 sion_int64 **sion_chunksizes);

Description: sion_get_locations returns pointers to internal fields, containing the number of chunks written by each task (sion_chunkcount) and their sizes (sion_chunksizes). This function is only needed if the sion file was opened for reading in serial mode.

Parameters description:
      sid         : sion file handle                               (input)
      size        : number of tasks wrote to the sion file           (out)
      chunks      : maximum number of chunks per tasks used          (out)
      globalskip  : distance in bytes between the  first bytes of
                    two subsequently chunks of a task                (out)
      start_of_varheader: start position of the meta data block
                          at the end of sion file                    (out)
      sion_localsizes:    field containing requested chunk size of
                          each task                                  (out)
      sion_globalranks:   field containing global unique id of
                          each task                                  (out)
      sion_chunkcount:    field containing number of chunks used by
                          each task                                  (out)
      sion_chunksizes:    field containing for each tassk and chunk  (out)
                          the number of bytes used in this chunk
                          access: sion_chunksizes[rank*chunks+chunknr]

Return Code:
        rc           :  1

top of page

#### ---> set file pointer to new position: `sion_seek()`

      int sion_seek(  int  sid,
                      int  rank,
                      int  chunknum,
                      long posinchunk );

Description: set the file pointer to a new position according to the specified parameters. This function can only be used the sion file was opened for reading or writing in serial mode. In write mode currently only the rank can be modified, SION_CURRENT_BLK and SION_CURRENT_POS are required. This function assumes, that the sion file has only one physical files. If the file was created with numfiles>1 the function sion_seek_fp() should be used, which has an additional output parameter fileptr.

Parameters description:
      sid         : sion file handle                                 (input)
      rank        : rank number (0,...)                              (input)
                    SION_CURRENT_RANK to select the current rank
      chunknum    : chunk number (0,...)                             (input)
                    SION_CURRENT_BLK to select the current block
      posinchunk  : position (0,...)                                 (input)
                    SION_CURRENT_POS to select the current position

Return Code:
      Return Code: 1 if file pointer can be moved to new position
                   0 otherwise

top of page

#### ---> set file pointer to new position (multi-file version): `sion_seek_fp()`

      int sion_seek(  int    sid,
                      int    rank,
                      int    chunknum,
                      long   posinchunk,
                      FILE **fileptr
     );

Description: set the file pointer to a new position according to the specified parameters. This function can only be used the sion file was opened for reading in serial mode. In write mode currently only the rank can be modified, SION_CURRENT_BLK and SION_CURRENT_POS are required. If the sion-file has multiple physical files the function will also change the current files pointer to the file which contains the data of the requested rank. Therefore utilities which uses the serial interface of SIONlib to access sion file should use this function to switch between different tasks of a sion file and should pass the pointer to the POSIX file pointer also to this function.

Parameters description:
      sid         : sion file handle                                 (input)
      rank        : rank number (0,...)                              (input)
                    SION_CURRENT_RANK to select the current rank
      chunknum    : chunk number (0,...)                             (input)
                    SION_CURRENT_BLK to select the current block
      posinchunk  : position (0,...)                                 (input)
                    SION_CURRENT_POS to select the current position

Return Code:
      Return Code: 1 if file pointer can be moved to new position
                   0 otherwise

top of page

#### ---> wrapper function for fwrite, ensures also free space interrnally: `sion_fwrite()`

      int sion_fwrite(  const void *data,
                        size_t      size,
                        size_t      n_items,
                        int         sid );

Description: is a wrapper function for C-ANSI function fwrite. Internally this function checks with sion_ensure_free_space if enough space is available.

Parameters description:
      data        : see fwrite documentation
      size        : see fwrite documentation
      nitems      : see fwrite documentation
      sid         : sion file handle                                 (input)

Return Code:
      Return Code: number of elements written (see fwrite documentation)

top of page

#### ---> wrapper function for fread: `sion_fread()`

      int sion_fread(  const void *data,
                       size_t      size,
                       size_t      n_items,
                       int         sid );

Description: is a wrapper function for C-ANSI function fread. Internally this function reads in a while loop until all data is read from file. Reading more data than stored in one chunk is with this wrapper possible.

Parameters description:
      data        : see fread documentation
      size        : see fread documentation
      nitems      : see fread documentation
      sid         : sion file handle                                 (input)

Return Code:
      Return Code: number of elements read (see fread documentation)

top of page