SIONlib: Scalable I/O library for parallel access to task-local files

SIONlib: API

┌───────────────────┬────────────────────┬───────────────────────────┐
│Parallel Interface │  Serial Interface  │     Common Interface      │
├───────────────────┼────────────────────┼───────────────────────────┤
│sion_paropen_mpi() │sion_open()         │sion_ensure_free_space()   │
├───────────────────┼────────────────────┼───────────────────────────┤
│sion_parclose_mpi()│sion_open_rank()    │sion_feof()                │
├───────────────────┼────────────────────┼───────────────────────────┤
│                   │sion_close()        │sion_bytes_avail_in_block()│
├───────────────────┼────────────────────┼───────────────────────────┤
│                   │sion_get_locations()│sion_seek()                │
├───────────────────┼────────────────────┼───────────────────────────┤
│                   │                    │sion_seek_fp()             │
├───────────────────┼────────────────────┼───────────────────────────┤
│                   │                    │sion_fwrite()              │
├───────────────────┼────────────────────┼───────────────────────────┤
│                   │                    │sion_fread()               │
└───────────────────┴────────────────────┴───────────────────────────┘

 
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

---> Parallel open, MPI version: sion_paropen_mpi()

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  int sion_paropen_mpi(    char          *fname,
                           char*          file_mode,
                           int           *numFiles,
                           MPI_Comm       gComm,
                           MPI_Comm      *lComm,
                           sion_int64    *chunksize,
                           sion_int32    *fsblksize,
                           int           *globalrank,
                           FILE         **fileptr,
                           char         **newfname);

  FSION_PAROPEN_MPI( FNAME, FILE_MODE, NUMFILES,
                     GCOMM, LCOMM, CHUNKSIZE, FSBLKSIZE,
                     GLOBALRANK, NEWFNAME, SID)
    CHARACTER*(*) FNAME, FILE_MODE, NEWFNAME
    INTEGER       NUMFILES, FSBLKSIZE, GLOBALRANK, SID
    INTEGER*8     CHUNKSIZE

Description:
This function opens a sion file in parallel for writing or reading data. It has
to be called from each processor at the same time (like a MPI collective
operation). In write-mode SIONlib will create one or more physical files on
disk containing data of all tasks (see also file format description). The
number has to be defined by the parameters numfiles or lComm. If numfiles is
greater than zero the tasks in gComm will be split to numfiles groups. The new
communicator will be returned afterwards by parameter lcomm. If numfile is less
than zero, the communicator lcomm has to be a local communicators. Tasks which
are in the same local communicator will be write to the same physical files.
All tasks in gComm has to be also a member of one of the local communicators.

The function opens on each task one of the physical files with the POSIX open
command in direct access mode and sets the file pointer to a task-specific
position. The POSIX file pointer will be returned by the parameter fileptr,
which can be used to write or read data from file with the POSIX file
operations.

Important: all parameters of the sion_open function are always pointer. This is
because most of the paramater are input values if the file is opened for write
and are output parameters if the file is opened fo read.

Parameters description:

  fname       :  name of file, should equal on all tasks         (input)
  file_mode   :  like the type parameter of fopen:               (input)
                 current recognize option: "rb", "wb"
  numFiles    :  number of physical files to use/used            (in/out)
                 <=0 groups of tasks write to the same file are
                     described by communicator lComm
                 >0  gComm will be split in numfiles local
                     communicators
  gComm       :  MPI communicator, which contains all tasks
                 involved in writing/reading sion file
                 typical: MPI_COMM_WORLD                         (input)
  lComm       :  MPI communicator, which contains all tasks
                 involved in writing/reading to the same
                 physical file
                 typical: MPI_COMM_WORLD                         (in/out)
  chunksize   :  requested space for this task                   (in/out)
  fsblcksize  :  blocksize of filesystem, must be equal on       (in/out)
                 all processors, -1 -> use fs default value
  globalrank  :  any global unique id for this task, will        (in/out)
                 be stored in sion file, usefull if comm is not
                 MPI_COMM_WORLD
                 typical: globalrank= rank in MPI_COMM_WORLD
  fileptr     :  filepointer for this task                       (out)
  newfname    :  pointer name of physical file, used by          (out)
                 current task, will not returned if newfname
                 is a NULL pointer, will be freed by sion_par_close()


Return Code:

  sid         :  sion file handle integer (0, ...)
                 -1 if error occured

[go_db] top of page

 
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

---> Parallel close, MPI version: sion_parclose_mpi()

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  int sion_parclose_mpi(   int      sid );

Description:
This function closes the sion in parallel on all tasks which was involved in
opening the sion_file. Therefore the local communicator, defined at
sion_paropen_mpi, will be used internally to collect all the meta data from
each task and to write it at the end of the sion file on task zero of the local
communicator. This function has to be called collectively on all tasks of the
global communicator given at opening time.

There is currently no fault tolerant handling of the meta-data. The varible
size meta-data will be held in memory on each task and will written to disk not
before calling this function. This means, that the file cannot be read again if
there were problem to write the file to end (like touching the disk space
limit). One reason for this, is that all operations between parallel open and
close are not collective, which allows to allocate additional chunks for task
independently.

Parameters description:

  sid         : sion file handle                                (input)

Return Code:

  rc         :  1 if close was ok

[go_db] top of page

 
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

---> Ensure free space for next write: sion_ensure_free_space()

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  int sion_ensure_free_space(int sid, long numbytes);

Description:
This function ensures that there is enough space available for writing numbytes
bytes to the sion file. sion_ensure_free_space allocates a new block at the end
of the sion file of size chunksize if numbytes cannot be written in the current
block. This function is only be needed if the caller function from the user
program do not count the number of bytes written. The function is a task local
function, which can be called independently from other MPI tasks. The function
moves in some cases the filepointer to a new position and flushes also the
local filepointer.

Parameters description:

  sid         : sion file handle                                 (input)
  numbytes    : number of bytes which will be written            (input)

Return Code:

  rc          :  1 if space could ensured, there is currently
                 no indicator if a new chunk was allocated

[go_db] top of page

 
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

---> check end of file (chunk): sion_feof()

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  int sion_feof(int sid);

Description:
This function indicates if the end of file is reached for this task. This means
that the file pointer is behind of the last bytes of last chunk of this task.
If all bytes of the current chunk are already read and there are more chunks
available for this task, sion_feof will advance the filepointer to the start
position of the next chunk in the sion file. The function is a task local
function, which can be called independently from other MPI tasks.

Parameters description:

  sid         : sion file handle                                 (input)

Return Code:

  rc         :  1 if end of last chunk reached
                0 otherwise

[go_db] top of page

 
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

---> get available bytes in current block (chunk): sion_bytes_avail_in_block()

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  long sion_bytes_avail_in_block( int sid );

Description:
This function returns the number of bytes currently available in the current
chunk (bytes not read). This function is a local function, which can be called
independently from other MPI tasks.

Parameters description:

  sid         : sion file handle                                 (input)

Return Code:

  rc         :  rc>0 number of bytes
                rc<=0 file position is after end of block

[go_db] top of page

 
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

---> open one rank of a sion file: sion_open_rank()

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  int sion_open_rank(    char  *fname,
                         char  *file_mode,
                         long  *chunksize,
                         int   *fsblocksize,
                         int   *rank,
                         FILE **fileptr);

Description:
This function opens a sion file for a specific rank. It can be used to open the
sion file independently from each task. E.g. if no MPI is available or only a
subset of tasks chunks are needed. Using this function the meta data at
beginning of the sion file are read from each task instead of read once and
distribute (sion_open). sion_open_rank reads only the tasks specific meta data
from the meta data block at the end of the sion file. Warning: Only read
operations are currently supported.

Parameters description:

  fname       :  name of file, should equal on all tasks         (input)
  file_mode   :  like the type parameter of fopen:               (input)
                 current recognize option: "rb", "wb"
  chunksize   :  chunksize on this task                          (in/out)
  fsblocksize :  blocksize of filesystem, must be equal on       (in/out)
                 all processors
  rank        :  rank number for which the file should be open   (in)
  fileptr     :  filepointer for this task                       (out)

Return Code:

  sid         :  sion file handle integer (0, ...)
                 -1 if error occured

[go_db] top of page

 
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

---> open all ranks of a sion file: sion_open()

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  int sion_open(    char *fname,
                    char* file_mode,
                    int  *ntasks,
                    int  *nfiles,
                    sion_int64 **chunksizes,
                    sion_int32  *fsblksize,
                    int  **globalranks,
                    FILE **fileptr);

Description:
This function opens all tasks of a sion file in serial mode. The meta-data of
all tasks will be loaded in memory, which allows to access different parts of
the file by using sion_seek_fp(), for example to switch between ranks. Default
position fo the filepointer is before the first byte of the first task.

In the case of multi-file sion-files sion_open() open internally each physical
file with sion_open recursively. sion_open manage for each physical file an own
sion file descriptor and data structure.

Warning: multi-file-support is not implemented in sion_open of the current
version (1.1p9).

Parameters description:

  fname       :  name of file, should equal on all tasks         (input)
  file_mode   :  like the type parameter of fopen:               (input)
                 current recognize option: "rb", "wb"
  ntasks      :  number of tasks used to write this file         (in/out)
  nfiles      :  number of physical files to be used             (in/out)
  chunksizes  :  array of chunksizes for each task               (in/out)
  fsblcksize  :  blocksize of filesystem-1 ->
                 use fs default value                            (in/out)

  globalranks :  array of global rank number for each task       (in/out)
  fileptr     :  filepointer for this task                       (out)

Return Code:

  sid         :  sion file handle integer (0, ...)
                 -1 if error occured

[go_db] top of page

 
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

---> close serial opened file: sion_close()

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  int sion_close(int sid);

Description:
This function closes a sion file which was opened in serial mode with sion_open
or sion_open_rank. In Write mode this function will also all meta data to the
meta data block of the sion file at the beginning and the end of the file. The
function is a task local function, which can be called independently from other
MPI tasks.

Parameters description:

   sid         : sion file handle                                 (input)

Return Code:

   rc          : 1 if ok

[go_db] top of page

 
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

---> get meta dat from a sion file: sion_get_locations()

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  int sion_get_locations(    int    sid,
                             int   *size,
                             int   *chunks,
                             sion_int64 *globalskip,
                             sion_int64 *start_of_varheader,
                             sion_int64 **sion_localsizes,
                             sion_int64 **sion_globalranks,
                             sion_int64 **sion_chunkcount,
                             sion_int64 **sion_chunksizes);

Description:
sion_get_locations returns pointers to internal fields, containing the number
of chunks written by each task (sion_chunkcount) and their sizes
(sion_chunksizes). This function is only needed if the sion file was opened for
reading in serial mode.

Parameters description:

  sid         : sion file handle                               (input)
  size        : number of tasks wrote to the sion file           (out)
  chunks      : maximum number of chunks per tasks used          (out)
  globalskip  : distance in bytes between the  first bytes of
                two subsequently chunks of a task                (out)
  start_of_varheader: start position of the meta data block
                      at the end of sion file                    (out)
  sion_localsizes:    field containing requested chunk size of
                      each task                                  (out)
  sion_globalranks:   field containing global unique id of
                      each task                                  (out)
  sion_chunkcount:    field containing number of chunks used by
                      each task                                  (out)
  sion_chunksizes:    field containing for each tassk and chunk  (out)
                      the number of bytes used in this chunk
                      access: sion_chunksizes[rank*chunks+chunknr]

Return Code:

    rc           :  1

[go_db] top of page

 
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

---> set file pointer to new position: sion_seek()

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  int sion_seek(  int  sid,
                  int  rank,
                  int  chunknum,
                  long posinchunk );

Description:
set the file pointer to a new position according to the specified parameters.
This function can only be used the sion file was opened for reading or writing
in serial mode. In write mode currently only the rank can be modified,
SION_CURRENT_BLK and SION_CURRENT_POS are required.

This function assumes, that the sion file has only one physical files. If the
file was created with numfiles>1 the function sion_seek_fp() should be used,
which has an additional output parameter fileptr.

Parameters description:

  sid         : sion file handle                                 (input)
  rank        : rank number (0,...)                              (input)
                SION_CURRENT_RANK to select the current rank
  chunknum    : chunk number (0,...)                             (input)
                SION_CURRENT_BLK to select the current block
  posinchunk  : position (0,...)                                 (input)
                SION_CURRENT_POS to select the current position

Return Code:


  Return Code: 1 if file pointer can be moved to new position
               0 otherwise

[go_db] top of page

 
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

---> set file pointer to new position (multi-file version): sion_seek_fp()

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  int sion_seek(  int    sid,
                  int    rank,
                  int    chunknum,
                  long   posinchunk,
                  FILE **fileptr
 );

Description:
set the file pointer to a new position according to the specified parameters.
This function can only be used the sion file was opened for reading in serial
mode. In write mode currently only the rank can be modified, SION_CURRENT_BLK
and SION_CURRENT_POS are required.

If the sion-file has multiple physical files the function will also change the
current files pointer to the file which contains the data of the requested
rank. Therefore utilities which uses the serial interface of SIONlib to access
sion file should use this function to switch between different tasks of a sion
file and should pass the pointer to the POSIX file pointer also to this
function.

Parameters description:

  sid         : sion file handle                                 (input)
  rank        : rank number (0,...)                              (input)
                SION_CURRENT_RANK to select the current rank
  chunknum    : chunk number (0,...)                             (input)
                SION_CURRENT_BLK to select the current block
  posinchunk  : position (0,...)                                 (input)
                SION_CURRENT_POS to select the current position

Return Code:


  Return Code: 1 if file pointer can be moved to new position
               0 otherwise

[go_db] top of page

---> wrapper function for fwrite, ensures also free space interrnally:
sion_fwrite()

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  int sion_fwrite(  const void *data,
                    size_t      size,
                    size_t      n_items,
                    int         sid );

Description:
is a wrapper function for C-ANSI function fwrite. Internally this function
checks with sion_ensure_free_space if enough space is available.

Parameters description:

  data        : see fwrite documentation
  size        : see fwrite documentation
  nitems      : see fwrite documentation
  sid         : sion file handle                                 (input)

Return Code:


  Return Code: number of elements written (see fwrite documentation)


[go_db] top of page

---> wrapper function for fread: sion_fread()

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  int sion_fread(  const void *data,
                   size_t      size,
                   size_t      n_items,
                   int         sid );

Description:
is a wrapper function for C-ANSI function fread. Internally this function reads
in a while loop until all data is read from file. Reading more data than stored
in one chunk is with this wrapper possible.

Parameters description:

  data        : see fread documentation
  size        : see fread documentation
  nitems      : see fread documentation
  sid         : sion file handle                                 (input)

Return Code:


  Return Code: number of elements read (see fread documentation)


[go_db] top of page

