Sion File Descriptor Structure. More...

#include <sion_filedesc.h>

Collaboration diagram for _sion_filedesc_struct:

Data Fields
_sion_fileptr *	fileptr

void *	dataptr

void *	keyvalptr

sion_int32	debug

sion_int32	sid

char *	fpbuffer

sion_int32	fpbuffer_size

sion_int32	usebuffer

char *	buffer

sion_int32	buffer_size

sion_int32	buffer_ptr

sion_int32	rank

sion_int64	currentpos

sion_int32	currentblocknr

sion_int64 *	blocksizes

sion_int32	lastchunknr

sion_int64	startpos

char *	fname

sion_int32	globalrank

sion_int32	ntasks

sion_int32	state

sion_int32	mode

sion_int32	endianness

sion_int32	swapbytes

sion_int32	filesionversion

sion_int32	filesionpatchlevel

sion_int32	fileversion

sion_int32	fsblksize

sion_int32	maxchunks

sion_int64	flag1

sion_int64	flag2

sion_int64	chunksize

sion_int64	chunksize_req

sion_int64	globalskip

sion_int64	end_of_header

sion_int64	start_of_varheader

sion_int64	start_of_data

sion_int32	nfiles

sion_int32	filenumber

sion_int32	maxusedchunks

sion_int32	dup_mode

sion_int32	dup_sel_rank

sion_uint64	dup_sel_key

sion_int32	lrank

sion_int32	ntotaltasksinfile

sion_int32	nlocaltasksinfile

sion_int32	filemanagedbytask

sion_int64 *	all_chunksizes

sion_int64 *	all_globalranks

sion_int64 *	all_localranks

sion_int64 *	all_startpointers

sion_int64 *	all_currentpos

sion_int64 *	all_currentblocknr

sion_int32 *	all_coll_collector

sion_int32 *	all_coll_collsize

sion_int32 *	all_coll_capability

void **	all_keyvalptr

sion_int64 *	all_blockcount

sion_int64 *	all_blocksizes

sion_int32	mapping_size

sion_int32 *	mapping

char *	prefix

_sion_filedesc **	multifiles

sion_int32	compress

sion_int32	keyvalmode

sion_int32	usecache

sion_int32	cachesize

sion_int32	cacheid

char	cachefn [SION_CACHE_FNLEN]

sion_int32	cachemode

char *	cacheptr

sion_int32	usecoll

sion_int32	collsize

sion_int32	collector

sion_int32	coll_capability

sion_int32	colldebug

sion_int32	collcmdused

sion_int32	fileptr_exported

sion_int32	collmergemode

sion_int32	collmsa

sion_int32	usebuddy

sion_int32	buddylevel

sion_int32	buddynr

void *	buddies

sion_int32	usehints

sion_int32	hinttype

Detailed Description

Sion File Descriptor Structure.

The sion file descriptor structure stores all the properties for the sion file. This properties are the META data of a sion file

DO NOT change types of elements, becauses these are directly used to read/write header

CHUNK: reserved space for each rank, each rank can have serveral chunks in a file

BLOCK: used space in a CHUNK

Definition at line 79 of file sion_filedesc.h.

Field Documentation

◆ all_blockcount

sion_int64* _sion_filedesc_struct::all_blockcount

size, only used if sion is opened from a single task for read

Definition at line 157 of file sion_filedesc.h.

◆ all_blocksizes

sion_int64* _sion_filedesc_struct::all_blocksizes

size*maxblocks, only used if sion is opened from a single task for read

Definition at line 158 of file sion_filedesc.h.

◆ all_chunksizes

sion_int64* _sion_filedesc_struct::all_chunksizes

list of all chunk sizes stored in this file

Definition at line 144 of file sion_filedesc.h.

◆ all_coll_capability

sion_int32* _sion_filedesc_struct::all_coll_capability

capability of task (FULL, ONLYSENDER)

Definition at line 152 of file sion_filedesc.h.

◆ all_coll_collector

sion_int32* _sion_filedesc_struct::all_coll_collector

only used on rank 0 if usecoll=1

Definition at line 150 of file sion_filedesc.h.

◆ all_coll_collsize

sion_int32* _sion_filedesc_struct::all_coll_collsize

only used on rank 0 if usecoll=1

Definition at line 151 of file sion_filedesc.h.

◆ all_currentblocknr

sion_int64* _sion_filedesc_struct::all_currentblocknr

only used if sion is opened from a single task for read

Definition at line 149 of file sion_filedesc.h.

◆ all_currentpos

sion_int64* _sion_filedesc_struct::all_currentpos

only used if sion is opened from a single task for read

Definition at line 148 of file sion_filedesc.h.

◆ all_globalranks

sion_int64* _sion_filedesc_struct::all_globalranks

list of all globalranks stored in this file

Definition at line 145 of file sion_filedesc.h.

◆ all_keyvalptr

void** _sion_filedesc_struct::all_keyvalptr

keyval data structure for each task

Definition at line 154 of file sion_filedesc.h.

◆ all_localranks

sion_int64* _sion_filedesc_struct::all_localranks

list of all localranks stored in this file

Definition at line 146 of file sion_filedesc.h.

◆ all_startpointers

sion_int64* _sion_filedesc_struct::all_startpointers

list of all start pointers stored in this file

Definition at line 147 of file sion_filedesc.h.

◆ blocksizes

sion_int64* _sion_filedesc_struct::blocksizes

Array of the blocksizes in all chunks of the current task

Definition at line 100 of file sion_filedesc.h.

◆ buddies

void* _sion_filedesc_struct::buddies

pointer to payload container structure describing buddies

Definition at line 195 of file sion_filedesc.h.

◆ buddylevel

sion_int32 _sion_filedesc_struct::buddylevel

buddy level (number of copies)

Definition at line 193 of file sion_filedesc.h.

◆ buddynr

sion_int32 _sion_filedesc_struct::buddynr

number of buddy file, 0 for original file

Definition at line 194 of file sion_filedesc.h.

◆ buffer

char* _sion_filedesc_struct::buffer

buffer for internal buffering data in sion_fwrite/sion_fread

Definition at line 90 of file sion_filedesc.h.

Referenced by _sion_buffer_get_data_ptr().

◆ buffer_ptr

sion_int32 _sion_filedesc_struct::buffer_ptr

index of the first free byte in buffer

Definition at line 92 of file sion_filedesc.h.

Referenced by _sion_buffer_get_data_ptr().

◆ buffer_size

sion_int32 _sion_filedesc_struct::buffer_size

buffer size

Definition at line 91 of file sion_filedesc.h.

◆ cachefn

char _sion_filedesc_struct::cachefn[SION_CACHE_FNLEN]

cache id

Definition at line 176 of file sion_filedesc.h.

◆ cacheid

sion_int32 _sion_filedesc_struct::cacheid

cache id

Definition at line 175 of file sion_filedesc.h.

◆ cachemode

sion_int32 _sion_filedesc_struct::cachemode

cache mode (TOMEM,TODISK)

Definition at line 177 of file sion_filedesc.h.

◆ cacheptr

char* _sion_filedesc_struct::cacheptr

pointer to cache

Definition at line 178 of file sion_filedesc.h.

◆ cachesize

sion_int32 _sion_filedesc_struct::cachesize

cache size in number of bytes

Definition at line 174 of file sion_filedesc.h.

◆ chunksize

sion_int64 _sion_filedesc_struct::chunksize

chunksize for the current task(From Meta block 1)

Definition at line 122 of file sion_filedesc.h.

◆ chunksize_req

sion_int64 _sion_filedesc_struct::chunksize_req

Requested chunksize for the current task

Definition at line 123 of file sion_filedesc.h.

◆ coll_capability

sion_int32 _sion_filedesc_struct::coll_capability

capability of task (FULL, ONLYSENDER)

Definition at line 184 of file sion_filedesc.h.

◆ collcmdused

sion_int32 _sion_filedesc_struct::collcmdused

indicates if collective call was already performed

Definition at line 186 of file sion_filedesc.h.

◆ colldebug

sion_int32 _sion_filedesc_struct::colldebug

print debug info on stderr, i level of detail, 0->off

Definition at line 185 of file sion_filedesc.h.

◆ collector

sion_int32 _sion_filedesc_struct::collector

tasks number of corresponding collector

Definition at line 183 of file sion_filedesc.h.

◆ collmergemode

sion_int32 _sion_filedesc_struct::collmergemode

merge data on collector task

Definition at line 188 of file sion_filedesc.h.

◆ collmsa

sion_int32 _sion_filedesc_struct::collmsa

adapt collector groups to peculiarities of the Modular Supercomputer Architecture (MSA)

Definition at line 189 of file sion_filedesc.h.

◆ collsize

sion_int32 _sion_filedesc_struct::collsize

number of tasks working together (for this task)

Definition at line 182 of file sion_filedesc.h.

◆ compress

sion_int32 _sion_filedesc_struct::compress

is set to 1 if data should be/is compressed

Definition at line 167 of file sion_filedesc.h.

◆ currentblocknr

sion_int32 _sion_filedesc_struct::currentblocknr

only used if opened for one rank, reading

Definition at line 97 of file sion_filedesc.h.

◆ currentpos

sion_int64 _sion_filedesc_struct::currentpos

only used if opened for one rank, reading

Definition at line 96 of file sion_filedesc.h.

◆ dataptr

void* _sion_filedesc_struct::dataptr

pointer to data structure for upper layers

Definition at line 83 of file sion_filedesc.h.

◆ debug

sion_int32 _sion_filedesc_struct::debug

0 or 1 for debugging

Definition at line 85 of file sion_filedesc.h.

◆ dup_mode

sion_int32 _sion_filedesc_struct::dup_mode

dup mode; used to indicate if datastructure was duplicated from another one

Definition at line 133 of file sion_filedesc.h.

◆ dup_sel_key

sion_uint64 _sion_filedesc_struct::dup_sel_key

Used to indicate if datastructure was duplicated from another one

Definition at line 135 of file sion_filedesc.h.

◆ dup_sel_rank

sion_int32 _sion_filedesc_struct::dup_sel_rank

Used to indicate if datastructure was duplicated from another one

Definition at line 134 of file sion_filedesc.h.

◆ end_of_header

sion_int64 _sion_filedesc_struct::end_of_header

End of META block 1 (beginning of the sion file)

Definition at line 125 of file sion_filedesc.h.

◆ endianness

sion_int32 _sion_filedesc_struct::endianness

Endianness used. Written symmetric. First (and last) byte indicates meta data. Second (and second last) byte indicates user data. 0 = little, 1 = big

Definition at line 110 of file sion_filedesc.h.

◆ filemanagedbytask

sion_int32 _sion_filedesc_struct::filemanagedbytask

task number of managing task of this the sion file (mapped-mode)

Definition at line 141 of file sion_filedesc.h.

◆ filenumber

sion_int32 _sion_filedesc_struct::filenumber

file number in the set

Definition at line 129 of file sion_filedesc.h.

◆ fileptr

_sion_fileptr* _sion_filedesc_struct::fileptr

Pointer to the current sion file

Definition at line 82 of file sion_filedesc.h.

Referenced by fsion_get_fileno_c().

◆ fileptr_exported

sion_int32 _sion_filedesc_struct::fileptr_exported

indicates that file pointer is exported via paramater to application

Definition at line 187 of file sion_filedesc.h.

◆ filesionpatchlevel

sion_int32 _sion_filedesc_struct::filesionpatchlevel

patch-level of library which wrote sion file

Definition at line 116 of file sion_filedesc.h.

◆ filesionversion

sion_int32 _sion_filedesc_struct::filesionversion

Version number of library which wrote sion file (version*100+sub_version)

Definition at line 115 of file sion_filedesc.h.

◆ fileversion

sion_int32 _sion_filedesc_struct::fileversion

Version number of sion file format

Definition at line 117 of file sion_filedesc.h.

◆ flag1

sion_int64 _sion_filedesc_struct::flag1

keyval-mode since file format version 5

Definition at line 120 of file sion_filedesc.h.

◆ flag2

sion_int64 _sion_filedesc_struct::flag2

placeholder for future flags

Definition at line 121 of file sion_filedesc.h.

◆ fname

char* _sion_filedesc_struct::fname

Name of the file

Definition at line 105 of file sion_filedesc.h.

◆ fpbuffer

char* _sion_filedesc_struct::fpbuffer

buffer for optimizing ANSI C I/O function fwrite/fread

Definition at line 87 of file sion_filedesc.h.

◆ fpbuffer_size

sion_int32 _sion_filedesc_struct::fpbuffer_size

buffer size

Definition at line 88 of file sion_filedesc.h.

◆ fsblksize

sion_int32 _sion_filedesc_struct::fsblksize

Filesystem block size

Definition at line 118 of file sion_filedesc.h.

◆ globalrank

sion_int32 _sion_filedesc_struct::globalrank

Globalrank of the task given by parameter or implicitly

Definition at line 106 of file sion_filedesc.h.

◆ globalskip

sion_int64 _sion_filedesc_struct::globalskip

number of bytes to skip to next block of same rank

Definition at line 124 of file sion_filedesc.h.

◆ hinttype

sion_int32 _sion_filedesc_struct::hinttype

type of hints

Definition at line 199 of file sion_filedesc.h.

◆ keyvalmode

sion_int32 _sion_filedesc_struct::keyvalmode

describes if and in which mode keyval will be used

Definition at line 170 of file sion_filedesc.h.

◆ keyvalptr

void* _sion_filedesc_struct::keyvalptr

pointer to data structure for key-value

Definition at line 84 of file sion_filedesc.h.

◆ lastchunknr

sion_int32 _sion_filedesc_struct::lastchunknr

number of last chunk used on this task (0..n)

Definition at line 101 of file sion_filedesc.h.

◆ lrank

sion_int32 _sion_filedesc_struct::lrank

local rank, needed to omit re-computation of mapping

Definition at line 138 of file sion_filedesc.h.

◆ mapping

sion_int32* _sion_filedesc_struct::mapping

pointer to mapping table if reading multi file

Definition at line 162 of file sion_filedesc.h.

◆ mapping_size

sion_int32 _sion_filedesc_struct::mapping_size

size of mapping table, -1 if not available

Definition at line 161 of file sion_filedesc.h.

◆ maxchunks

sion_int32 _sion_filedesc_struct::maxchunks

Max number of chunks per task

Definition at line 119 of file sion_filedesc.h.

◆ maxusedchunks

sion_int32 _sion_filedesc_struct::maxusedchunks

Max number of chunks per task already used by the tasks

Definition at line 130 of file sion_filedesc.h.

◆ mode

sion_int32 _sion_filedesc_struct::mode

Mode used to open the file(rb/wb)

Definition at line 109 of file sion_filedesc.h.

◆ multifiles

_sion_filedesc** _sion_filedesc_struct::multifiles

pointer to field containing data structures of sion files of a sion multi-file

Definition at line 164 of file sion_filedesc.h.

◆ nfiles

sion_int32 _sion_filedesc_struct::nfiles

Number of files in the set

Definition at line 128 of file sion_filedesc.h.

◆ nlocaltasksinfile

sion_int32 _sion_filedesc_struct::nlocaltasksinfile

number of local tasks in this the sion file (mapped-mode)

Definition at line 140 of file sion_filedesc.h.

◆ ntasks

sion_int32 _sion_filedesc_struct::ntasks

Number of tasks using the sion file

Definition at line 107 of file sion_filedesc.h.

Referenced by _sion_alloc_filedesc_coll_arrays().

◆ ntotaltasksinfile

sion_int32 _sion_filedesc_struct::ntotaltasksinfile

total number of tasks in this the sion file (mapped-mode)

Definition at line 139 of file sion_filedesc.h.

◆ prefix

char* _sion_filedesc_struct::prefix

Filenames of all files in the set

Definition at line 163 of file sion_filedesc.h.

◆ rank

sion_int32 _sion_filedesc_struct::rank

Rank of the task writing the file (global rank)

Definition at line 95 of file sion_filedesc.h.

◆ sid

sion_int32 _sion_filedesc_struct::sid

sid of data structure

Definition at line 86 of file sion_filedesc.h.

◆ start_of_data

sion_int64 _sion_filedesc_struct::start_of_data

Start of the data in the sion file

Definition at line 127 of file sion_filedesc.h.

◆ start_of_varheader

sion_int64 _sion_filedesc_struct::start_of_varheader

Beginning of the META block 2 (end of the sion file)

Definition at line 126 of file sion_filedesc.h.

◆ startpos

sion_int64 _sion_filedesc_struct::startpos

Start position for the current task, start of first chunk (fix for task)

Definition at line 102 of file sion_filedesc.h.

◆ state

sion_int32 _sion_filedesc_struct::state

INTERNAL STATE

Definition at line 108 of file sion_filedesc.h.

◆ swapbytes

sion_int32 _sion_filedesc_struct::swapbytes

swapping bytes is needed for this sion file

Definition at line 114 of file sion_filedesc.h.

◆ usebuddy

sion_int32 _sion_filedesc_struct::usebuddy

is set to 1 if buddy mode is selected

Definition at line 192 of file sion_filedesc.h.

◆ usebuffer

sion_int32 _sion_filedesc_struct::usebuffer

is set to 1 if data is/should be buffered internally in sion_fwrite/sion_fread

Definition at line 89 of file sion_filedesc.h.

◆ usecache

sion_int32 _sion_filedesc_struct::usecache

is set to 1 if data is/should be cached in persistent memory

Definition at line 173 of file sion_filedesc.h.

◆ usecoll

sion_int32 _sion_filedesc_struct::usecoll

is set to 1 if collective operation should be used for I/O

Definition at line 181 of file sion_filedesc.h.

◆ usehints

sion_int32 _sion_filedesc_struct::usehints

is set to 1 if hints should be applied

Definition at line 198 of file sion_filedesc.h.

The documentation for this struct was generated from the following file:

src/lib/sion_filedesc.h

Data Fields