name: Sof
synopsis:

>     g++ [flags ...] file ... -l /isip/tools/lib/$ISIP_BINARY/lib_io.a

>     #include <Sof.h>

>     Sof();
>     boolean open(const SysString& filename, File::MODE mode = File::READ_ONLY,
>     	       File::TYPE type = File::DEF_TYPE);
>     boolean put(const SysString& name, long size);
>     long write(const void* ptr, long size, long nitems);

quick start:

>     long j = 27;
>     Sof sof1;
>     SysString file1(L"temp.out");
>     if (sof1.open(file1, File::WRITE_ONLY, File::BINARY) != true) {
>      return Error::handle(file1, L"open", Error::TEST, __FILE__, __LINE__);
>     }
>     SysString cname(L"MyLong");
>     sof1.put(cname, 0, sizeof(long));
>     sof1.write(&j, sizeof(j), 1);
>     sof1.close();

description:
Sof is used by all data objects in the ISIP environment to unify and simplify I/O. An index is maintained pointing to all objects within the file, but all data is left on the disk until needed. Sof deals with both ASCII and binary files. Binary files inherently handle machine architecture differences with automatic byte transformations. User input parameter files in the ISIP environment are ASCII Sof files, but binary mode is typically used for large quantities of data for the obvious efficiency gains. Since Sof is at the base of all ISIP software, it has to be very efficient. To accomplish efficiency, the objects are stored in a binary tree and a symbol table is used to hold the object class names.
dependencies:

- File
- SofSymbolTable
- SofList
- Checksum

public constants:

- define the class name:

      static const SysString CLASS_NAME = L"Sof";

- define constants for file parsing:

      static const long MAX_MAGIC_LENGTH = 22;

- define the version number:

      static const SysString VERSION = L"v1.0";

- define message size and class name size:

      static const long MESSAGE_SIZE = 1024;

      static const long CLASS_NAME_SIZE = 256;

- define deletion characters:

      static const char DELETE_CHAR_TEXT = ' ';

      static const char DELETE_CHAR_BINARY = '\0';

- define new line characters:

      static const char NEWLINE_BUFFER[] = '\n';

      static const long NEWLINE_LENGTH = strlen(NEWLINE_BUFFER);

- define descriptor string related to these byte modes:

      static const SysString BSTR_NATIVE = L"abcd";

      static const SysString BSTR_SWAP = L"dcba";

- define the numbers that these string should convert to:

      static const long BCODE_NATIVE = 1633837924;

      static const long BCODE_SWAP = 1684234849;

- define the length of the byte-string for testing:

      static const long BSTR_LEN = 4;

- define constants to determine if the machine supports IEEE floating point:

      static const float IEEE_FLOAT_NATIVE = 16777999408082104352768.000000;

      static const float IEEE_FLOAT_SWAP = 261007875622861537280.000000;

- define constants for the implicit tag number regions:
  (if you go over 1.07 billion implicitly numbered tags, it will error)

      static const long NO_TAG = SofList::NO_TAG;

      static const long ALL_TAGS = NO_TAG + 1;

      static const long FREE_TAG = NO_TAG + 2;

      static const long IMPLICIT_END = -1073741824;

      static const long IMPLICIT_START = (NO_TAG + 3);

- define the buffer size to be used:
  when the sof binary index is written, it is aligned at this buffer size to expedite i/o.

      static const long BUFFER_SIZE = FILE::BUF_SIZE;

- define positions in the sequentially output symbol table:

      static const long SYMBOL_BASE = (2 * sizeof(uint32) + sizeof(uint16) + sizeof(ubyte));

      static const long SYMBOL_OFFSET_NEXT = 0;

      static const long SYMBOL_OFFSET_CKSM = SYMBOL_OFFSET_NEXT + sizeof(byte);

      static const long SYMBOL_OFFSET_IND = SYMBOL_OFFSET_CKSM + sizeof(uint16);

      static const long SYMBOL_OFFSET_REFS = SYMBOL_OFFSET_IND + sizeof(int32);

      static const long SYMBOL_OFFSET_NAME = SYMBOL_OFFSET_REFS + sizeof(int32);

- define positions in the sequentially output binary index:

      static const long INDEX_SIZE = (4 * sizeof(long) + sizeof(uint16) + sizeof(byte));

      static const long INDEX_OFFSET_NEXT = 0;

      static const long INDEX_OFFSET_CKSM = INDEX_OFFSET_NEXT + sizeof(byte);

      static const long INDEX_OFFSET_POS = INDEX_OFFSET_CKSM + sizeof(uint16);

      static const long INDEX_OFFSET_TAG = INDEX_OFFSET_POS + sizeof(int32);

      static const long INDEX_OFFSET_SIZE = INDEX_OFFSET_TAG + sizeof(int32);

      static const long INDEX_OFFSET_NAME = INDEX_OFFSET_SIZE + sizeof(int32);

- define a constant for the skip table size (for partial i/o):

      static const long SKIP_TABLE_GROUP = 10;

- define the default value(s) of the class data:

      static const unichar DEF_DELIMITER = L'#';

      static const long DEF_POS = -1;

      static const long DEF_IMPLICIT = IMPLICIT_START;

      static const boolean DEF_DISABLE_LOCK = true;

- define default arguments to methods:

      static const long ANY_SIZE = -1;

error codes:

- error code indicating Sof class general error:

      static const long ERR = 10000;

- error code indicating that the current file is not an Sof file:

      static const long ERR_NOTSOF = 10001;

- error code indicating object not found:

      static const long ERR_NOOBJ = 10002;

- error code indicating tag not found:

      static const long ERR_NOTAG = 10003;

- error code indicating error updating Sof headers:

      static const long ERR_UPDATEH = 10011;

- error code indicating error updating Sof file structure:

      static const long ERR_UPDATE = 10012;

- error code indicating error closing file:

      static const long ERR_CLOSE = 10015;

- error code indicating out of object's range:

      static const long ERR_RANGE = 10020;

- error code indicating no current position set to any object:

      static const long ERR_CURPOS = 10021;

- error code indicating IEEE floating point standard not supported on architecture:

      static const long ERR_FLOAT = 10042;

- error code indicating failure to read Sof header:

      static const long ERR_READHD = 10043;

- error code indicating file type mismatch:

      static const long ERR_TYPE = 10044;

- error code indicating invalid data in Sof file structure:

      static const long ERR_STRUCT = 10045;

- error code indicating failure to write ASCII label to Sof file:

      static const long ERR_LABEL = 10051;

- error code indicating too many implicitly tagged objects in file:

      static const long ERR_IMPLIC = 10061;

- error code indicating attempt to use a reserved Sof tag:

      static const long ERR_TAG = 10062;

- error code indicating no implicitly numbered tags allowed in binary files:

      static const long ERR_BINARY = 10063;

- error code indicating an attempt to switch objects in partial write mode:

      static const long ERR_PARTIAL = 10070;

- error code indicating an attempt to resize while not in partial write mode

      static const long ERR_RESIZE = 10071;

protected data:

- general information about a file:
  - define current filename

        SysString name_d;

  - define expanded filename

        SysString expanded_name_d;

  - define type of file (text or binary)

        File::TYPE file_type_d;

- I/O related variables:
  - define active file pointer

        File fp_d;

- auxiliary information:
  - define object tag delimiter

        unichar delimiter_d;

  - define end of header, start of data

        long end_of_header_d;

  - define end of data, start of index

        long end_of_data_d;

  - define start of current object data

        long cur_data_d;

  - define current file position (from start of file)

        long cur_pos_d;

  - define next free implicit tag number

        long implicit_count_d;

  - define file locking

        boolean disable_lock_d;

        long lock_retry_d;

        long lock_delay_d;

  - define symbol table used for names

        SofSymbolTable table_d;

  - define list used to process the object index

        SofList index_d;

- scratch space for magic number manipulations:
  - define sof file tag

        SysString cname_d;

  - define sof version

        SysString version_d;

  - define sof magic number

        SysString magic_d;

- the checksum read from the file header:

      ushort file_checksum_d;

- variables used to facilitate partial i/o:

      boolen partial_write_d;

      boolen partial_read_d;

      SofParser* vec_parser_d;

      int32* skip_table_d

      long skip_table_incr_d;

      long last_skip_table_pos_d;

      long vec_start_pos_d;

      long vec_size_d;

      long vec_curr_elem_d;

- debug parameters:

      Integral::DEBUG debug_level_d;

- memory manager:

      static MemoryManager mgr_d;

- checksum object for binary files:

      static Checksum cksm_d;

required public methods:

- static methods:

      static const SysString& name();

      static boolean diagnose(Integral::DEBUG debug_level);

- debug methods:

      boolean setDebug(Integral::DEBUG debug_level);

      boolean debug(const unichar* msg) const;

- destructor/constructor(s):

      ~Sof();

      Sof();

- assign methods:

      boolean assign(const Sof& arg);

- operator= methods:

      Sof& operator=(const Sof& arg);

- i/o methods:

      these methods are omitted because Sof object cannot write itself
      to an Sof file

- equality methods:

      these methods are omitted because they are not useful for Sof objects

- memory management methods: the clear method is class-specific as it is different from the required clear method

      static void* operator new(size_t size);

      static void* operator new[](size_t size);

      static void operator delete(void* ptr);

      static void operator delete[](void* ptr);

      static boolean setGrowSize(long grow_size);

      boolean clear(Integral::CMODE ctype_a = Integral::DEF_CMODE);

class-specific public methods:

- memory size methods:

      long memSize();

- file operations: open/close methods

      boolean open(const unichar* filename, File::MODE mode = File::READ_ONLY, File::TYPE type = File::DEF_TYPE);

      boolean open(const SysString& filename, File::MODE mode = File::READ_ONLY, File::TYPE type = File::DEF_TYPE);

      boolean close();

      static boolean closeAll();

- file operations: set methods

      boolean setLockDisable(boolean disable);

      boolean setLockRetry(long retry, long delay = File::DEF_LOCK_DELAY);

      boolean setDelimiter(unichar chr);

- file operations: get methods

      unichar getDelimiter() const;

      const SysString& getName() const;

      const SysString& getExpandedName() const;

- file operations: file queries

      boolean isSof();

      static boolean isSof(const SysString& filename);

      boolean isText() const;

      boolean isBinary() const;

      boolean isOpen() const;

      boolean isSameFile(Sof& sof_object);

- object operations: search methods

      boolean find(const SysString& name, long tag);

      long first(const SysString& name);

      long last(const SysString& name);

      long next(const SysString& name, long cur_tag);

      long prev(const SysString& name, long cur_tag);

- object operations: put methods

      boolean put(const SysString& name, long size);

      boolean put(const SysString& name, long tag, long size);

- object operations: remove methods

      boolean remove(const SysString& name, long object_tag);

      boolean remove(long name, long object_tag);

      boolean remove(const SysString& name);

      boolean remove(long name);

- object operations: copy methods

      boolean copy(long o_tag, Sof& i_sof, const SysString& i_cname, long i_tag);

      boolean copy(long o_tag, Sof& i_sof, long i_cname, long i_tag);

      boolean copy(Sof& i_sof, const SysString& i_cname);

      boolean copy(Sof& i_sof, long i_cname);

- object operations: get methods

      long getObjectSize(const SysString& name, long tag);

      long getObjectSize();

- index operations: count methods

      long getCount(const SysString& name);

      long getCount();

      long getNameCount();

- enumerate object methods:

      boolean enumerate(SofList& index, SofSymbolTable& table) const;

- positioning methods: (relative to current object)

      boolean seek(long offset, File::SEEK whence);

      long tell() const;

      boolean rewind();

- binary i/o methods:

      long read(void* ptr, long size, long nitems);

      long write(const void* ptr, long size, long nitems);

- text i/o methods:

      boolean gets(SysString& ptr, long size = BUFFER_SIZE);

      boolean puts(const SysString& ptr);

      boolean writeLabelPrefix(const SysString& param_name);

      boolean writeLabelSuffix(const SysString& param_name);

- indention control and line wrapping methods: (for text files)

      boolean increaseIndention();

      boolean decreaseIndention();

      boolean setLineWrap(long ncols);

      long getLineLength() const;

      long getLineWrap() const;

- methods to start and stop partial i/o mode:

      boolean startPartialWrite();

      boolean stopPartialWrite();

      boolean startPartialRead();

      boolean stopPartialRead();

- get methods for partial i/o variables:

      boolean getPartialWrite() const;

      boolean getPartialRead() const;

      int32* getSkipTable();

      const int32* getSkipTable() const;

      long getSkipTableIncr() const;

      long getSkipTablePos() const;

      long getStartPos() const;

      long getVecSize() const;

      long getVecCurrentElement() const;

      const SofParser& getVecParser() const;

      SofParser& getVecParser();

- set methods for partial i/o variables:

      boolean setSkipTableIncr(long pos);

      boolean setSkipTablePos(long pos);

      boolean setStartPos(long pos);

      boolean setVecSize(long size);

      boolean setVecCurrentElement(long elem);

- support methods for partial i/o:

      boolean clearSkipTable();

      boolean writeSkipTable();

      boolean readSkipTable();

- resize entry methods (only be called in partial write mode)

      boolean resize(long size);

private methods:

- copy constructor methods:

      Sof(const Sof& arg);

- free memory methods:

      boolean cleanUp();

      boolean freeIndex();

- update methods: update the header and index

      boolean update();

- binary index i/o methods

      boolean readIndex();

      boolean writeIndex();

- symbol table i/o methods

      boolean readTable()

      boolean writeTable();

- open read methods: open an sof file for read access

      boolean openRead();

      boolean openReadText();

      boolean openReadBinary();

- open read methods: open sof file for reading the index

      boolean openReadIndexText();

      boolean openReadIndexBinary();

- open write methods: open a file for write access

      boolean openWrite(File::MODE access_mode);

- text file label handling methods

      boolean writeLabel(long name, long tag);

      boolean writeLabel(const SysString& name, long tag);

      boolean parseLabel(SysString& name, long& tag, const SysString& buf);

      boolean skipLabel();

- seek data methods: seek to the data of current object

      boolean seekData();

      boolean seekDataText();

      boolean seekDataBinary();

- internal objects manipulation methods

      boolean find(long name, long tag);

      boolean add(const SysString& name, long size, long tag = FREE_TAG);

      boolean add(long name, long size, long tag = FREE_TAG);

- resize methods: resize the current entry, preserving current data. the method later should only be called in partial write mode.

      boolean resize(const SysString& name, long tag, long size);

- binary i/o methods: read/write with byte swapping without range checking

      long fwrite(const void* ptr, long size, long nitems);

      long fread(void* ptr, long size, long nitems);

- position methods:

      boolean fseek(long offset, File::SEEK whence);

      long ftell();

- copy segment methods: efficient methods for clearing out a section of a file and copying a section of a file from one place to another.

      boolean clearSpace(long nbytes);

      boolean copySegment(long src_pos, long size);

      boolean copySegment(long src_pos, long dest_pos, long size);

- error pointer setting methods:

      boolean setErrorPointer();

- register pointer maintaining methods: maintain an array of all open Sof pointers (part of clean exit)

      boolean registerPtr();

      boolean unRegisterPtr();

friend classes:

    friend class AudioFile;

examples:

- This example shows declaring Sof file objects and then creating temporary file names. The next step involves writing data into these files in text and binary format and reading some of the data that was written.

        Sof sof0;
        Sof sof1;
        SysString file1;
        SysString file2;
        SysString long_cname(L"Long");
        file1.assign(L"/tmp/foo_bin.sof");
        file2.assign(L"/tmp/foo_asc.sof");

        // open files for write (text & binary)
        //
        if (!sof0.open(file1, File::WRITE_ONLY)) {
          return Error::handle(long_cname, L"open", Error::TEST, __FILE__, __LINE__);
        }
        if (!sof1.open(file2, File::WRITE_ONLY, File::BINARY)) {
          return Error::handle(long_cname, L"open", Error::TEST, __FILE__, __LINE__);
        }

        // add a long entry to the files
        //
        long j = 27;
        sof0.put(long_cname, Sof::ANY_SIZE);
        sof1.put(long_cname, 0, sizeof(long));

        // write the text value
        //
        SysString output;
        output.assign((long)j, (unichar*)("value = %ld\n"));
        sof0.puts(output);

        // write the binary value
        //
        sof1.write(&j, sizeof(j), 1);

        // add another long entry to the files
        //
        j = 277;
        sof0.put(long_cname, Sof::ANY_SIZE);
        sof1.put(long_cname, 1, sizeof(long));

        // close the files
        //
        sof0.close();
        sof1.close();

        // open files again in read plus
        //
        sof0.open(file1, File::READ_PLUS);

        // read all the text values
        //
        long tag;
        for (tag = sof0.first(long_cname); tag != Sof::NO_TAG;
             tag = sof0.next(long_cname, tag)) {

          // find the node
          //
          sof0.find(long_cname, tag);
         }
        sof0.close();

notes:

- The preliminary implementation of Sof does not balance trees or compact the indices to the symbol table, but these changes could be easily added to the code. Additionally, the tree iteration routines could be algorithmically optimized. Even with these planned enhancements, a designer need not worry about lost effort. All Sof file formats henceforth will be backwards compatible. Note that multiple objects of the same class name can be written to a file.
- Note that a 16 bit checksum is used for each symbol table entry and index entry in binary mode.