name: SofParser

synopsis:

 
g++ [flags ...] file ... -l /isip/tools/lib/$ISIP_BINARY/lib_io.a

#include <SofParser.h>

SofParser();
boolean setImplicitParam();
boolean setNest();
long countTokens(const SysString& param);

quick start:

SofParser parser;
static String implicit((unichar*)SofParser::IMPLICIT_PARAM);

// interface for data-only object
//
parser.setImplicitParam();
 
SysString pname(L"values");
pname.assign(implicit);

parser.setNest();

description:

The file parsing routines used to extract information from user-formatted parameter files are far from new technology. the tendency in software design, however, is to re-invent these methods for every new piece of software. this creates a two-fold problem. first, a significant amount of time is wasted in redeveloping existing technology. secondly, different implementors may choose to develop languages with varying complexity, so file conventions accepted by one program may not work with another.

SofParser is a parser class for parsing name=value from ASCII Sof files. it sits atop Sof in our environment.

dependencies:

public constants:

define the class name

static const SysString CLASS_NAME = L"SofParser";

define the buffer size used for the preparser

static const long MAX_BUFFER_SIZE = SysString::MAX_LENGTH;

define implicit parameter constants

static const long IMPLICIT_NAME = SofSymbolTable::NO_SYMB - 1;

static const SysString IMPLICIT_PARAM = L"__SOF_PARSER_IMPLICIT_PARAM";

define states to control the state machine

GPI stands for good previous input; LWS stands for last character whitespace

enum STATE { NO_GPI = 0, COMMENT_OP_NO_GPI, LITERAL_OP, GPI,
    QUOTE_OP, LITERAL_OP_IN_QUOTE, GPI_LWS, COMMENT_OP_GPI, TERMINAL_TRIM_LWS,
    BLOCK_GPI, BLOCK_GPI_LWS,DEF_STATE = NO_GPI };

static const SysString STATE_NO_GPI = L"no_gpi";

static const SysString STATE_COMMENT_OP_NO_GPI = L"comment_op_no_gpi";

static const SysString STATE_LITERAL_OP = L"literal_op";

static const SysString STATE_GPI = L"gpi";

static const SysString STATE_QUOTE_OP = L"quote_op";

static const SysString STATE_LITERAL_OP_IN_QUOTE = L"literal_op_in_quote";

static const SysString STATE_GPI_LWS = L"gpi_lws";

static const SysString STATE_COMMENT_OP_GPI = L"comment_op_gpi";

static const SysString STATE_TERMINAL_TRIM_LWS = L"terminal_trim_lws";

static const SysString STATE_BLOCK_GPI = L"block_gpi";

static const SysString STATE_BLOCK_GPI_LWS = L"block_gpi_lws";

define characters that have special meaning to the state machine

static const unichar LITERAL_CHAR = L'\\'';

static const unichar QUOTE_CHAR = L'"';

static const unichar NEWLINE_CHAR = L'\n';

static const unichar NULL_CHAR = L'\0';

static const unichar SPACE_CHAR = L' ';

define the default values of the class data

static const unichar DEF_TERMINATOR_CHAR = L';';

static const unichar DEF_ASSIGNMENT_CHAR = L'=';

static const unichar DEF_COMMENT_CHAR = L'\'';

static const unichar DEF_BLOCKSTART_CHAR = L'{';

static const unichar DEF_BLOCKSTOP_CHAR = L'}';

static const unichar DEF_DELIMITER_CHAR = L',';

define the default arguments to methods

static const long NO_SYMB = -1;

static const long ALL_TOKENS = -1;

static const long FULL_OBJECT = -1;

static const long BASE_TAG = 0;

error codes:

error code indicating SofParser class general error
```
static const long ERR = 10300;
```
error code indicating attempt to parse empty object
```
static const long ERR_PARSE = 10302;
```
error code indicating error in parsing statement
```
static const long ERR_LVALUE = 10303;
```
error code indicating error in parsing left value
```
static const long ERR_RVALUE = 10304;
```
error code indicating error in parsing right value
```
static const long ERR_RANGE = 10305;
```

protected data:

storage for the parameters and values:
```
SofSymbolTable table_d;
```
```
SofList index_d;
```
keep track of what parameters are accessed:
```
boolean* used_d;
```
```
long used_size_d;
```

special characters:

SysChar terminator_char_d;

SysChar assignment_char_d;

SysChar comment_char_d;

SysChar blockstart_char_d;

SysChar blockstop_char_d;

SysChar delimiter_char_d;

states and variables in the preParse FSM:

STATE state_d;

long param_d;

long block_count_d;

long token_count_d;

long token_start_d;

long token_stop_d;

positions within the current statement:

long statement_asgn_d;

long statement_term_d;

long statement_last_token_d;

the limits of our file space:
```
long base_position_d;
```
```
long full_size_d;
```

a few flags for the parser:

boolean open_index_d;

boolean implicit_object_d;

boolean implicit_block_d;

boolean nest_d;

boolean is_token_d;

boolean load_error_d;

debugging parameters:
```
Integral::DEBUG debug_level_d;
```

required public methods:

static methods:

static const SysString& name();

static boolean diagnose(Integral::DEBUG debug_level);

debug methods:

boolean debug(const unichar* msg) const;

boolean setDebug(Integral::DEBUG debug_level);

destructor/constructor(s):

~SofParser();

SofParser();

SofParser(const SofParser& arg);

assign methods:

these methods are omitted because they are not useful for SofParser objects

operator= methods:

these methods are omitted because they are not useful for SofParser objects

i/o methods:

these methods are omitted because SofParser can not write itself to an sof file

equality methods:

 these methods are omitted because they are not useful for SofParser objects

memory management methods:

new and delete are omitted because memory for SofParser objects
is not managed by the MemoryManager class. the clear method is omitted so the
user is forced to use the destructor for handling the SofParser

class-specific public methods:

constructors:

SofParser(const SysChar terminator, const SysChar assignment, const SysChar comment, const SysChar block_start, const SysChar block_stop, const SysChar delim);

memory size methods:
```
long memSize();
```
load methods: parse through and build index from current file space
```
boolean load(Sof& sof_a, long size = FULL_OBJECT);
```

read methods: read in text from the file

boolean read(SysString& value, Sof& sof, long size = FULL_OBJECT);

getEntry methods: position the Sof file and return the size of the parameter, the second method allows partial input/output

long getEntry(Sof& sof, const SysString& param, long tag = BASE_TAG);

long getEntry(Sof& sof, const SysString& param, long token_offset, long num_tokens);

parameter names interfacing methods

long countParams();

boolean checkParams(const SysString* param_list, long num);

boolean checkParams(Sof& sof);

long getParams(SysString* param_list, long max_num);

data-only objects interfacing methods

static const SysString& implicitPname();

boolean setImplicitParam();

boolean setNest();

parameter checking methods

boolean isPresent(const SysString& param_name);

long countTokens(const SysString& param);

reset method: used between calls
```
boolean reset();
```

private methods:

preParse methods: this is an FSM which cleans the format of the input text for easier parsing. it destructively changes the input buffer.
```
boolean preParse(long& olen, unichar* buffer, long ilen, long offset);
```
assignName methods: assigns the parameter at the current position in the state machine
```
boolean assignName(const unichar* buffer, long clen);
```

get state name methods

boolean setString(SysString& str, STATE state) const;

reportIndexError(long param, long tag, long pos);

examples:

this is a simple example using SofParser for scalar class. First the file format is checked and then the sof file object is loaded. The data is read from the default parameter file. The amount of data read depends upon the file format.

       SofParser parser;
       String buffer;
       SysString pname(L"value");
       String cname(L"Buffer");
       
       // create an sof file
       //
       Sof sof0;
       String tmp_file;
       Integral::makeTemp(tmp_file);
       
       // open the file in write mode
       //
       sof0.open(tmp_file, File::WRITE_ONLY);
       
       // add an entry to sof file
       //
       sof0.put(cname, 0, Sof::ANY_SIZE);
       String data(L"value = some text;\n");
       sof0.puts(data);
       
       sof0.close();
       
       // open the file in read mode
       //
       sof0.open(tmp_file);
       
       // find object in index and make this current
       //
       sof0.find(cname, 0);
       
       // load the parser
       //
       parser.load(sof0);
       
       // read the elements
       //
       if (!parser.read(buffer, sof0, parser.getEntry(sof0, pname))) {
         return false;
       }
       
       // close the sof file
       //
       sof0.close();
       buffer.debug(L"read this buffer");

notes:

SofParser also supports the string parsing routines typical for parameter settings, including tokenization, quoting, and literal character translation. While the user has some control of special characters, the finite state transducer is hard-coded for a pre-defined assignment language format.