testing class LanguageModelXML
  diagnosing class LanguageModelXML: 
    testing required public methods...
      testing debug method...
      testing clear method...
      testing eq method...
      testing assign method...
    testing class-specific protected methods: i/o methods...
      Processing file: /cavs/hse/ies/d002/isip/users/wholland/new_tools/class/pr/LanguageModelXML/diagnose_input_combined.sof
        reading model
          reading levels
            reading grammars
              Parsing the grammar...
              The parsed document:
                <grammar root='word'>
                  <tag>
                    unimportant
                    junk
                  </tag>
                  <metadata>
                    author:
                    Dhruva
                  </metadata>
                  <rule id='word'>
                    <item weight='0.5'>
                      symbol0
                    </item>
                    symbol1
                    <one-of>
                      <item repeat='1-' repeat-prob='0.98'>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                      <item>
                        <ruleref uri='#rule3'/>
                      </item>
                    </one-of>
                    symbol1
                  </rule>
                  <rule id='rule2'>
                    <item weight='0.1'>
                      symbol2a
                    </item>
                    <item weight='0.2'>
                      symbol2b
                    </item>
                    <item weight='0.3'>
                      symbol2c
                    </item>
                  </rule>
                  <rule id='rule3'>
                    symbol6
                    <item>
                      <one-of>
                        <item>
                          symbol4
                        </item>
                        <item>
                          symbol5
                        </item>
                      </one-of>
                    </item>
                    symbol7
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='word'>
              |1     |2     |  <rule id='word'>
              |2     |3     |    <item weight='0.5'>
              |3     |3     |      symbol0
              |4     |2     |    </item>
              |5     |2     |    symbol1
              |6     |3     |    <one-of>
              |7     |4     |      <item repeat='1-' repeat-prob='0.98'>
              |8     |4     |        symbol1
              |9     |5     |        <one-of>
              |10    |6     |          <item>
              |11    |6     |            <ruleref uri='#rule2'/>
              |12    |5     |          </item>
              |13    |4     |        </one-of>
              |14    |4     |        symbol3
              |15    |3     |      </item>
              |16    |4     |      <item>
              |17    |4     |        <ruleref uri='#rule3'/>
              |18    |3     |      </item>
              |19    |2     |    </one-of>
              |20    |2     |    symbol1
              |21    |1     |  </rule>
              |22    |2     |  <rule id='rule2'>
              |23    |3     |    <item weight='0.1'>
              |24    |3     |      symbol2a
              |25    |2     |    </item>
              |26    |3     |    <item weight='0.2'>
              |27    |3     |      symbol2b
              |28    |2     |    </item>
              |29    |3     |    <item weight='0.3'>
              |30    |3     |      symbol2c
              |31    |2     |    </item>
              |32    |1     |  </rule>
              |33    |2     |  <rule id='rule3'>
              |34    |2     |    symbol6
              |35    |3     |    <item>
              |36    |4     |      <one-of>
              |37    |5     |        <item>
              |38    |5     |          symbol4
              |39    |4     |        </item>
              |40    |5     |        <item>
              |41    |5     |          symbol5
              |42    |4     |        </item>
              |43    |3     |      </one-of>
              |44    |2     |    </item>
              |45    |2     |    symbol7
              |46    |1     |  </rule>
              |47    |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='word'>
                  <rule id='word'>
                    <item weight='0.5'>
                      symbol0
                    </item>
                    symbol1
                    <one-of>
                      <item repeat='1-' repeat-prob='0.98'>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                      <item>
                        <ruleref uri='#rule3'/>
                      </item>
                    </one-of>
                    symbol1
                  </rule>
                  <rule id='rule2'>
                    <item weight='0.1'>
                      symbol2a
                    </item>
                    <item weight='0.2'>
                      symbol2b
                    </item>
                    <item weight='0.3'>
                      symbol2c
                    </item>
                  </rule>
                  <rule id='rule3'>
                    symbol6
                    <item>
                      <one-of>
                        <item>
                          symbol4
                        </item>
                        <item>
                          symbol5
                        </item>
                      </one-of>
                    </item>
                    symbol7
                  </rule>
                </grammar>
            reading level tag
              Parsing the grammar...
              The parsed document:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='level0'>
              |1     |2     |  <rule id='level0'>
              |2     |2     |    word
              |3     |1     |  </rule>
              |4     |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
            reading symbols
            reading context mapping
              <HashTable<Context,Ulong>::dummy context hash table:> num_items_d = 10
              <HashTable<Context,Ulong>::dummy context hash table:> load_factor_d = 0.75
              <HashTable<Context,Ulong>::dummy context hash table:> capacity_d = 128
              <HashTable<Context,Ulong>::dummy context hash table:> alloc_d = SYSTEM
                <CircularDelayLine<Ulong>::0> index_d = 0
                <CircularDelayLine<Ulong>::0> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 0
                  <Ulong::0> value_d = 0
                <CircularDelayLine<Ulong>::1> index_d = 0
                <CircularDelayLine<Ulong>::1> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 1
                  <Ulong::1> value_d = 1
                <CircularDelayLine<Ulong>::2> index_d = 0
                <CircularDelayLine<Ulong>::2> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 2
                  <Ulong::2> value_d = 2
                <CircularDelayLine<Ulong>::3> index_d = 0
                <CircularDelayLine<Ulong>::3> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 3
                  <Ulong::3> value_d = 3
                <CircularDelayLine<Ulong>::4> index_d = 0
                <CircularDelayLine<Ulong>::4> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 4
                  <Ulong::4> value_d = 4
                <CircularDelayLine<Ulong>::5> index_d = 0
                <CircularDelayLine<Ulong>::5> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 5
                  <Ulong::5> value_d = 5
                <CircularDelayLine<Ulong>::6> index_d = 0
                <CircularDelayLine<Ulong>::6> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 6
                  <Ulong::6> value_d = 6
                <CircularDelayLine<Ulong>::7> index_d = 0
                <CircularDelayLine<Ulong>::7> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 7
                  <Ulong::7> value_d = 7
                <CircularDelayLine<Ulong>::8> index_d = 0
                <CircularDelayLine<Ulong>::8> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 8
                  <Ulong::8> value_d = 8
                <CircularDelayLine<Ulong>::9> index_d = 0
                <CircularDelayLine<Ulong>::9> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 9
                  <Ulong::9> value_d = 9
        Level 0
          Graph 0
          <?xml version='1.0' encoding='utf-8'?>
          <grammar root='word'>
            <rule id='word'>
              <item weight='0.5'> symbol0 </item>
              symbol1
              <one-of>
                <item repeat='1-' repeat-prob='0.98'>
                  symbol1
                  <one-of>
                    <item>
                      <ruleref uri='#rule2'/>
                    </item>
                  </one-of>
                  symbol3
                </item>
                <item>
                  <ruleref uri='#rule3'/>
                </item>
              </one-of>
              symbol1
            </rule>
            <rule id='rule2'>
              <item weight='0.1'> symbol2a </item>
              <item weight='0.2'> symbol2b </item>
              <item weight='0.3'> symbol2c </item>
            </rule>
            <rule id='rule3'>
              symbol6
              <item>
                <one-of>
                  <item> symbol4 </item>
                  <item> symbol5 </item>
                </one-of>
              </item>
              symbol7
            </rule>
          </grammar>
          
        performing XML->ABNF conversion
          level 0
            graph 0
              preprocessing
                Handling repeat requests...
                The modified grammar:
                <rule id='word'>
                  <item weight='0.5'>
                    symbol0
                  </item>
                  symbol1
                  <one-of>
                    <item>
                      <item repeat-prob='0.98' repeat='1-'>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                    </item>
                    <item>
                      <ruleref uri='#rule3'/>
                    </item>
                  </one-of>
                  symbol1
                </rule>
                Handling repeat requests...
                The modified grammar:
                <rule id='rule2'>
                  <item weight='0.1'>
                    symbol2a
                  </item>
                  <item weight='0.2'>
                    symbol2b
                  </item>
                  <item weight='0.3'>
                    symbol2c
                  </item>
                </rule>
                Handling repeat requests...
                The modified grammar:
                <rule id='rule3'>
                  symbol6
                  <item>
                    <one-of>
                      <item>
                        symbol4
                      </item>
                      <item>
                        symbol5
                      </item>
                    </one-of>
                  </item>
                  symbol7
                </rule>
              performing conversion
              ABNF equivalent:
                word -> ((symbol0) symbol1 (((symbol1 (RULE2) symbol3)+)|(RULE3)) symbol1)
              ABNF equivalent:
                rule2 -> ((symbol2a) (symbol2b) (symbol2c))
              ABNF equivalent:
                rule3 -> (symbol6 (((symbol4)|(symbol5))) symbol7)
      Processing file: /cavs/hse/ies/d002/isip/users/wholland/new_tools/class/pr/LanguageModelXML/diagnose_input_grammar_empty.sof
        reading model
          reading levels
            reading grammars
              Parsing the grammar...
              The parsed document:
                <grammar root='word'/>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |0     |<grammar root='word'/>
              The document after removal of unwanted tags:
                <grammar root='word'/>
            reading level tag
              Parsing the grammar...
              The parsed document:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='level0'>
              |1     |2     |  <rule id='level0'>
              |2     |2     |    word
              |3     |1     |  </rule>
              |4     |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
            reading symbols
            reading context mapping
              <HashTable<Context,Ulong>::dummy context hash table:> num_items_d = 0
              <HashTable<Context,Ulong>::dummy context hash table:> load_factor_d = 0.75
              <HashTable<Context,Ulong>::dummy context hash table:> capacity_d = 128
              <HashTable<Context,Ulong>::dummy context hash table:> alloc_d = SYSTEM
        Level 0
        performing XML->ABNF conversion
          level 0
      Processing file: /cavs/hse/ies/d002/isip/users/wholland/new_tools/class/pr/LanguageModelXML/diagnose_input_nesting.sof
        reading model
          reading levels
            reading grammars
              Parsing the grammar...
              The parsed document:
                <grammar root='word'>
                  <rule id='word'>
                    <item>
                      symbol0
                    </item>
                    <item>
                      symbol1
                    </item>
                    <one-of>
                      <item>
                        <item>
                          symbol1
                        </item>
                        <one-of>
                          <item>
                            symbol2a
                          </item>
                          <item>
                            symbol2b
                          </item>
                          <item>
                            symbol2c
                          </item>
                        </one-of>
                        <item>
                          symbol3
                        </item>
                      </item>
                      <item>
                        <item>
                          symbol6
                        </item>
                        <item>
                          <one-of>
                            <item>
                              symbol4
                            </item>
                            <item>
                              symbol5
                            </item>
                          </one-of>
                        </item>
                        <item>
                          symbol7
                        </item>
                      </item>
                    </one-of>
                    <item>
                      symbol1
                    </item>
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='word'>
              |1     |2     |  <rule id='word'>
              |2     |3     |    <item>
              |3     |3     |      symbol0
              |4     |2     |    </item>
              |5     |3     |    <item>
              |6     |3     |      symbol1
              |7     |2     |    </item>
              |8     |3     |    <one-of>
              |9     |4     |      <item>
              |10    |5     |        <item>
              |11    |5     |          symbol1
              |12    |4     |        </item>
              |13    |5     |        <one-of>
              |14    |6     |          <item>
              |15    |6     |            symbol2a
              |16    |5     |          </item>
              |17    |6     |          <item>
              |18    |6     |            symbol2b
              |19    |5     |          </item>
              |20    |6     |          <item>
              |21    |6     |            symbol2c
              |22    |5     |          </item>
              |23    |4     |        </one-of>
              |24    |5     |        <item>
              |25    |5     |          symbol3
              |26    |4     |        </item>
              |27    |3     |      </item>
              |28    |4     |      <item>
              |29    |5     |        <item>
              |30    |5     |          symbol6
              |31    |4     |        </item>
              |32    |5     |        <item>
              |33    |6     |          <one-of>
              |34    |7     |            <item>
              |35    |7     |              symbol4
              |36    |6     |            </item>
              |37    |7     |            <item>
              |38    |7     |              symbol5
              |39    |6     |            </item>
              |40    |5     |          </one-of>
              |41    |4     |        </item>
              |42    |5     |        <item>
              |43    |5     |          symbol7
              |44    |4     |        </item>
              |45    |3     |      </item>
              |46    |2     |    </one-of>
              |47    |3     |    <item>
              |48    |3     |      symbol1
              |49    |2     |    </item>
              |50    |1     |  </rule>
              |51    |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='word'>
                  <rule id='word'>
                    <item>
                      symbol0
                    </item>
                    <item>
                      symbol1
                    </item>
                    <one-of>
                      <item>
                        <item>
                          symbol1
                        </item>
                        <one-of>
                          <item>
                            symbol2a
                          </item>
                          <item>
                            symbol2b
                          </item>
                          <item>
                            symbol2c
                          </item>
                        </one-of>
                        <item>
                          symbol3
                        </item>
                      </item>
                      <item>
                        <item>
                          symbol6
                        </item>
                        <item>
                          <one-of>
                            <item>
                              symbol4
                            </item>
                            <item>
                              symbol5
                            </item>
                          </one-of>
                        </item>
                        <item>
                          symbol7
                        </item>
                      </item>
                    </one-of>
                    <item>
                      symbol1
                    </item>
                  </rule>
                </grammar>
            reading level tag
              Parsing the grammar...
              The parsed document:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='level0'>
              |1     |2     |  <rule id='level0'>
              |2     |2     |    word
              |3     |1     |  </rule>
              |4     |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
            reading symbols
            reading context mapping
              <HashTable<Context,Ulong>::dummy context hash table:> num_items_d = 10
              <HashTable<Context,Ulong>::dummy context hash table:> load_factor_d = 0.75
              <HashTable<Context,Ulong>::dummy context hash table:> capacity_d = 128
              <HashTable<Context,Ulong>::dummy context hash table:> alloc_d = SYSTEM
                <CircularDelayLine<Ulong>::0> index_d = 0
                <CircularDelayLine<Ulong>::0> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 0
                  <Ulong::0> value_d = 0
                <CircularDelayLine<Ulong>::1> index_d = 0
                <CircularDelayLine<Ulong>::1> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 1
                  <Ulong::1> value_d = 1
                <CircularDelayLine<Ulong>::2> index_d = 0
                <CircularDelayLine<Ulong>::2> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 2
                  <Ulong::2> value_d = 2
                <CircularDelayLine<Ulong>::3> index_d = 0
                <CircularDelayLine<Ulong>::3> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 3
                  <Ulong::3> value_d = 3
                <CircularDelayLine<Ulong>::4> index_d = 0
                <CircularDelayLine<Ulong>::4> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 4
                  <Ulong::4> value_d = 4
                <CircularDelayLine<Ulong>::5> index_d = 0
                <CircularDelayLine<Ulong>::5> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 5
                  <Ulong::5> value_d = 5
                <CircularDelayLine<Ulong>::6> index_d = 0
                <CircularDelayLine<Ulong>::6> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 6
                  <Ulong::6> value_d = 6
                <CircularDelayLine<Ulong>::7> index_d = 0
                <CircularDelayLine<Ulong>::7> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 7
                  <Ulong::7> value_d = 7
                <CircularDelayLine<Ulong>::8> index_d = 0
                <CircularDelayLine<Ulong>::8> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 8
                  <Ulong::8> value_d = 8
                <CircularDelayLine<Ulong>::9> index_d = 0
                <CircularDelayLine<Ulong>::9> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 9
                  <Ulong::9> value_d = 9
        Level 0
          Graph 0
          <?xml version='1.0' encoding='utf-8'?>
          <grammar root='word'>
            <rule id='word'>
              <item> symbol0 </item>
              <item> symbol1 </item>
              <one-of>
                <item>
                  <item> symbol1 </item>
                  <one-of>
                    <item> symbol2a </item>
                    <item> symbol2b </item>
                    <item> symbol2c </item>
                  </one-of>
                  <item> symbol3 </item>
                </item>
                <item>
                  <item> symbol6 </item>
                  <item>
                    <one-of>
                      <item> symbol4 </item>
                      <item> symbol5 </item>
                    </one-of>
                  </item>
                  <item> symbol7 </item>
                </item>
              </one-of>
              <item> symbol1 </item>
            </rule>
          </grammar>
          
        performing XML->ABNF conversion
          level 0
            graph 0
              preprocessing
                Handling repeat requests...
                The modified grammar:
                <rule id='word'>
                  <item>
                    symbol0
                  </item>
                  <item>
                    symbol1
                  </item>
                  <one-of>
                    <item>
                      <item>
                        symbol1
                      </item>
                      <one-of>
                        <item>
                          symbol2a
                        </item>
                        <item>
                          symbol2b
                        </item>
                        <item>
                          symbol2c
                        </item>
                      </one-of>
                      <item>
                        symbol3
                      </item>
                    </item>
                    <item>
                      <item>
                        symbol6
                      </item>
                      <item>
                        <one-of>
                          <item>
                            symbol4
                          </item>
                          <item>
                            symbol5
                          </item>
                        </one-of>
                      </item>
                      <item>
                        symbol7
                      </item>
                    </item>
                  </one-of>
                  <item>
                    symbol1
                  </item>
                </rule>
              performing conversion
              ABNF equivalent:
                word -> ((symbol0) (symbol1) (((symbol1) ((symbol2a)|(symbol2b)|(symbol2c)|(symbol3)) (symbol3))|((symbol6) (((symbol4)|(symbol5))) (symbol7))|(symbol1)) (symbol1))
      Processing file: /cavs/hse/ies/d002/isip/users/wholland/new_tools/class/pr/LanguageModelXML/diagnose_input_nesting_2.sof
        reading model
          reading levels
            reading grammars
              Parsing the grammar...
              The parsed document:
                <grammar root='word'>
                  <rule id='word'>
                    <item>
                      symbol0
                    </item>
                    <item>
                      symbol1
                    </item>
                    <one-of>
                      <item>
                        <item>
                          symbol1
                        </item>
                        <one-of>
                          <item>
                            symbol2a
                          </item>
                          <item>
                            symbol2b
                          </item>
                          <item>
                            symbol2c
                          </item>
                        </one-of>
                        <item>
                          symbol3
                        </item>
                      </item>
                      <item>
                        <item>
                          <one-of>
                            <item>
                              symbol4
                            </item>
                            <item>
                              symbol5
                            </item>
                          </one-of>
                        </item>
                        <item>
                          symbol6
                        </item>
                        <item>
                          symbol7
                        </item>
                      </item>
                    </one-of>
                    <item>
                      symbol1
                    </item>
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='word'>
              |1     |2     |  <rule id='word'>
              |2     |3     |    <item>
              |3     |3     |      symbol0
              |4     |2     |    </item>
              |5     |3     |    <item>
              |6     |3     |      symbol1
              |7     |2     |    </item>
              |8     |3     |    <one-of>
              |9     |4     |      <item>
              |10    |5     |        <item>
              |11    |5     |          symbol1
              |12    |4     |        </item>
              |13    |5     |        <one-of>
              |14    |6     |          <item>
              |15    |6     |            symbol2a
              |16    |5     |          </item>
              |17    |6     |          <item>
              |18    |6     |            symbol2b
              |19    |5     |          </item>
              |20    |6     |          <item>
              |21    |6     |            symbol2c
              |22    |5     |          </item>
              |23    |4     |        </one-of>
              |24    |5     |        <item>
              |25    |5     |          symbol3
              |26    |4     |        </item>
              |27    |3     |      </item>
              |28    |4     |      <item>
              |29    |5     |        <item>
              |30    |6     |          <one-of>
              |31    |7     |            <item>
              |32    |7     |              symbol4
              |33    |6     |            </item>
              |34    |7     |            <item>
              |35    |7     |              symbol5
              |36    |6     |            </item>
              |37    |5     |          </one-of>
              |38    |4     |        </item>
              |39    |5     |        <item>
              |40    |5     |          symbol6
              |41    |4     |        </item>
              |42    |5     |        <item>
              |43    |5     |          symbol7
              |44    |4     |        </item>
              |45    |3     |      </item>
              |46    |2     |    </one-of>
              |47    |3     |    <item>
              |48    |3     |      symbol1
              |49    |2     |    </item>
              |50    |1     |  </rule>
              |51    |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='word'>
                  <rule id='word'>
                    <item>
                      symbol0
                    </item>
                    <item>
                      symbol1
                    </item>
                    <one-of>
                      <item>
                        <item>
                          symbol1
                        </item>
                        <one-of>
                          <item>
                            symbol2a
                          </item>
                          <item>
                            symbol2b
                          </item>
                          <item>
                            symbol2c
                          </item>
                        </one-of>
                        <item>
                          symbol3
                        </item>
                      </item>
                      <item>
                        <item>
                          <one-of>
                            <item>
                              symbol4
                            </item>
                            <item>
                              symbol5
                            </item>
                          </one-of>
                        </item>
                        <item>
                          symbol6
                        </item>
                        <item>
                          symbol7
                        </item>
                      </item>
                    </one-of>
                    <item>
                      symbol1
                    </item>
                  </rule>
                </grammar>
            reading level tag
              Parsing the grammar...
              The parsed document:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='level0'>
              |1     |2     |  <rule id='level0'>
              |2     |2     |    word
              |3     |1     |  </rule>
              |4     |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
            reading symbols
            reading context mapping
              <HashTable<Context,Ulong>::dummy context hash table:> num_items_d = 10
              <HashTable<Context,Ulong>::dummy context hash table:> load_factor_d = 0.75
              <HashTable<Context,Ulong>::dummy context hash table:> capacity_d = 128
              <HashTable<Context,Ulong>::dummy context hash table:> alloc_d = SYSTEM
                <CircularDelayLine<Ulong>::0> index_d = 0
                <CircularDelayLine<Ulong>::0> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 0
                  <Ulong::0> value_d = 0
                <CircularDelayLine<Ulong>::1> index_d = 0
                <CircularDelayLine<Ulong>::1> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 1
                  <Ulong::1> value_d = 1
                <CircularDelayLine<Ulong>::2> index_d = 0
                <CircularDelayLine<Ulong>::2> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 2
                  <Ulong::2> value_d = 2
                <CircularDelayLine<Ulong>::3> index_d = 0
                <CircularDelayLine<Ulong>::3> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 3
                  <Ulong::3> value_d = 3
                <CircularDelayLine<Ulong>::4> index_d = 0
                <CircularDelayLine<Ulong>::4> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 4
                  <Ulong::4> value_d = 4
                <CircularDelayLine<Ulong>::5> index_d = 0
                <CircularDelayLine<Ulong>::5> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 5
                  <Ulong::5> value_d = 5
                <CircularDelayLine<Ulong>::6> index_d = 0
                <CircularDelayLine<Ulong>::6> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 6
                  <Ulong::6> value_d = 6
                <CircularDelayLine<Ulong>::7> index_d = 0
                <CircularDelayLine<Ulong>::7> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 7
                  <Ulong::7> value_d = 7
                <CircularDelayLine<Ulong>::8> index_d = 0
                <CircularDelayLine<Ulong>::8> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 8
                  <Ulong::8> value_d = 8
                <CircularDelayLine<Ulong>::9> index_d = 0
                <CircularDelayLine<Ulong>::9> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 9
                  <Ulong::9> value_d = 9
        Level 0
          Graph 0
          <?xml version='1.0' encoding='utf-8'?>
          <grammar root='word'>
            <rule id='word'>
              <item> symbol0 </item>
              <item> symbol1 </item>
              <one-of>
                <item>
                  <item> symbol1 </item>
                  <one-of>
                    <item> symbol2a </item>
                    <item> symbol2b </item>
                    <item> symbol2c </item>
                  </one-of>
                  <item> symbol3 </item>
                </item>
                <item>
                  <item>
                    <one-of>
                      <item> symbol4 </item>
                      <item> symbol5 </item>
                    </one-of>
                  </item>
                  <item> symbol6 </item>
                  <item> symbol7 </item>
                </item>
              </one-of>
              <item> symbol1 </item>
            </rule>
          </grammar>
          
        performing XML->ABNF conversion
          level 0
            graph 0
              preprocessing
                Handling repeat requests...
                The modified grammar:
                <rule id='word'>
                  <item>
                    symbol0
                  </item>
                  <item>
                    symbol1
                  </item>
                  <one-of>
                    <item>
                      <item>
                        symbol1
                      </item>
                      <one-of>
                        <item>
                          symbol2a
                        </item>
                        <item>
                          symbol2b
                        </item>
                        <item>
                          symbol2c
                        </item>
                      </one-of>
                      <item>
                        symbol3
                      </item>
                    </item>
                    <item>
                      <item>
                        <one-of>
                          <item>
                            symbol4
                          </item>
                          <item>
                            symbol5
                          </item>
                        </one-of>
                      </item>
                      <item>
                        symbol6
                      </item>
                      <item>
                        symbol7
                      </item>
                    </item>
                  </one-of>
                  <item>
                    symbol1
                  </item>
                </rule>
              performing conversion
              ABNF equivalent:
                word -> ((symbol0) (symbol1) (((symbol1) ((symbol2a)|(symbol2b)|(symbol2c)|(symbol3)) (symbol3))|((((symbol4)|(symbol5))) (symbol6) (symbol7))|(symbol1)) (symbol1))
      Processing file: /cavs/hse/ies/d002/isip/users/wholland/new_tools/class/pr/LanguageModelXML/diagnose_input_nesting_3.sof
        reading model
          reading levels
            reading grammars
              Parsing the grammar...
              The parsed document:
                <grammar root='word'>
                  <rule id='word'>
                    <one-of>
                      <item>
                        worda
                      </item>
                      <item>
                        wordb
                      </item>
                      <item>
                        wordc
                      </item>
                    </one-of>
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='word'>
              |1     |2     |  <rule id='word'>
              |2     |3     |    <one-of>
              |3     |4     |      <item>
              |4     |4     |        worda
              |5     |3     |      </item>
              |6     |4     |      <item>
              |7     |4     |        wordb
              |8     |3     |      </item>
              |9     |4     |      <item>
              |10    |4     |        wordc
              |11    |3     |      </item>
              |12    |2     |    </one-of>
              |13    |1     |  </rule>
              |14    |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='word'>
                  <rule id='word'>
                    <one-of>
                      <item>
                        worda
                      </item>
                      <item>
                        wordb
                      </item>
                      <item>
                        wordc
                      </item>
                    </one-of>
                  </rule>
                </grammar>
            reading level tag
              Parsing the grammar...
              The parsed document:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='level0'>
              |1     |2     |  <rule id='level0'>
              |2     |2     |    word
              |3     |1     |  </rule>
              |4     |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
            reading symbols
            reading context mapping
              <HashTable<Context,Ulong>::dummy context hash table:> num_items_d = 3
              <HashTable<Context,Ulong>::dummy context hash table:> load_factor_d = 0.75
              <HashTable<Context,Ulong>::dummy context hash table:> capacity_d = 128
              <HashTable<Context,Ulong>::dummy context hash table:> alloc_d = SYSTEM
                <CircularDelayLine<Ulong>::0> index_d = 0
                <CircularDelayLine<Ulong>::0> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 0
                  <Ulong::0> value_d = 0
                <CircularDelayLine<Ulong>::1> index_d = 0
                <CircularDelayLine<Ulong>::1> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 1
                  <Ulong::1> value_d = 1
                <CircularDelayLine<Ulong>::2> index_d = 0
                <CircularDelayLine<Ulong>::2> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 2
                  <Ulong::2> value_d = 2
        Level 0
          Graph 0
          <?xml version='1.0' encoding='utf-8'?>
          <grammar root='word'>
            <rule id='word'>
              <one-of>
                <item> worda </item>
                <item> wordb </item>
                <item> wordc </item>
              </one-of>
            </rule>
          </grammar>
          
        performing XML->ABNF conversion
          level 0
            graph 0
              preprocessing
                Handling repeat requests...
                The modified grammar:
                <rule id='word'>
                  <one-of>
                    <item>
                      worda
                    </item>
                    <item>
                      wordb
                    </item>
                    <item>
                      wordc
                    </item>
                  </one-of>
                </rule>
              performing conversion
              ABNF equivalent:
                word -> (((worda)|(wordb)|(wordc)))
      Processing file: /cavs/hse/ies/d002/isip/users/wholland/new_tools/class/pr/LanguageModelXML/diagnose_input_repeat.sof
        reading model
          reading levels
            reading grammars
              Parsing the grammar...
              The parsed document:
                <grammar root='word'>
                  <rule id='word'>
                    symbol0
                    symbol1
                    <one-of>
                      <item repeat='1-'>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                      <item>
                        <ruleref uri='#rule3'/>
                      </item>
                    </one-of>
                    symbol1
                  </rule>
                  <rule id='rule2'>
                    <item>
                      symbol2a
                    </item>
                    <item>
                      symbol2b
                    </item>
                    <item>
                      symbol2c
                    </item>
                  </rule>
                  <rule id='rule3'>
                    symbol6
                    <item>
                      <one-of>
                        <item>
                          symbol4
                        </item>
                        <item>
                          symbol5
                        </item>
                      </one-of>
                    </item>
                    symbol7
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='word'>
              |1     |2     |  <rule id='word'>
              |2     |2     |    symbol0
              |3     |2     |    symbol1
              |4     |3     |    <one-of>
              |5     |4     |      <item repeat='1-'>
              |6     |4     |        symbol1
              |7     |5     |        <one-of>
              |8     |6     |          <item>
              |9     |6     |            <ruleref uri='#rule2'/>
              |10    |5     |          </item>
              |11    |4     |        </one-of>
              |12    |4     |        symbol3
              |13    |3     |      </item>
              |14    |4     |      <item>
              |15    |4     |        <ruleref uri='#rule3'/>
              |16    |3     |      </item>
              |17    |2     |    </one-of>
              |18    |2     |    symbol1
              |19    |1     |  </rule>
              |20    |2     |  <rule id='rule2'>
              |21    |3     |    <item>
              |22    |3     |      symbol2a
              |23    |2     |    </item>
              |24    |3     |    <item>
              |25    |3     |      symbol2b
              |26    |2     |    </item>
              |27    |3     |    <item>
              |28    |3     |      symbol2c
              |29    |2     |    </item>
              |30    |1     |  </rule>
              |31    |2     |  <rule id='rule3'>
              |32    |2     |    symbol6
              |33    |3     |    <item>
              |34    |4     |      <one-of>
              |35    |5     |        <item>
              |36    |5     |          symbol4
              |37    |4     |        </item>
              |38    |5     |        <item>
              |39    |5     |          symbol5
              |40    |4     |        </item>
              |41    |3     |      </one-of>
              |42    |2     |    </item>
              |43    |2     |    symbol7
              |44    |1     |  </rule>
              |45    |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='word'>
                  <rule id='word'>
                    symbol0
                    symbol1
                    <one-of>
                      <item repeat='1-'>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                      <item>
                        <ruleref uri='#rule3'/>
                      </item>
                    </one-of>
                    symbol1
                  </rule>
                  <rule id='rule2'>
                    <item>
                      symbol2a
                    </item>
                    <item>
                      symbol2b
                    </item>
                    <item>
                      symbol2c
                    </item>
                  </rule>
                  <rule id='rule3'>
                    symbol6
                    <item>
                      <one-of>
                        <item>
                          symbol4
                        </item>
                        <item>
                          symbol5
                        </item>
                      </one-of>
                    </item>
                    symbol7
                  </rule>
                </grammar>
            reading level tag
              Parsing the grammar...
              The parsed document:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='level0'>
              |1     |2     |  <rule id='level0'>
              |2     |2     |    word
              |3     |1     |  </rule>
              |4     |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
            reading symbols
            reading context mapping
              <HashTable<Context,Ulong>::dummy context hash table:> num_items_d = 10
              <HashTable<Context,Ulong>::dummy context hash table:> load_factor_d = 0.75
              <HashTable<Context,Ulong>::dummy context hash table:> capacity_d = 128
              <HashTable<Context,Ulong>::dummy context hash table:> alloc_d = SYSTEM
                <CircularDelayLine<Ulong>::0> index_d = 0
                <CircularDelayLine<Ulong>::0> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 0
                  <Ulong::0> value_d = 0
                <CircularDelayLine<Ulong>::1> index_d = 0
                <CircularDelayLine<Ulong>::1> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 1
                  <Ulong::1> value_d = 1
                <CircularDelayLine<Ulong>::2> index_d = 0
                <CircularDelayLine<Ulong>::2> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 2
                  <Ulong::2> value_d = 2
                <CircularDelayLine<Ulong>::3> index_d = 0
                <CircularDelayLine<Ulong>::3> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 3
                  <Ulong::3> value_d = 3
                <CircularDelayLine<Ulong>::4> index_d = 0
                <CircularDelayLine<Ulong>::4> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 4
                  <Ulong::4> value_d = 4
                <CircularDelayLine<Ulong>::5> index_d = 0
                <CircularDelayLine<Ulong>::5> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 5
                  <Ulong::5> value_d = 5
                <CircularDelayLine<Ulong>::6> index_d = 0
                <CircularDelayLine<Ulong>::6> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 6
                  <Ulong::6> value_d = 6
                <CircularDelayLine<Ulong>::7> index_d = 0
                <CircularDelayLine<Ulong>::7> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 7
                  <Ulong::7> value_d = 7
                <CircularDelayLine<Ulong>::8> index_d = 0
                <CircularDelayLine<Ulong>::8> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 8
                  <Ulong::8> value_d = 8
                <CircularDelayLine<Ulong>::9> index_d = 0
                <CircularDelayLine<Ulong>::9> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 9
                  <Ulong::9> value_d = 9
        Level 0
          Graph 0
          <?xml version='1.0' encoding='utf-8'?>
          <grammar root='word'>
            <rule id='word'>
              symbol0
              symbol1
              <one-of>
                <item repeat='1-'>
                  symbol1
                  <one-of>
                    <item>
                      <ruleref uri='#rule2'/>
                    </item>
                  </one-of>
                  symbol3
                </item>
                <item>
                  <ruleref uri='#rule3'/>
                </item>
              </one-of>
              symbol1
            </rule>
            <rule id='rule2'>
              <item> symbol2a </item>
              <item> symbol2b </item>
              <item> symbol2c </item>
            </rule>
            <rule id='rule3'>
              symbol6
              <item>
                <one-of>
                  <item> symbol4 </item>
                  <item> symbol5 </item>
                </one-of>
              </item>
              symbol7
            </rule>
          </grammar>
          
        performing XML->ABNF conversion
          level 0
            graph 0
              preprocessing
                Handling repeat requests...
                The modified grammar:
                <rule id='word'>
                  symbol0
                  symbol1
                  <one-of>
                    <item>
                      <item repeat='1-'>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                    </item>
                    <item>
                      <ruleref uri='#rule3'/>
                    </item>
                  </one-of>
                  symbol1
                </rule>
                Handling repeat requests...
                The modified grammar:
                <rule id='rule2'>
                  <item>
                    symbol2a
                  </item>
                  <item>
                    symbol2b
                  </item>
                  <item>
                    symbol2c
                  </item>
                </rule>
                Handling repeat requests...
                The modified grammar:
                <rule id='rule3'>
                  symbol6
                  <item>
                    <one-of>
                      <item>
                        symbol4
                      </item>
                      <item>
                        symbol5
                      </item>
                    </one-of>
                  </item>
                  symbol7
                </rule>
              performing conversion
              ABNF equivalent:
                word -> (symbol0 symbol1 (((symbol1 (RULE2) symbol3)+)|(RULE3)) symbol1)
              ABNF equivalent:
                rule2 -> ((symbol2a) (symbol2b) (symbol2c))
              ABNF equivalent:
                rule3 -> (symbol6 (((symbol4)|(symbol5))) symbol7)
      Processing file: /cavs/hse/ies/d002/isip/users/wholland/new_tools/class/pr/LanguageModelXML/diagnose_input_repeat_0-.sof
        reading model
          reading levels
            reading grammars
              Parsing the grammar...
              The parsed document:
                <grammar root='word'>
                  <rule id='word'>
                    symbol0
                    symbol1
                    <one-of>
                      <item repeat='0-'>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                      <item>
                        <ruleref uri='#rule3'/>
                      </item>
                    </one-of>
                    symbol1
                  </rule>
                  <rule id='rule2'>
                    <item>
                      symbol2a
                    </item>
                    <item>
                      symbol2b
                    </item>
                    <item>
                      symbol2c
                    </item>
                  </rule>
                  <rule id='rule3'>
                    symbol6
                    <item>
                      <one-of>
                        <item>
                          symbol4
                        </item>
                        <item>
                          symbol5
                        </item>
                      </one-of>
                    </item>
                    symbol7
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='word'>
              |1     |2     |  <rule id='word'>
              |2     |2     |    symbol0
              |3     |2     |    symbol1
              |4     |3     |    <one-of>
              |5     |4     |      <item repeat='0-'>
              |6     |4     |        symbol1
              |7     |5     |        <one-of>
              |8     |6     |          <item>
              |9     |6     |            <ruleref uri='#rule2'/>
              |10    |5     |          </item>
              |11    |4     |        </one-of>
              |12    |4     |        symbol3
              |13    |3     |      </item>
              |14    |4     |      <item>
              |15    |4     |        <ruleref uri='#rule3'/>
              |16    |3     |      </item>
              |17    |2     |    </one-of>
              |18    |2     |    symbol1
              |19    |1     |  </rule>
              |20    |2     |  <rule id='rule2'>
              |21    |3     |    <item>
              |22    |3     |      symbol2a
              |23    |2     |    </item>
              |24    |3     |    <item>
              |25    |3     |      symbol2b
              |26    |2     |    </item>
              |27    |3     |    <item>
              |28    |3     |      symbol2c
              |29    |2     |    </item>
              |30    |1     |  </rule>
              |31    |2     |  <rule id='rule3'>
              |32    |2     |    symbol6
              |33    |3     |    <item>
              |34    |4     |      <one-of>
              |35    |5     |        <item>
              |36    |5     |          symbol4
              |37    |4     |        </item>
              |38    |5     |        <item>
              |39    |5     |          symbol5
              |40    |4     |        </item>
              |41    |3     |      </one-of>
              |42    |2     |    </item>
              |43    |2     |    symbol7
              |44    |1     |  </rule>
              |45    |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='word'>
                  <rule id='word'>
                    symbol0
                    symbol1
                    <one-of>
                      <item repeat='0-'>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                      <item>
                        <ruleref uri='#rule3'/>
                      </item>
                    </one-of>
                    symbol1
                  </rule>
                  <rule id='rule2'>
                    <item>
                      symbol2a
                    </item>
                    <item>
                      symbol2b
                    </item>
                    <item>
                      symbol2c
                    </item>
                  </rule>
                  <rule id='rule3'>
                    symbol6
                    <item>
                      <one-of>
                        <item>
                          symbol4
                        </item>
                        <item>
                          symbol5
                        </item>
                      </one-of>
                    </item>
                    symbol7
                  </rule>
                </grammar>
            reading level tag
              Parsing the grammar...
              The parsed document:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='level0'>
              |1     |2     |  <rule id='level0'>
              |2     |2     |    word
              |3     |1     |  </rule>
              |4     |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
            reading symbols
            reading context mapping
              <HashTable<Context,Ulong>::dummy context hash table:> num_items_d = 10
              <HashTable<Context,Ulong>::dummy context hash table:> load_factor_d = 0.75
              <HashTable<Context,Ulong>::dummy context hash table:> capacity_d = 128
              <HashTable<Context,Ulong>::dummy context hash table:> alloc_d = SYSTEM
                <CircularDelayLine<Ulong>::0> index_d = 0
                <CircularDelayLine<Ulong>::0> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 0
                  <Ulong::0> value_d = 0
                <CircularDelayLine<Ulong>::1> index_d = 0
                <CircularDelayLine<Ulong>::1> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 1
                  <Ulong::1> value_d = 1
                <CircularDelayLine<Ulong>::2> index_d = 0
                <CircularDelayLine<Ulong>::2> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 2
                  <Ulong::2> value_d = 2
                <CircularDelayLine<Ulong>::3> index_d = 0
                <CircularDelayLine<Ulong>::3> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 3
                  <Ulong::3> value_d = 3
                <CircularDelayLine<Ulong>::4> index_d = 0
                <CircularDelayLine<Ulong>::4> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 4
                  <Ulong::4> value_d = 4
                <CircularDelayLine<Ulong>::5> index_d = 0
                <CircularDelayLine<Ulong>::5> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 5
                  <Ulong::5> value_d = 5
                <CircularDelayLine<Ulong>::6> index_d = 0
                <CircularDelayLine<Ulong>::6> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 6
                  <Ulong::6> value_d = 6
                <CircularDelayLine<Ulong>::7> index_d = 0
                <CircularDelayLine<Ulong>::7> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 7
                  <Ulong::7> value_d = 7
                <CircularDelayLine<Ulong>::8> index_d = 0
                <CircularDelayLine<Ulong>::8> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 8
                  <Ulong::8> value_d = 8
                <CircularDelayLine<Ulong>::9> index_d = 0
                <CircularDelayLine<Ulong>::9> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 9
                  <Ulong::9> value_d = 9
        Level 0
          Graph 0
          <?xml version='1.0' encoding='utf-8'?>
          <grammar root='word'>
            <rule id='word'>
              symbol0
              symbol1
              <one-of>
                <item repeat='0-'>
                  symbol1
                  <one-of>
                    <item>
                      <ruleref uri='#rule2'/>
                    </item>
                  </one-of>
                  symbol3
                </item>
                <item>
                  <ruleref uri='#rule3'/>
                </item>
              </one-of>
              symbol1
            </rule>
            <rule id='rule2'>
              <item> symbol2a </item>
              <item> symbol2b </item>
              <item> symbol2c </item>
            </rule>
            <rule id='rule3'>
              symbol6
              <item>
                <one-of>
                  <item> symbol4 </item>
                  <item> symbol5 </item>
                </one-of>
              </item>
              symbol7
            </rule>
          </grammar>
          
        performing XML->ABNF conversion
          level 0
            graph 0
              preprocessing
                Handling repeat requests...
                The modified grammar:
                <rule id='word'>
                  symbol0
                  symbol1
                  <one-of>
                    <item repeat='1-'>
                      <one-of>
                        <item>
                          <ruleref special='NULL'/>
                        </item>
                        <item>
                          symbol1
                          <one-of>
                            <item>
                              <ruleref uri='#rule2'/>
                            </item>
                          </one-of>
                          symbol3
                        </item>
                      </one-of>
                    </item>
                    <item>
                      <ruleref uri='#rule3'/>
                    </item>
                  </one-of>
                  symbol1
                </rule>
                Handling repeat requests...
                The modified grammar:
                <rule id='rule2'>
                  <item>
                    symbol2a
                  </item>
                  <item>
                    symbol2b
                  </item>
                  <item>
                    symbol2c
                  </item>
                </rule>
                Handling repeat requests...
                The modified grammar:
                <rule id='rule3'>
                  symbol6
                  <item>
                    <one-of>
                      <item>
                        symbol4
                      </item>
                      <item>
                        symbol5
                      </item>
                    </one-of>
                  </item>
                  symbol7
                </rule>
              performing conversion
              ABNF equivalent:
                word -> (symbol0 symbol1 (((((epsilon))|(symbol1 (RULE2) symbol3)))+|(RULE3)) symbol1)
              ABNF equivalent:
                rule2 -> ((symbol2a) (symbol2b) (symbol2c))
              ABNF equivalent:
                rule3 -> (symbol6 (((symbol4)|(symbol5))) symbol7)
      Processing file: /cavs/hse/ies/d002/isip/users/wholland/new_tools/class/pr/LanguageModelXML/diagnose_input_repeat_0-N.sof
        reading model
          reading levels
            reading grammars
              Parsing the grammar...
              The parsed document:
                <grammar root='word'>
                  <rule id='word'>
                    symbol0
                    symbol1
                    <one-of>
                      <item repeat='0-2'>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                      <item>
                        <ruleref uri='#rule3'/>
                      </item>
                    </one-of>
                    symbol1
                  </rule>
                  <rule id='rule2'>
                    <item>
                      symbol2a
                    </item>
                    <item>
                      symbol2b
                    </item>
                    <item>
                      symbol2c
                    </item>
                  </rule>
                  <rule id='rule3'>
                    symbol6
                    <item>
                      <one-of>
                        <item>
                          symbol4
                        </item>
                        <item>
                          symbol5
                        </item>
                      </one-of>
                    </item>
                    symbol7
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='word'>
              |1     |2     |  <rule id='word'>
              |2     |2     |    symbol0
              |3     |2     |    symbol1
              |4     |3     |    <one-of>
              |5     |4     |      <item repeat='0-2'>
              |6     |4     |        symbol1
              |7     |5     |        <one-of>
              |8     |6     |          <item>
              |9     |6     |            <ruleref uri='#rule2'/>
              |10    |5     |          </item>
              |11    |4     |        </one-of>
              |12    |4     |        symbol3
              |13    |3     |      </item>
              |14    |4     |      <item>
              |15    |4     |        <ruleref uri='#rule3'/>
              |16    |3     |      </item>
              |17    |2     |    </one-of>
              |18    |2     |    symbol1
              |19    |1     |  </rule>
              |20    |2     |  <rule id='rule2'>
              |21    |3     |    <item>
              |22    |3     |      symbol2a
              |23    |2     |    </item>
              |24    |3     |    <item>
              |25    |3     |      symbol2b
              |26    |2     |    </item>
              |27    |3     |    <item>
              |28    |3     |      symbol2c
              |29    |2     |    </item>
              |30    |1     |  </rule>
              |31    |2     |  <rule id='rule3'>
              |32    |2     |    symbol6
              |33    |3     |    <item>
              |34    |4     |      <one-of>
              |35    |5     |        <item>
              |36    |5     |          symbol4
              |37    |4     |        </item>
              |38    |5     |        <item>
              |39    |5     |          symbol5
              |40    |4     |        </item>
              |41    |3     |      </one-of>
              |42    |2     |    </item>
              |43    |2     |    symbol7
              |44    |1     |  </rule>
              |45    |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='word'>
                  <rule id='word'>
                    symbol0
                    symbol1
                    <one-of>
                      <item repeat='0-2'>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                      <item>
                        <ruleref uri='#rule3'/>
                      </item>
                    </one-of>
                    symbol1
                  </rule>
                  <rule id='rule2'>
                    <item>
                      symbol2a
                    </item>
                    <item>
                      symbol2b
                    </item>
                    <item>
                      symbol2c
                    </item>
                  </rule>
                  <rule id='rule3'>
                    symbol6
                    <item>
                      <one-of>
                        <item>
                          symbol4
                        </item>
                        <item>
                          symbol5
                        </item>
                      </one-of>
                    </item>
                    symbol7
                  </rule>
                </grammar>
            reading level tag
              Parsing the grammar...
              The parsed document:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='level0'>
              |1     |2     |  <rule id='level0'>
              |2     |2     |    word
              |3     |1     |  </rule>
              |4     |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
            reading symbols
            reading context mapping
              <HashTable<Context,Ulong>::dummy context hash table:> num_items_d = 10
              <HashTable<Context,Ulong>::dummy context hash table:> load_factor_d = 0.75
              <HashTable<Context,Ulong>::dummy context hash table:> capacity_d = 128
              <HashTable<Context,Ulong>::dummy context hash table:> alloc_d = SYSTEM
                <CircularDelayLine<Ulong>::0> index_d = 0
                <CircularDelayLine<Ulong>::0> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 0
                  <Ulong::0> value_d = 0
                <CircularDelayLine<Ulong>::1> index_d = 0
                <CircularDelayLine<Ulong>::1> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 1
                  <Ulong::1> value_d = 1
                <CircularDelayLine<Ulong>::2> index_d = 0
                <CircularDelayLine<Ulong>::2> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 2
                  <Ulong::2> value_d = 2
                <CircularDelayLine<Ulong>::3> index_d = 0
                <CircularDelayLine<Ulong>::3> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 3
                  <Ulong::3> value_d = 3
                <CircularDelayLine<Ulong>::4> index_d = 0
                <CircularDelayLine<Ulong>::4> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 4
                  <Ulong::4> value_d = 4
                <CircularDelayLine<Ulong>::5> index_d = 0
                <CircularDelayLine<Ulong>::5> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 5
                  <Ulong::5> value_d = 5
                <CircularDelayLine<Ulong>::6> index_d = 0
                <CircularDelayLine<Ulong>::6> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 6
                  <Ulong::6> value_d = 6
                <CircularDelayLine<Ulong>::7> index_d = 0
                <CircularDelayLine<Ulong>::7> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 7
                  <Ulong::7> value_d = 7
                <CircularDelayLine<Ulong>::8> index_d = 0
                <CircularDelayLine<Ulong>::8> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 8
                  <Ulong::8> value_d = 8
                <CircularDelayLine<Ulong>::9> index_d = 0
                <CircularDelayLine<Ulong>::9> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 9
                  <Ulong::9> value_d = 9
        Level 0
          Graph 0
          <?xml version='1.0' encoding='utf-8'?>
          <grammar root='word'>
            <rule id='word'>
              symbol0
              symbol1
              <one-of>
                <item repeat='0-2'>
                  symbol1
                  <one-of>
                    <item>
                      <ruleref uri='#rule2'/>
                    </item>
                  </one-of>
                  symbol3
                </item>
                <item>
                  <ruleref uri='#rule3'/>
                </item>
              </one-of>
              symbol1
            </rule>
            <rule id='rule2'>
              <item> symbol2a </item>
              <item> symbol2b </item>
              <item> symbol2c </item>
            </rule>
            <rule id='rule3'>
              symbol6
              <item>
                <one-of>
                  <item> symbol4 </item>
                  <item> symbol5 </item>
                </one-of>
              </item>
              symbol7
            </rule>
          </grammar>
          
        performing XML->ABNF conversion
          level 0
            graph 0
              preprocessing
                Handling repeat requests...
                The modified grammar:
                <rule id='word'>
                  symbol0
                  symbol1
                  <one-of>
                    <item>
                      <one-of>
                        <item>
                          <ruleref special='NULL'/>
                        </item>
                        <item>
                          symbol1
                          <one-of>
                            <item>
                              <ruleref uri='#rule2'/>
                            </item>
                          </one-of>
                          symbol3
                        </item>
                      </one-of>
                      <one-of>
                        <item>
                          <ruleref special='NULL'/>
                        </item>
                        <item>
                          symbol1
                          <one-of>
                            <item>
                              <ruleref uri='#rule2'/>
                            </item>
                          </one-of>
                          symbol3
                        </item>
                      </one-of>
                    </item>
                    <item>
                      <ruleref uri='#rule3'/>
                    </item>
                  </one-of>
                  symbol1
                </rule>
                Handling repeat requests...
                The modified grammar:
                <rule id='rule2'>
                  <item>
                    symbol2a
                  </item>
                  <item>
                    symbol2b
                  </item>
                  <item>
                    symbol2c
                  </item>
                </rule>
                Handling repeat requests...
                The modified grammar:
                <rule id='rule3'>
                  symbol6
                  <item>
                    <one-of>
                      <item>
                        symbol4
                      </item>
                      <item>
                        symbol5
                      </item>
                    </one-of>
                  </item>
                  symbol7
                </rule>
              performing conversion
              ABNF equivalent:
                word -> (symbol0 symbol1 (((((epsilon))|(symbol1 (RULE2) symbol3)|((epsilon))|(symbol1 (RULE2) symbol3)) (((epsilon))|(symbol1 (RULE2) symbol3)))|(RULE3)) symbol1)
              ABNF equivalent:
                rule2 -> ((symbol2a) (symbol2b) (symbol2c))
              ABNF equivalent:
                rule3 -> (symbol6 (((symbol4)|(symbol5))) symbol7)
      Processing file: /cavs/hse/ies/d002/isip/users/wholland/new_tools/class/pr/LanguageModelXML/diagnose_input_repeat_M.sof
        reading model
          reading levels
            reading grammars
              Parsing the grammar...
              The parsed document:
                <grammar root='word'>
                  <rule id='word'>
                    symbol0
                    symbol1
                    <one-of>
                      <item repeat='5'>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                      <item>
                        <ruleref uri='#rule3'/>
                      </item>
                    </one-of>
                    symbol1
                  </rule>
                  <rule id='rule2'>
                    <item>
                      symbol2a
                    </item>
                    <item>
                      symbol2b
                    </item>
                    <item>
                      symbol2c
                    </item>
                  </rule>
                  <rule id='rule3'>
                    symbol6
                    <item>
                      <one-of>
                        <item>
                          symbol4
                        </item>
                        <item>
                          symbol5
                        </item>
                      </one-of>
                    </item>
                    symbol7
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='word'>
              |1     |2     |  <rule id='word'>
              |2     |2     |    symbol0
              |3     |2     |    symbol1
              |4     |3     |    <one-of>
              |5     |4     |      <item repeat='5'>
              |6     |4     |        symbol1
              |7     |5     |        <one-of>
              |8     |6     |          <item>
              |9     |6     |            <ruleref uri='#rule2'/>
              |10    |5     |          </item>
              |11    |4     |        </one-of>
              |12    |4     |        symbol3
              |13    |3     |      </item>
              |14    |4     |      <item>
              |15    |4     |        <ruleref uri='#rule3'/>
              |16    |3     |      </item>
              |17    |2     |    </one-of>
              |18    |2     |    symbol1
              |19    |1     |  </rule>
              |20    |2     |  <rule id='rule2'>
              |21    |3     |    <item>
              |22    |3     |      symbol2a
              |23    |2     |    </item>
              |24    |3     |    <item>
              |25    |3     |      symbol2b
              |26    |2     |    </item>
              |27    |3     |    <item>
              |28    |3     |      symbol2c
              |29    |2     |    </item>
              |30    |1     |  </rule>
              |31    |2     |  <rule id='rule3'>
              |32    |2     |    symbol6
              |33    |3     |    <item>
              |34    |4     |      <one-of>
              |35    |5     |        <item>
              |36    |5     |          symbol4
              |37    |4     |        </item>
              |38    |5     |        <item>
              |39    |5     |          symbol5
              |40    |4     |        </item>
              |41    |3     |      </one-of>
              |42    |2     |    </item>
              |43    |2     |    symbol7
              |44    |1     |  </rule>
              |45    |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='word'>
                  <rule id='word'>
                    symbol0
                    symbol1
                    <one-of>
                      <item repeat='5'>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                      <item>
                        <ruleref uri='#rule3'/>
                      </item>
                    </one-of>
                    symbol1
                  </rule>
                  <rule id='rule2'>
                    <item>
                      symbol2a
                    </item>
                    <item>
                      symbol2b
                    </item>
                    <item>
                      symbol2c
                    </item>
                  </rule>
                  <rule id='rule3'>
                    symbol6
                    <item>
                      <one-of>
                        <item>
                          symbol4
                        </item>
                        <item>
                          symbol5
                        </item>
                      </one-of>
                    </item>
                    symbol7
                  </rule>
                </grammar>
            reading level tag
              Parsing the grammar...
              The parsed document:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='level0'>
              |1     |2     |  <rule id='level0'>
              |2     |2     |    word
              |3     |1     |  </rule>
              |4     |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
            reading symbols
            reading context mapping
              <HashTable<Context,Ulong>::dummy context hash table:> num_items_d = 10
              <HashTable<Context,Ulong>::dummy context hash table:> load_factor_d = 0.75
              <HashTable<Context,Ulong>::dummy context hash table:> capacity_d = 128
              <HashTable<Context,Ulong>::dummy context hash table:> alloc_d = SYSTEM
                <CircularDelayLine<Ulong>::0> index_d = 0
                <CircularDelayLine<Ulong>::0> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 0
                  <Ulong::0> value_d = 0
                <CircularDelayLine<Ulong>::1> index_d = 0
                <CircularDelayLine<Ulong>::1> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 1
                  <Ulong::1> value_d = 1
                <CircularDelayLine<Ulong>::2> index_d = 0
                <CircularDelayLine<Ulong>::2> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 2
                  <Ulong::2> value_d = 2
                <CircularDelayLine<Ulong>::3> index_d = 0
                <CircularDelayLine<Ulong>::3> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 3
                  <Ulong::3> value_d = 3
                <CircularDelayLine<Ulong>::4> index_d = 0
                <CircularDelayLine<Ulong>::4> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 4
                  <Ulong::4> value_d = 4
                <CircularDelayLine<Ulong>::5> index_d = 0
                <CircularDelayLine<Ulong>::5> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 5
                  <Ulong::5> value_d = 5
                <CircularDelayLine<Ulong>::6> index_d = 0
                <CircularDelayLine<Ulong>::6> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 6
                  <Ulong::6> value_d = 6
                <CircularDelayLine<Ulong>::7> index_d = 0
                <CircularDelayLine<Ulong>::7> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 7
                  <Ulong::7> value_d = 7
                <CircularDelayLine<Ulong>::8> index_d = 0
                <CircularDelayLine<Ulong>::8> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 8
                  <Ulong::8> value_d = 8
                <CircularDelayLine<Ulong>::9> index_d = 0
                <CircularDelayLine<Ulong>::9> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 9
                  <Ulong::9> value_d = 9
        Level 0
          Graph 0
          <?xml version='1.0' encoding='utf-8'?>
          <grammar root='word'>
            <rule id='word'>
              symbol0
              symbol1
              <one-of>
                <item repeat='5'>
                  symbol1
                  <one-of>
                    <item>
                      <ruleref uri='#rule2'/>
                    </item>
                  </one-of>
                  symbol3
                </item>
                <item>
                  <ruleref uri='#rule3'/>
                </item>
              </one-of>
              symbol1
            </rule>
            <rule id='rule2'>
              <item> symbol2a </item>
              <item> symbol2b </item>
              <item> symbol2c </item>
            </rule>
            <rule id='rule3'>
              symbol6
              <item>
                <one-of>
                  <item> symbol4 </item>
                  <item> symbol5 </item>
                </one-of>
              </item>
              symbol7
            </rule>
          </grammar>
          
        performing XML->ABNF conversion
          level 0
            graph 0
              preprocessing
                Handling repeat requests...
                The modified grammar:
                <rule id='word'>
                  symbol0
                  symbol1
                  <one-of>
                    <item>
                      <item>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                      <item>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                      <item>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                      <item>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                      <item>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                    </item>
                    <item>
                      <ruleref uri='#rule3'/>
                    </item>
                  </one-of>
                  symbol1
                </rule>
                Handling repeat requests...
                The modified grammar:
                <rule id='rule2'>
                  <item>
                    symbol2a
                  </item>
                  <item>
                    symbol2b
                  </item>
                  <item>
                    symbol2c
                  </item>
                </rule>
                Handling repeat requests...
                The modified grammar:
                <rule id='rule3'>
                  symbol6
                  <item>
                    <one-of>
                      <item>
                        symbol4
                      </item>
                      <item>
                        symbol5
                      </item>
                    </one-of>
                  </item>
                  symbol7
                </rule>
              performing conversion
              ABNF equivalent:
                word -> (symbol0 symbol1 (((symbol1 (RULE2) symbol3) (symbol1 (RULE2) symbol3) (symbol1 (RULE2) symbol3) (symbol1 (RULE2) symbol3) (symbol1 (RULE2) symbol3))|(RULE3)) symbol1)
              ABNF equivalent:
                rule2 -> ((symbol2a) (symbol2b) (symbol2c))
              ABNF equivalent:
                rule3 -> (symbol6 (((symbol4)|(symbol5))) symbol7)
      Processing file: /cavs/hse/ies/d002/isip/users/wholland/new_tools/class/pr/LanguageModelXML/diagnose_input_repeat_M-.sof
        reading model
          reading levels
            reading grammars
              Parsing the grammar...
              The parsed document:
                <grammar root='word'>
                  <rule id='word'>
                    symbol0
                    symbol1
                    <one-of>
                      <item repeat='2-'>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                      <item>
                        <ruleref uri='#rule3'/>
                      </item>
                    </one-of>
                    symbol1
                  </rule>
                  <rule id='rule2'>
                    <item>
                      symbol2a
                    </item>
                    <item>
                      symbol2b
                    </item>
                    <item>
                      symbol2c
                    </item>
                  </rule>
                  <rule id='rule3'>
                    symbol6
                    <item>
                      <one-of>
                        <item>
                          symbol4
                        </item>
                        <item>
                          symbol5
                        </item>
                      </one-of>
                    </item>
                    symbol7
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='word'>
              |1     |2     |  <rule id='word'>
              |2     |2     |    symbol0
              |3     |2     |    symbol1
              |4     |3     |    <one-of>
              |5     |4     |      <item repeat='2-'>
              |6     |4     |        symbol1
              |7     |5     |        <one-of>
              |8     |6     |          <item>
              |9     |6     |            <ruleref uri='#rule2'/>
              |10    |5     |          </item>
              |11    |4     |        </one-of>
              |12    |4     |        symbol3
              |13    |3     |      </item>
              |14    |4     |      <item>
              |15    |4     |        <ruleref uri='#rule3'/>
              |16    |3     |      </item>
              |17    |2     |    </one-of>
              |18    |2     |    symbol1
              |19    |1     |  </rule>
              |20    |2     |  <rule id='rule2'>
              |21    |3     |    <item>
              |22    |3     |      symbol2a
              |23    |2     |    </item>
              |24    |3     |    <item>
              |25    |3     |      symbol2b
              |26    |2     |    </item>
              |27    |3     |    <item>
              |28    |3     |      symbol2c
              |29    |2     |    </item>
              |30    |1     |  </rule>
              |31    |2     |  <rule id='rule3'>
              |32    |2     |    symbol6
              |33    |3     |    <item>
              |34    |4     |      <one-of>
              |35    |5     |        <item>
              |36    |5     |          symbol4
              |37    |4     |        </item>
              |38    |5     |        <item>
              |39    |5     |          symbol5
              |40    |4     |        </item>
              |41    |3     |      </one-of>
              |42    |2     |    </item>
              |43    |2     |    symbol7
              |44    |1     |  </rule>
              |45    |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='word'>
                  <rule id='word'>
                    symbol0
                    symbol1
                    <one-of>
                      <item repeat='2-'>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                      <item>
                        <ruleref uri='#rule3'/>
                      </item>
                    </one-of>
                    symbol1
                  </rule>
                  <rule id='rule2'>
                    <item>
                      symbol2a
                    </item>
                    <item>
                      symbol2b
                    </item>
                    <item>
                      symbol2c
                    </item>
                  </rule>
                  <rule id='rule3'>
                    symbol6
                    <item>
                      <one-of>
                        <item>
                          symbol4
                        </item>
                        <item>
                          symbol5
                        </item>
                      </one-of>
                    </item>
                    symbol7
                  </rule>
                </grammar>
            reading level tag
              Parsing the grammar...
              The parsed document:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='level0'>
              |1     |2     |  <rule id='level0'>
              |2     |2     |    word
              |3     |1     |  </rule>
              |4     |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
            reading symbols
            reading context mapping
              <HashTable<Context,Ulong>::dummy context hash table:> num_items_d = 10
              <HashTable<Context,Ulong>::dummy context hash table:> load_factor_d = 0.75
              <HashTable<Context,Ulong>::dummy context hash table:> capacity_d = 128
              <HashTable<Context,Ulong>::dummy context hash table:> alloc_d = SYSTEM
                <CircularDelayLine<Ulong>::0> index_d = 0
                <CircularDelayLine<Ulong>::0> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 0
                  <Ulong::0> value_d = 0
                <CircularDelayLine<Ulong>::1> index_d = 0
                <CircularDelayLine<Ulong>::1> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 1
                  <Ulong::1> value_d = 1
                <CircularDelayLine<Ulong>::2> index_d = 0
                <CircularDelayLine<Ulong>::2> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 2
                  <Ulong::2> value_d = 2
                <CircularDelayLine<Ulong>::3> index_d = 0
                <CircularDelayLine<Ulong>::3> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 3
                  <Ulong::3> value_d = 3
                <CircularDelayLine<Ulong>::4> index_d = 0
                <CircularDelayLine<Ulong>::4> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 4
                  <Ulong::4> value_d = 4
                <CircularDelayLine<Ulong>::5> index_d = 0
                <CircularDelayLine<Ulong>::5> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 5
                  <Ulong::5> value_d = 5
                <CircularDelayLine<Ulong>::6> index_d = 0
                <CircularDelayLine<Ulong>::6> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 6
                  <Ulong::6> value_d = 6
                <CircularDelayLine<Ulong>::7> index_d = 0
                <CircularDelayLine<Ulong>::7> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 7
                  <Ulong::7> value_d = 7
                <CircularDelayLine<Ulong>::8> index_d = 0
                <CircularDelayLine<Ulong>::8> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 8
                  <Ulong::8> value_d = 8
                <CircularDelayLine<Ulong>::9> index_d = 0
                <CircularDelayLine<Ulong>::9> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 9
                  <Ulong::9> value_d = 9
        Level 0
          Graph 0
          <?xml version='1.0' encoding='utf-8'?>
          <grammar root='word'>
            <rule id='word'>
              symbol0
              symbol1
              <one-of>
                <item repeat='2-'>
                  symbol1
                  <one-of>
                    <item>
                      <ruleref uri='#rule2'/>
                    </item>
                  </one-of>
                  symbol3
                </item>
                <item>
                  <ruleref uri='#rule3'/>
                </item>
              </one-of>
              symbol1
            </rule>
            <rule id='rule2'>
              <item> symbol2a </item>
              <item> symbol2b </item>
              <item> symbol2c </item>
            </rule>
            <rule id='rule3'>
              symbol6
              <item>
                <one-of>
                  <item> symbol4 </item>
                  <item> symbol5 </item>
                </one-of>
              </item>
              symbol7
            </rule>
          </grammar>
          
        performing XML->ABNF conversion
          level 0
            graph 0
              preprocessing
                Handling repeat requests...
                The modified grammar:
                <rule id='word'>
                  symbol0
                  symbol1
                  <one-of>
                    <item>
                      <item>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                      <item repeat='1-'>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                    </item>
                    <item>
                      <ruleref uri='#rule3'/>
                    </item>
                  </one-of>
                  symbol1
                </rule>
                Handling repeat requests...
                The modified grammar:
                <rule id='rule2'>
                  <item>
                    symbol2a
                  </item>
                  <item>
                    symbol2b
                  </item>
                  <item>
                    symbol2c
                  </item>
                </rule>
                Handling repeat requests...
                The modified grammar:
                <rule id='rule3'>
                  symbol6
                  <item>
                    <one-of>
                      <item>
                        symbol4
                      </item>
                      <item>
                        symbol5
                      </item>
                    </one-of>
                  </item>
                  symbol7
                </rule>
              performing conversion
              ABNF equivalent:
                word -> (symbol0 symbol1 (((symbol1 (RULE2) symbol3) (symbol1 (RULE2) symbol3)+)|(RULE3)) symbol1)
              ABNF equivalent:
                rule2 -> ((symbol2a) (symbol2b) (symbol2c))
              ABNF equivalent:
                rule3 -> (symbol6 (((symbol4)|(symbol5))) symbol7)
      Processing file: /cavs/hse/ies/d002/isip/users/wholland/new_tools/class/pr/LanguageModelXML/diagnose_input_repeat_M-N.sof
        reading model
          reading levels
            reading grammars
              Parsing the grammar...
              The parsed document:
                <grammar root='word'>
                  <rule id='word'>
                    symbol0
                    symbol1
                    <one-of>
                      <item repeat='2-4'>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                      <item>
                        <ruleref uri='#rule3'/>
                      </item>
                    </one-of>
                    symbol1
                  </rule>
                  <rule id='rule2'>
                    <item>
                      symbol2a
                    </item>
                    <item>
                      symbol2b
                    </item>
                    <item>
                      symbol2c
                    </item>
                  </rule>
                  <rule id='rule3'>
                    symbol6
                    <item>
                      <one-of>
                        <item>
                          symbol4
                        </item>
                        <item>
                          symbol5
                        </item>
                      </one-of>
                    </item>
                    symbol7
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='word'>
              |1     |2     |  <rule id='word'>
              |2     |2     |    symbol0
              |3     |2     |    symbol1
              |4     |3     |    <one-of>
              |5     |4     |      <item repeat='2-4'>
              |6     |4     |        symbol1
              |7     |5     |        <one-of>
              |8     |6     |          <item>
              |9     |6     |            <ruleref uri='#rule2'/>
              |10    |5     |          </item>
              |11    |4     |        </one-of>
              |12    |4     |        symbol3
              |13    |3     |      </item>
              |14    |4     |      <item>
              |15    |4     |        <ruleref uri='#rule3'/>
              |16    |3     |      </item>
              |17    |2     |    </one-of>
              |18    |2     |    symbol1
              |19    |1     |  </rule>
              |20    |2     |  <rule id='rule2'>
              |21    |3     |    <item>
              |22    |3     |      symbol2a
              |23    |2     |    </item>
              |24    |3     |    <item>
              |25    |3     |      symbol2b
              |26    |2     |    </item>
              |27    |3     |    <item>
              |28    |3     |      symbol2c
              |29    |2     |    </item>
              |30    |1     |  </rule>
              |31    |2     |  <rule id='rule3'>
              |32    |2     |    symbol6
              |33    |3     |    <item>
              |34    |4     |      <one-of>
              |35    |5     |        <item>
              |36    |5     |          symbol4
              |37    |4     |        </item>
              |38    |5     |        <item>
              |39    |5     |          symbol5
              |40    |4     |        </item>
              |41    |3     |      </one-of>
              |42    |2     |    </item>
              |43    |2     |    symbol7
              |44    |1     |  </rule>
              |45    |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='word'>
                  <rule id='word'>
                    symbol0
                    symbol1
                    <one-of>
                      <item repeat='2-4'>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                      <item>
                        <ruleref uri='#rule3'/>
                      </item>
                    </one-of>
                    symbol1
                  </rule>
                  <rule id='rule2'>
                    <item>
                      symbol2a
                    </item>
                    <item>
                      symbol2b
                    </item>
                    <item>
                      symbol2c
                    </item>
                  </rule>
                  <rule id='rule3'>
                    symbol6
                    <item>
                      <one-of>
                        <item>
                          symbol4
                        </item>
                        <item>
                          symbol5
                        </item>
                      </one-of>
                    </item>
                    symbol7
                  </rule>
                </grammar>
            reading level tag
              Parsing the grammar...
              The parsed document:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='level0'>
              |1     |2     |  <rule id='level0'>
              |2     |2     |    word
              |3     |1     |  </rule>
              |4     |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
            reading symbols
            reading context mapping
              <HashTable<Context,Ulong>::dummy context hash table:> num_items_d = 10
              <HashTable<Context,Ulong>::dummy context hash table:> load_factor_d = 0.75
              <HashTable<Context,Ulong>::dummy context hash table:> capacity_d = 128
              <HashTable<Context,Ulong>::dummy context hash table:> alloc_d = SYSTEM
                <CircularDelayLine<Ulong>::0> index_d = 0
                <CircularDelayLine<Ulong>::0> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 0
                  <Ulong::0> value_d = 0
                <CircularDelayLine<Ulong>::1> index_d = 0
                <CircularDelayLine<Ulong>::1> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 1
                  <Ulong::1> value_d = 1
                <CircularDelayLine<Ulong>::2> index_d = 0
                <CircularDelayLine<Ulong>::2> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 2
                  <Ulong::2> value_d = 2
                <CircularDelayLine<Ulong>::3> index_d = 0
                <CircularDelayLine<Ulong>::3> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 3
                  <Ulong::3> value_d = 3
                <CircularDelayLine<Ulong>::4> index_d = 0
                <CircularDelayLine<Ulong>::4> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 4
                  <Ulong::4> value_d = 4
                <CircularDelayLine<Ulong>::5> index_d = 0
                <CircularDelayLine<Ulong>::5> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 5
                  <Ulong::5> value_d = 5
                <CircularDelayLine<Ulong>::6> index_d = 0
                <CircularDelayLine<Ulong>::6> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 6
                  <Ulong::6> value_d = 6
                <CircularDelayLine<Ulong>::7> index_d = 0
                <CircularDelayLine<Ulong>::7> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 7
                  <Ulong::7> value_d = 7
                <CircularDelayLine<Ulong>::8> index_d = 0
                <CircularDelayLine<Ulong>::8> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 8
                  <Ulong::8> value_d = 8
                <CircularDelayLine<Ulong>::9> index_d = 0
                <CircularDelayLine<Ulong>::9> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 9
                  <Ulong::9> value_d = 9
        Level 0
          Graph 0
          <?xml version='1.0' encoding='utf-8'?>
          <grammar root='word'>
            <rule id='word'>
              symbol0
              symbol1
              <one-of>
                <item repeat='2-4'>
                  symbol1
                  <one-of>
                    <item>
                      <ruleref uri='#rule2'/>
                    </item>
                  </one-of>
                  symbol3
                </item>
                <item>
                  <ruleref uri='#rule3'/>
                </item>
              </one-of>
              symbol1
            </rule>
            <rule id='rule2'>
              <item> symbol2a </item>
              <item> symbol2b </item>
              <item> symbol2c </item>
            </rule>
            <rule id='rule3'>
              symbol6
              <item>
                <one-of>
                  <item> symbol4 </item>
                  <item> symbol5 </item>
                </one-of>
              </item>
              symbol7
            </rule>
          </grammar>
          
        performing XML->ABNF conversion
          level 0
            graph 0
              preprocessing
                Handling repeat requests...
                The modified grammar:
                <rule id='word'>
                  symbol0
                  symbol1
                  <one-of>
                    <item>
                      <item>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                      <item>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                      <one-of>
                        <item>
                          <ruleref special='NULL'/>
                        </item>
                        <item>
                          symbol1
                          <one-of>
                            <item>
                              <ruleref uri='#rule2'/>
                            </item>
                          </one-of>
                          symbol3
                        </item>
                      </one-of>
                      <one-of>
                        <item>
                          <ruleref special='NULL'/>
                        </item>
                        <item>
                          symbol1
                          <one-of>
                            <item>
                              <ruleref uri='#rule2'/>
                            </item>
                          </one-of>
                          symbol3
                        </item>
                      </one-of>
                    </item>
                    <item>
                      <ruleref uri='#rule3'/>
                    </item>
                  </one-of>
                  symbol1
                </rule>
                Handling repeat requests...
                The modified grammar:
                <rule id='rule2'>
                  <item>
                    symbol2a
                  </item>
                  <item>
                    symbol2b
                  </item>
                  <item>
                    symbol2c
                  </item>
                </rule>
                Handling repeat requests...
                The modified grammar:
                <rule id='rule3'>
                  symbol6
                  <item>
                    <one-of>
                      <item>
                        symbol4
                      </item>
                      <item>
                        symbol5
                      </item>
                    </one-of>
                  </item>
                  symbol7
                </rule>
              performing conversion
              ABNF equivalent:
                word -> (symbol0 symbol1 (((symbol1 (RULE2) symbol3) (symbol1 (RULE2) symbol3) (((epsilon))|(symbol1 (RULE2) symbol3)|((epsilon))|(symbol1 (RULE2) symbol3)) (((epsilon))|(symbol1 (RULE2) symbol3)))|(RULE3)) symbol1)
              ABNF equivalent:
                rule2 -> ((symbol2a) (symbol2b) (symbol2c))
              ABNF equivalent:
                rule3 -> (symbol6 (((symbol4)|(symbol5))) symbol7)
      Processing file: /cavs/hse/ies/d002/isip/users/wholland/new_tools/class/pr/LanguageModelXML/diagnose_input_repeat_prob.sof
        reading model
          reading levels
            reading grammars
              Parsing the grammar...
              The parsed document:
                <grammar root='word'>
                  <rule id='word'>
                    symbol0
                    symbol1
                    <one-of>
                      <item repeat='1-' repeat-prob='0.98'>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                      <item>
                        <ruleref uri='#rule3'/>
                      </item>
                    </one-of>
                    symbol1
                  </rule>
                  <rule id='rule2'>
                    <item>
                      symbol2a
                    </item>
                    <item>
                      symbol2b
                    </item>
                    <item>
                      symbol2c
                    </item>
                  </rule>
                  <rule id='rule3'>
                    symbol6
                    <item>
                      <one-of>
                        <item>
                          symbol4
                        </item>
                        <item>
                          symbol5
                        </item>
                      </one-of>
                    </item>
                    symbol7
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='word'>
              |1     |2     |  <rule id='word'>
              |2     |2     |    symbol0
              |3     |2     |    symbol1
              |4     |3     |    <one-of>
              |5     |4     |      <item repeat='1-' repeat-prob='0.98'>
              |6     |4     |        symbol1
              |7     |5     |        <one-of>
              |8     |6     |          <item>
              |9     |6     |            <ruleref uri='#rule2'/>
              |10    |5     |          </item>
              |11    |4     |        </one-of>
              |12    |4     |        symbol3
              |13    |3     |      </item>
              |14    |4     |      <item>
              |15    |4     |        <ruleref uri='#rule3'/>
              |16    |3     |      </item>
              |17    |2     |    </one-of>
              |18    |2     |    symbol1
              |19    |1     |  </rule>
              |20    |2     |  <rule id='rule2'>
              |21    |3     |    <item>
              |22    |3     |      symbol2a
              |23    |2     |    </item>
              |24    |3     |    <item>
              |25    |3     |      symbol2b
              |26    |2     |    </item>
              |27    |3     |    <item>
              |28    |3     |      symbol2c
              |29    |2     |    </item>
              |30    |1     |  </rule>
              |31    |2     |  <rule id='rule3'>
              |32    |2     |    symbol6
              |33    |3     |    <item>
              |34    |4     |      <one-of>
              |35    |5     |        <item>
              |36    |5     |          symbol4
              |37    |4     |        </item>
              |38    |5     |        <item>
              |39    |5     |          symbol5
              |40    |4     |        </item>
              |41    |3     |      </one-of>
              |42    |2     |    </item>
              |43    |2     |    symbol7
              |44    |1     |  </rule>
              |45    |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='word'>
                  <rule id='word'>
                    symbol0
                    symbol1
                    <one-of>
                      <item repeat='1-' repeat-prob='0.98'>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                      <item>
                        <ruleref uri='#rule3'/>
                      </item>
                    </one-of>
                    symbol1
                  </rule>
                  <rule id='rule2'>
                    <item>
                      symbol2a
                    </item>
                    <item>
                      symbol2b
                    </item>
                    <item>
                      symbol2c
                    </item>
                  </rule>
                  <rule id='rule3'>
                    symbol6
                    <item>
                      <one-of>
                        <item>
                          symbol4
                        </item>
                        <item>
                          symbol5
                        </item>
                      </one-of>
                    </item>
                    symbol7
                  </rule>
                </grammar>
            reading level tag
              Parsing the grammar...
              The parsed document:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='level0'>
              |1     |2     |  <rule id='level0'>
              |2     |2     |    word
              |3     |1     |  </rule>
              |4     |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
            reading symbols
            reading context mapping
              <HashTable<Context,Ulong>::dummy context hash table:> num_items_d = 10
              <HashTable<Context,Ulong>::dummy context hash table:> load_factor_d = 0.75
              <HashTable<Context,Ulong>::dummy context hash table:> capacity_d = 128
              <HashTable<Context,Ulong>::dummy context hash table:> alloc_d = SYSTEM
                <CircularDelayLine<Ulong>::0> index_d = 0
                <CircularDelayLine<Ulong>::0> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 0
                  <Ulong::0> value_d = 0
                <CircularDelayLine<Ulong>::1> index_d = 0
                <CircularDelayLine<Ulong>::1> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 1
                  <Ulong::1> value_d = 1
                <CircularDelayLine<Ulong>::2> index_d = 0
                <CircularDelayLine<Ulong>::2> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 2
                  <Ulong::2> value_d = 2
                <CircularDelayLine<Ulong>::3> index_d = 0
                <CircularDelayLine<Ulong>::3> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 3
                  <Ulong::3> value_d = 3
                <CircularDelayLine<Ulong>::4> index_d = 0
                <CircularDelayLine<Ulong>::4> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 4
                  <Ulong::4> value_d = 4
                <CircularDelayLine<Ulong>::5> index_d = 0
                <CircularDelayLine<Ulong>::5> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 5
                  <Ulong::5> value_d = 5
                <CircularDelayLine<Ulong>::6> index_d = 0
                <CircularDelayLine<Ulong>::6> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 6
                  <Ulong::6> value_d = 6
                <CircularDelayLine<Ulong>::7> index_d = 0
                <CircularDelayLine<Ulong>::7> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 7
                  <Ulong::7> value_d = 7
                <CircularDelayLine<Ulong>::8> index_d = 0
                <CircularDelayLine<Ulong>::8> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 8
                  <Ulong::8> value_d = 8
                <CircularDelayLine<Ulong>::9> index_d = 0
                <CircularDelayLine<Ulong>::9> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 9
                  <Ulong::9> value_d = 9
        Level 0
          Graph 0
          <?xml version='1.0' encoding='utf-8'?>
          <grammar root='word'>
            <rule id='word'>
              symbol0
              symbol1
              <one-of>
                <item repeat='1-' repeat-prob='0.98'>
                  symbol1
                  <one-of>
                    <item>
                      <ruleref uri='#rule2'/>
                    </item>
                  </one-of>
                  symbol3
                </item>
                <item>
                  <ruleref uri='#rule3'/>
                </item>
              </one-of>
              symbol1
            </rule>
            <rule id='rule2'>
              <item> symbol2a </item>
              <item> symbol2b </item>
              <item> symbol2c </item>
            </rule>
            <rule id='rule3'>
              symbol6
              <item>
                <one-of>
                  <item> symbol4 </item>
                  <item> symbol5 </item>
                </one-of>
              </item>
              symbol7
            </rule>
          </grammar>
          
        performing XML->ABNF conversion
          level 0
            graph 0
              preprocessing
                Handling repeat requests...
                The modified grammar:
                <rule id='word'>
                  symbol0
                  symbol1
                  <one-of>
                    <item>
                      <item repeat-prob='0.98' repeat='1-'>
                        symbol1
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        symbol3
                      </item>
                    </item>
                    <item>
                      <ruleref uri='#rule3'/>
                    </item>
                  </one-of>
                  symbol1
                </rule>
                Handling repeat requests...
                The modified grammar:
                <rule id='rule2'>
                  <item>
                    symbol2a
                  </item>
                  <item>
                    symbol2b
                  </item>
                  <item>
                    symbol2c
                  </item>
                </rule>
                Handling repeat requests...
                The modified grammar:
                <rule id='rule3'>
                  symbol6
                  <item>
                    <one-of>
                      <item>
                        symbol4
                      </item>
                      <item>
                        symbol5
                      </item>
                    </one-of>
                  </item>
                  symbol7
                </rule>
              performing conversion
              ABNF equivalent:
                word -> (symbol0 symbol1 (((symbol1 (RULE2) symbol3)+)|(RULE3)) symbol1)
              ABNF equivalent:
                rule2 -> ((symbol2a) (symbol2b) (symbol2c))
              ABNF equivalent:
                rule3 -> (symbol6 (((symbol4)|(symbol5))) symbol7)
      Processing file: /cavs/hse/ies/d002/isip/users/wholland/new_tools/class/pr/LanguageModelXML/diagnose_input_ruleref.sof
        reading model
          reading levels
            reading grammars
              Parsing the grammar...
              The parsed document:
                <grammar root='word'>
                  <rule id='word'>
                    <item>
                      symbol0
                    </item>
                    <item>
                      symbol1
                    </item>
                    <one-of>
                      <item>
                        <item>
                          symbol1
                        </item>
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        <item>
                          symbol3
                        </item>
                      </item>
                      <item>
                        <ruleref uri='#rule3'/>
                      </item>
                    </one-of>
                    <item>
                      symbol1
                    </item>
                  </rule>
                  <rule id='rule2'>
                    <item>
                      symbol2a
                    </item>
                    <item>
                      symbol2b
                    </item>
                    <item>
                      symbol2c
                    </item>
                  </rule>
                  <rule id='rule3'>
                    <item>
                      symbol6
                    </item>
                    <item>
                      <one-of>
                        <item>
                          symbol4
                        </item>
                        <item>
                          symbol5
                        </item>
                      </one-of>
                    </item>
                    <item>
                      symbol7
                    </item>
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='word'>
              |1     |2     |  <rule id='word'>
              |2     |3     |    <item>
              |3     |3     |      symbol0
              |4     |2     |    </item>
              |5     |3     |    <item>
              |6     |3     |      symbol1
              |7     |2     |    </item>
              |8     |3     |    <one-of>
              |9     |4     |      <item>
              |10    |5     |        <item>
              |11    |5     |          symbol1
              |12    |4     |        </item>
              |13    |5     |        <one-of>
              |14    |6     |          <item>
              |15    |6     |            <ruleref uri='#rule2'/>
              |16    |5     |          </item>
              |17    |4     |        </one-of>
              |18    |5     |        <item>
              |19    |5     |          symbol3
              |20    |4     |        </item>
              |21    |3     |      </item>
              |22    |4     |      <item>
              |23    |4     |        <ruleref uri='#rule3'/>
              |24    |3     |      </item>
              |25    |2     |    </one-of>
              |26    |3     |    <item>
              |27    |3     |      symbol1
              |28    |2     |    </item>
              |29    |1     |  </rule>
              |30    |2     |  <rule id='rule2'>
              |31    |3     |    <item>
              |32    |3     |      symbol2a
              |33    |2     |    </item>
              |34    |3     |    <item>
              |35    |3     |      symbol2b
              |36    |2     |    </item>
              |37    |3     |    <item>
              |38    |3     |      symbol2c
              |39    |2     |    </item>
              |40    |1     |  </rule>
              |41    |2     |  <rule id='rule3'>
              |42    |3     |    <item>
              |43    |3     |      symbol6
              |44    |2     |    </item>
              |45    |3     |    <item>
              |46    |4     |      <one-of>
              |47    |5     |        <item>
              |48    |5     |          symbol4
              |49    |4     |        </item>
              |50    |5     |        <item>
              |51    |5     |          symbol5
              |52    |4     |        </item>
              |53    |3     |      </one-of>
              |54    |2     |    </item>
              |55    |3     |    <item>
              |56    |3     |      symbol7
              |57    |2     |    </item>
              |58    |1     |  </rule>
              |59    |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='word'>
                  <rule id='word'>
                    <item>
                      symbol0
                    </item>
                    <item>
                      symbol1
                    </item>
                    <one-of>
                      <item>
                        <item>
                          symbol1
                        </item>
                        <one-of>
                          <item>
                            <ruleref uri='#rule2'/>
                          </item>
                        </one-of>
                        <item>
                          symbol3
                        </item>
                      </item>
                      <item>
                        <ruleref uri='#rule3'/>
                      </item>
                    </one-of>
                    <item>
                      symbol1
                    </item>
                  </rule>
                  <rule id='rule2'>
                    <item>
                      symbol2a
                    </item>
                    <item>
                      symbol2b
                    </item>
                    <item>
                      symbol2c
                    </item>
                  </rule>
                  <rule id='rule3'>
                    <item>
                      symbol6
                    </item>
                    <item>
                      <one-of>
                        <item>
                          symbol4
                        </item>
                        <item>
                          symbol5
                        </item>
                      </one-of>
                    </item>
                    <item>
                      symbol7
                    </item>
                  </rule>
                </grammar>
            reading level tag
              Parsing the grammar...
              The parsed document:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='level0'>
              |1     |2     |  <rule id='level0'>
              |2     |2     |    word
              |3     |1     |  </rule>
              |4     |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
            reading symbols
            reading context mapping
              <HashTable<Context,Ulong>::dummy context hash table:> num_items_d = 10
              <HashTable<Context,Ulong>::dummy context hash table:> load_factor_d = 0.75
              <HashTable<Context,Ulong>::dummy context hash table:> capacity_d = 128
              <HashTable<Context,Ulong>::dummy context hash table:> alloc_d = SYSTEM
                <CircularDelayLine<Ulong>::0> index_d = 0
                <CircularDelayLine<Ulong>::0> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 0
                  <Ulong::0> value_d = 0
                <CircularDelayLine<Ulong>::1> index_d = 0
                <CircularDelayLine<Ulong>::1> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 1
                  <Ulong::1> value_d = 1
                <CircularDelayLine<Ulong>::2> index_d = 0
                <CircularDelayLine<Ulong>::2> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 2
                  <Ulong::2> value_d = 2
                <CircularDelayLine<Ulong>::3> index_d = 0
                <CircularDelayLine<Ulong>::3> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 3
                  <Ulong::3> value_d = 3
                <CircularDelayLine<Ulong>::4> index_d = 0
                <CircularDelayLine<Ulong>::4> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 4
                  <Ulong::4> value_d = 4
                <CircularDelayLine<Ulong>::5> index_d = 0
                <CircularDelayLine<Ulong>::5> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 5
                  <Ulong::5> value_d = 5
                <CircularDelayLine<Ulong>::6> index_d = 0
                <CircularDelayLine<Ulong>::6> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 6
                  <Ulong::6> value_d = 6
                <CircularDelayLine<Ulong>::7> index_d = 0
                <CircularDelayLine<Ulong>::7> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 7
                  <Ulong::7> value_d = 7
                <CircularDelayLine<Ulong>::8> index_d = 0
                <CircularDelayLine<Ulong>::8> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 8
                  <Ulong::8> value_d = 8
                <CircularDelayLine<Ulong>::9> index_d = 0
                <CircularDelayLine<Ulong>::9> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 9
                  <Ulong::9> value_d = 9
        Level 0
          Graph 0
          <?xml version='1.0' encoding='utf-8'?>
          <grammar root='word'>
            <rule id='word'>
              <item> symbol0 </item>
              <item> symbol1 </item>
              <one-of>
                <item>
                  <item> symbol1 </item>
                  <one-of>
                    <item>
                      <ruleref uri='#rule2'/>
                    </item>
                  </one-of>
                  <item> symbol3 </item>
                </item>
                <item>
                  <ruleref uri='#rule3'/>
                </item>
              </one-of>
              <item> symbol1 </item>
            </rule>
            <rule id='rule2'>
              <item> symbol2a </item>
              <item> symbol2b </item>
              <item> symbol2c </item>
            </rule>
            <rule id='rule3'>
              <item> symbol6 </item>
              <item>
                <one-of>
                  <item> symbol4 </item>
                  <item> symbol5 </item>
                </one-of>
              </item>
              <item> symbol7 </item>
            </rule>
          </grammar>
          
        performing XML->ABNF conversion
          level 0
            graph 0
              preprocessing
                Handling repeat requests...
                The modified grammar:
                <rule id='word'>
                  <item>
                    symbol0
                  </item>
                  <item>
                    symbol1
                  </item>
                  <one-of>
                    <item>
                      <item>
                        symbol1
                      </item>
                      <one-of>
                        <item>
                          <ruleref uri='#rule2'/>
                        </item>
                      </one-of>
                      <item>
                        symbol3
                      </item>
                    </item>
                    <item>
                      <ruleref uri='#rule3'/>
                    </item>
                  </one-of>
                  <item>
                    symbol1
                  </item>
                </rule>
                Handling repeat requests...
                The modified grammar:
                <rule id='rule2'>
                  <item>
                    symbol2a
                  </item>
                  <item>
                    symbol2b
                  </item>
                  <item>
                    symbol2c
                  </item>
                </rule>
                Handling repeat requests...
                The modified grammar:
                <rule id='rule3'>
                  <item>
                    symbol6
                  </item>
                  <item>
                    <one-of>
                      <item>
                        symbol4
                      </item>
                      <item>
                        symbol5
                      </item>
                    </one-of>
                  </item>
                  <item>
                    symbol7
                  </item>
                </rule>
              performing conversion
              ABNF equivalent:
                word -> ((symbol0) (symbol1) (((symbol1) ((RULE2)|(symbol3)) (symbol3))|(RULE3)|(symbol1)) (symbol1))
              ABNF equivalent:
                rule2 -> ((symbol2a) (symbol2b) (symbol2c))
              ABNF equivalent:
                rule3 -> ((symbol6) (((symbol4)|(symbol5))) (symbol7))
      Processing file: /cavs/hse/ies/d002/isip/users/wholland/new_tools/class/pr/LanguageModelXML/diagnose_input_ruleref_null.sof
        reading model
          reading levels
            reading grammars
              Parsing the grammar...
              The parsed document:
                <grammar root='specials' isip_rule_null='slowcoach'>
                  <rule id='specials'>
                    <item>
                      symbol0
                    </item>
                    <item>
                      symbol1
                    </item>
                    <one-of>
                      <item>
                        <item>
                          symbol1
                        </item>
                        <one-of>
                          <item>
                            <ruleref special='NULL'/>
                          </item>
                          <item>
                            symbol2
                          </item>
                        </one-of>
                        <item>
                          symbol3
                        </item>
                      </item>
                    </one-of>
                    <item>
                      symbol1
                    </item>
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='specials' isip_rule_null='slowcoach'>
              |1     |2     |  <rule id='specials'>
              |2     |3     |    <item>
              |3     |3     |      symbol0
              |4     |2     |    </item>
              |5     |3     |    <item>
              |6     |3     |      symbol1
              |7     |2     |    </item>
              |8     |3     |    <one-of>
              |9     |4     |      <item>
              |10    |5     |        <item>
              |11    |5     |          symbol1
              |12    |4     |        </item>
              |13    |5     |        <one-of>
              |14    |6     |          <item>
              |15    |6     |            <ruleref special='NULL'/>
              |16    |5     |          </item>
              |17    |6     |          <item>
              |18    |6     |            symbol2
              |19    |5     |          </item>
              |20    |4     |        </one-of>
              |21    |5     |        <item>
              |22    |5     |          symbol3
              |23    |4     |        </item>
              |24    |3     |      </item>
              |25    |2     |    </one-of>
              |26    |3     |    <item>
              |27    |3     |      symbol1
              |28    |2     |    </item>
              |29    |1     |  </rule>
              |30    |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='specials' isip_rule_null='slowcoach'>
                  <rule id='specials'>
                    <item>
                      symbol0
                    </item>
                    <item>
                      symbol1
                    </item>
                    <one-of>
                      <item>
                        <item>
                          symbol1
                        </item>
                        <one-of>
                          <item>
                            <ruleref special='NULL'/>
                          </item>
                          <item>
                            symbol2
                          </item>
                        </one-of>
                        <item>
                          symbol3
                        </item>
                      </item>
                    </one-of>
                    <item>
                      symbol1
                    </item>
                  </rule>
                </grammar>
            reading level tag
              Parsing the grammar...
              The parsed document:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='level0'>
              |1     |2     |  <rule id='level0'>
              |2     |2     |    word
              |3     |1     |  </rule>
              |4     |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
            reading symbols
            reading context mapping
              <HashTable<Context,Ulong>::dummy context hash table:> num_items_d = 4
              <HashTable<Context,Ulong>::dummy context hash table:> load_factor_d = 0.75
              <HashTable<Context,Ulong>::dummy context hash table:> capacity_d = 128
              <HashTable<Context,Ulong>::dummy context hash table:> alloc_d = SYSTEM
                <CircularDelayLine<Ulong>::0> index_d = 0
                <CircularDelayLine<Ulong>::0> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 0
                  <Ulong::0> value_d = 0
                <CircularDelayLine<Ulong>::1> index_d = 0
                <CircularDelayLine<Ulong>::1> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 1
                  <Ulong::1> value_d = 1
                <CircularDelayLine<Ulong>::2> index_d = 0
                <CircularDelayLine<Ulong>::2> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 2
                  <Ulong::2> value_d = 2
                <CircularDelayLine<Ulong>::3> index_d = 0
                <CircularDelayLine<Ulong>::3> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 3
                  <Ulong::3> value_d = 3
        Level 0
          Graph 0
          <?xml version='1.0' encoding='utf-8'?>
          <grammar root='specials' isip_rule_null='slowcoach'>
            <rule id='specials'>
              <item> symbol0 </item>
              <item> symbol1 </item>
              <one-of>
                <item>
                  <item> symbol1 </item>
                  <one-of>
                    <item>
                      <ruleref special='NULL'/>
                    </item>
                    <item> symbol2 </item>
                  </one-of>
                  <item> symbol3 </item>
                </item>
              </one-of>
              <item> symbol1 </item>
            </rule>
          </grammar>
          
        performing XML->ABNF conversion
          level 0
            graph 0
              preprocessing
                Handling repeat requests...
                The modified grammar:
                <rule id='specials'>
                  <item>
                    symbol0
                  </item>
                  <item>
                    symbol1
                  </item>
                  <one-of>
                    <item>
                      <item>
                        symbol1
                      </item>
                      <one-of>
                        <item>
                          <ruleref special='NULL'/>
                        </item>
                        <item>
                          symbol2
                        </item>
                      </one-of>
                      <item>
                        symbol3
                      </item>
                    </item>
                  </one-of>
                  <item>
                    symbol1
                  </item>
                </rule>
              performing conversion
              ABNF equivalent:
                specials -> ((symbol0) (symbol1) (((symbol1) (((epsilon))|(symbol2)|(symbol3)) (symbol3))|(symbol1)) (symbol1))
      Processing file: /cavs/hse/ies/d002/isip/users/wholland/new_tools/class/pr/LanguageModelXML/diagnose_input_simple.sof
        reading model
          reading levels
            reading grammars
              Parsing the grammar...
              The parsed document:
                <grammar root='word'>
                  <rule id='word'>
                    <item>
                      worda
                    </item>
                    <item>
                      wordb
                    </item>
                    <item>
                      wordc
                    </item>
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='word'>
              |1     |2     |  <rule id='word'>
              |2     |3     |    <item>
              |3     |3     |      worda
              |4     |2     |    </item>
              |5     |3     |    <item>
              |6     |3     |      wordb
              |7     |2     |    </item>
              |8     |3     |    <item>
              |9     |3     |      wordc
              |10    |2     |    </item>
              |11    |1     |  </rule>
              |12    |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='word'>
                  <rule id='word'>
                    <item>
                      worda
                    </item>
                    <item>
                      wordb
                    </item>
                    <item>
                      wordc
                    </item>
                  </rule>
                </grammar>
            reading level tag
              Parsing the grammar...
              The parsed document:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='level0'>
              |1     |2     |  <rule id='level0'>
              |2     |2     |    word
              |3     |1     |  </rule>
              |4     |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
            reading symbols
            reading context mapping
              <HashTable<Context,Ulong>::dummy context hash table:> num_items_d = 3
              <HashTable<Context,Ulong>::dummy context hash table:> load_factor_d = 0.75
              <HashTable<Context,Ulong>::dummy context hash table:> capacity_d = 128
              <HashTable<Context,Ulong>::dummy context hash table:> alloc_d = SYSTEM
                <CircularDelayLine<Ulong>::0> index_d = 0
                <CircularDelayLine<Ulong>::0> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 0
                  <Ulong::0> value_d = 0
                <CircularDelayLine<Ulong>::1> index_d = 0
                <CircularDelayLine<Ulong>::1> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 1
                  <Ulong::1> value_d = 1
                <CircularDelayLine<Ulong>::2> index_d = 0
                <CircularDelayLine<Ulong>::2> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 2
                  <Ulong::2> value_d = 2
        Level 0
          Graph 0
          <?xml version='1.0' encoding='utf-8'?>
          <grammar root='word'>
            <rule id='word'>
              <item> worda </item>
              <item> wordb </item>
              <item> wordc </item>
            </rule>
          </grammar>
          
        performing XML->ABNF conversion
          level 0
            graph 0
              preprocessing
                Handling repeat requests...
                The modified grammar:
                <rule id='word'>
                  <item>
                    worda
                  </item>
                  <item>
                    wordb
                  </item>
                  <item>
                    wordc
                  </item>
                </rule>
              performing conversion
              ABNF equivalent:
                word -> ((worda) (wordb) (wordc))
      Processing file: /cavs/hse/ies/d002/isip/users/wholland/new_tools/class/pr/LanguageModelXML/diagnose_input_weights.sof
        reading model
          reading levels
            reading grammars
              Parsing the grammar...
              The parsed document:
                <grammar root='word'>
                  <rule id='word'>
                    <one-of>
                      <item weight='0.1'>
                        worda
                      </item>
                      <item weight='0.2'>
                        wordb
                      </item>
                      <item weight='0.3'>
                        wordc
                      </item>
                    </one-of>
                    <item weight='0.5'>
                      wordz
                    </item>
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='word'>
              |1     |2     |  <rule id='word'>
              |2     |3     |    <one-of>
              |3     |4     |      <item weight='0.1'>
              |4     |4     |        worda
              |5     |3     |      </item>
              |6     |4     |      <item weight='0.2'>
              |7     |4     |        wordb
              |8     |3     |      </item>
              |9     |4     |      <item weight='0.3'>
              |10    |4     |        wordc
              |11    |3     |      </item>
              |12    |2     |    </one-of>
              |13    |3     |    <item weight='0.5'>
              |14    |3     |      wordz
              |15    |2     |    </item>
              |16    |1     |  </rule>
              |17    |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='word'>
                  <rule id='word'>
                    <one-of>
                      <item weight='0.1'>
                        worda
                      </item>
                      <item weight='0.2'>
                        wordb
                      </item>
                      <item weight='0.3'>
                        wordc
                      </item>
                    </one-of>
                    <item weight='0.5'>
                      wordz
                    </item>
                  </rule>
                </grammar>
            reading level tag
              Parsing the grammar...
              The parsed document:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='level0'>
              |1     |2     |  <rule id='level0'>
              |2     |2     |    word
              |3     |1     |  </rule>
              |4     |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
            reading symbols
            reading context mapping
              <HashTable<Context,Ulong>::dummy context hash table:> num_items_d = 4
              <HashTable<Context,Ulong>::dummy context hash table:> load_factor_d = 0.75
              <HashTable<Context,Ulong>::dummy context hash table:> capacity_d = 128
              <HashTable<Context,Ulong>::dummy context hash table:> alloc_d = SYSTEM
                <CircularDelayLine<Ulong>::0> index_d = 0
                <CircularDelayLine<Ulong>::0> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 0
                  <Ulong::0> value_d = 0
                <CircularDelayLine<Ulong>::1> index_d = 0
                <CircularDelayLine<Ulong>::1> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 1
                  <Ulong::1> value_d = 1
                <CircularDelayLine<Ulong>::2> index_d = 0
                <CircularDelayLine<Ulong>::2> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 2
                  <Ulong::2> value_d = 2
                <CircularDelayLine<Ulong>::3> index_d = 0
                <CircularDelayLine<Ulong>::3> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 3
                  <Ulong::3> value_d = 3
        Level 0
          Graph 0
          <?xml version='1.0' encoding='utf-8'?>
          <grammar root='word'>
            <rule id='word'>
              <one-of>
                <item weight='0.1'> worda </item>
                <item weight='0.2'> wordb </item>
                <item weight='0.3'> wordc </item>
              </one-of>
              <item weight='0.5'> wordz </item>
            </rule>
          </grammar>
          
        performing XML->ABNF conversion
          level 0
            graph 0
              preprocessing
                Handling repeat requests...
                The modified grammar:
                <rule id='word'>
                  <one-of>
                    <item weight='0.1'>
                      worda
                    </item>
                    <item weight='0.2'>
                      wordb
                    </item>
                    <item weight='0.3'>
                      wordc
                    </item>
                  </one-of>
                  <item weight='0.5'>
                    wordz
                  </item>
                </rule>
              performing conversion
              ABNF equivalent:
                word -> (((worda)|(wordb)|(wordc)|(wordz)) (wordz))
      Processing file: /cavs/hse/ies/d002/isip/users/wholland/new_tools/class/pr/LanguageModelXML/diagnose_input_state.sof
        reading model
          reading levels
            reading grammars
              Parsing the grammar...
              The parsed document:
                <grammar root='word'>
                  <rule id='word'>
                    <item repeat='1-'>
                      S12
                    </item>
                    <item repeat='1-' repeat-prob='.2' weight='.4'>
                      S13
                    </item>
                    <item repeat='1-'>
                      S14
                    </item>
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='word'>
              |1     |2     |  <rule id='word'>
              |2     |3     |    <item repeat='1-'>
              |3     |3     |      S12
              |4     |2     |    </item>
              |5     |3     |    <item repeat='1-' repeat-prob='.2' weight='.4'>
              |6     |3     |      S13
              |7     |2     |    </item>
              |8     |3     |    <item repeat='1-'>
              |9     |3     |      S14
              |10    |2     |    </item>
              |11    |1     |  </rule>
              |12    |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='word'>
                  <rule id='word'>
                    <item repeat='1-'>
                      S12
                    </item>
                    <item repeat='1-' repeat-prob='.2' weight='.4'>
                      S13
                    </item>
                    <item repeat='1-'>
                      S14
                    </item>
                  </rule>
                </grammar>
            reading level tag
              Parsing the grammar...
              The parsed document:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
              XMLParser: parsed token information:
              |input |depth | xml form of input
              |0     |1     |<grammar root='level0'>
              |1     |2     |  <rule id='level0'>
              |2     |2     |    word
              |3     |1     |  </rule>
              |4     |0     |</grammar>
              The document after removal of unwanted tags:
                <grammar root='level0'>
                  <rule id='level0'>
                    word
                  </rule>
                </grammar>
            reading symbols
            reading context mapping
              <HashTable<Context,Ulong>::dummy context hash table:> num_items_d = 3
              <HashTable<Context,Ulong>::dummy context hash table:> load_factor_d = 0.75
              <HashTable<Context,Ulong>::dummy context hash table:> capacity_d = 128
              <HashTable<Context,Ulong>::dummy context hash table:> alloc_d = SYSTEM
                <CircularDelayLine<Ulong>::0> index_d = 0
                <CircularDelayLine<Ulong>::0> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 0
                  <Ulong::0> value_d = 0
                <CircularDelayLine<Ulong>::1> index_d = 0
                <CircularDelayLine<Ulong>::1> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 1
                  <Ulong::1> value_d = 1
                <CircularDelayLine<Ulong>::2> index_d = 0
                <CircularDelayLine<Ulong>::2> debug_level_d = none
                delay line:
                  <Vector<Ulong>::v_d> length_d = 1
                  <Vector<Ulong>::v_d> capacity_d = 1
                  <Vector<Ulong>::v_d> v_d[0]
                    <Ulong::> value_d = 2
                  <Ulong::2> value_d = 2
        Level 0
          Graph 0
          <?xml version='1.0' encoding='utf-8'?>
          <grammar root='word'>
            <rule id='word'>
              <item repeat='1-'> S12 </item>
              <item repeat='1-' repeat-prob='.2' weight='.4'> S13 </item>
              <item repeat='1-'> S14 </item>
            </rule>
          </grammar>
          
        performing XML->ABNF conversion
          level 0
            graph 0
              preprocessing
                Handling repeat requests...
                The modified grammar:
                <rule id='word'>
                  <item>
                    <item repeat='1-'>
                      S12
                    </item>
                  </item>
                  <item>
                    <item repeat-prob='.2' weight='.4' repeat='1-'>
                      S13
                    </item>
                  </item>
                  <item>
                    <item repeat='1-'>
                      S14
                    </item>
                  </item>
                </rule>
              performing conversion
              ABNF equivalent:
                word -> (((s12)+) ((s13)+) ((s14)+))
    performing BNF->XML conversion
      level 0
        graph 0
          XML equivalent:
            <?xml version='1.0' encoding='utf-8'?>
            <grammar root='S'>
              <rule id='S'>
                one
                <ruleref uri='#TWO'/>
              </rule>
              <rule id='TWO'>
                <one-of>
                  <item>
                    two
                    <ruleref uri='#TWO'/>
                  </item>
                  <item>
                    <ruleref special='NULL'/>
                  </item>
                </one-of>
              </rule>
            </grammar>
        diagnostics passed for class LanguageModelXML
      tests passed for class LanguageModelXML
