Initial commit

1 files changed, 263 insertions, 0 deletions

diff --git a/web/yacco2/library/errhdling.w b/web/yacco2/library/errhdling.w
new file mode 100644
index 0000000000..ff1a706775
--- /dev/null
+++ b/web/yacco2/library/errhdling.w
@@ -0,0 +1,263 @@
+@q  file: errhdling.w @>
+@q%   Copyright Dave Bone 1998 - 2015@>
+@q% /*@>
+@q%    This Source Code Form is subject to the terms of the Mozilla Public@>
+@q%    License, v. 2.0. If a copy of the MPL was not distributed with this@>
+@q%    file, You can obtain one at http://mozilla.org/MPL/2.0/.@>
+@q% */@>
+
+@** Error detection and handling.\fbreak
+Let's review how this can be done.
+Within a grammar's production there are points where  an invalid symbol
+could  arrive.
+If one does not program for it, the parser will go kapout.
+So what are the options open to a grammar writer?
+First there is a ``{\bf failed}'' directive in the ``fsm'' construct
+that will field aborted parses.
+It is the last chance to deal with errors in a rather 
+insensitive way.
+If  there are many contexts within the grammar that could go
+wrong then this approach is too insensitive to be specific 
+about the context's  
+error point.
+Though  the errant current token is available
+to report on, what was the inapproproiate context that threw it?
+Well u could try to figure it out from the remnants on the parse stack.
+
+To deal with specific error points, the \QUEshift{}, \ALLshift{}, 
+and \INVshift{} symbols can catch
+errant tokens, or one can be very specific in specifiying the errant T to catch.
+This last option can be very daunting when one has 500+ T to deal with
+and lets be honest not really appropriate.
+This was why i introduced the meta-terminals \QUEshift{} and \ALLshift.
+To catch a rogue and associate  syntax directed code to handle the situation,
+these symbols MUST be within prefix subrules where they are the 
+last symbol in the subrule's symbol string.
+What does this mean?
+Having a string of symbols where these catch T symbols are burried
+ within a larger symbol string
+means the subrule's containing these symbols 
+will not be executed as its sentence has not been
+completely recognized.
+For example:\fbreak
+\INDENT{.4in}{\subrule{} a \QUEshift{} b --- will not handle the error
+at the \QUEshift{} point}
+\INDENT{.4in}{\subrule{} a Rqueshift b --- will catch the problem}
+\INDENT{.7in}{Rule Rqueshift \subrule{} \QUEshift{}...}
+\INDENT{.9in}{will catch the error with appropriate syntax directed code directive }
+\fbreak
+Caution: The ranking of meta-terminal shifts: 1 and a 
+2 and a 3 ---\QUEshift{}, \INVshift{}, \ALLshift{}\fbreak
+The \QUEshift{} symbol is checked first for its presence within the current parse state
+followed by the \INVshift{} symbol as it is normally used to
+get out of a quasi-ambigous parse. The \ALLshift{} aka wild shifter 
+is the last to be checked in the parse state.
+It is their presence within the parse state that activates their use.
+The \QUEshift{} is an error statement and was my reason to put it at the head of the 
+conditional shifts.
+So watch your shifts as this could catch u like me.
+Remove 1 of the 2 competing shift symbols: \ALLshift or \INVshift. For the moment i
+have not issued an error message on this situation.
+@^ To do error message conditional shift ranking both \ALLshift{} and \INVshift{} in state@> 
+@^ To do error message conditional shift \INVshift{} takes precedence over \ALLshift{}@> 
+\fbreak
+\fbreak 
+Dictate no 1: Last symbol in subrule's symbol string must be the catcher in the Error\fbreak
+Make sure your error catch point has \QUEshift{} or \ALLshift{} as its 
+last symbol within the symbol string and let your syntax directed code
+decree the error escape route to be taken.
+Yeah that's fine but what if the symbol string to be recognized 
+contains many catch points?
+Just make each symbol string segment a separate rule with the error code catch point
+being the last symbol in the string competing with its legitimate accepted T symbols and
+use these rules  within another rule's subrule as part of its symbol string to be recognized!
+The lr algorithm is a collection of various symbol string configurations
+per state
+in various accepted T points along their parsing.
+So by transitive closure these prefix rules get included in the state to
+be  
+recognized along with the other similar prefix symbols.
+When the prefix rule's ```rhs'' boundary is recognized, depending
+on the error catcher used, the reduce will fire either in good form or
+as an error.
+\fbreak
+\fbreak
+What to do when an error is detected?\fbreak
+For now i have not thought out error correction strategies though i am marginally
+aware of the backtracking techniques.
+I will now discuss current programming options open to the grammar writer.
+Depending on the context, the thread could abort which is the most drastic.
+This takes place when no error catching is programmed
+and \O2 issues a runtime message on the aborted grammar with its run stack goodies.
+This might be okay to get things going but isn't too appropriate within a 
+production environment.
+Well the catch points have 2 programming options available:\fbreak
+\INDENT{.4in}{1) return an error token back to the calling grammar
+and stop parsing of the active grammar}
+\INDENT{.4in}{2) abort the parse and field it using the ``failed'' 
+directive to  return an error T}
+Point 1 should be your main course of action.
+That is both macros |RSVP| and |RSVP_FSM|  return a T back to the calling
+grammar through the accept queue facility
+as if the parse was successfull. This is what point 2 does using the |RSVP_FSM| macro
+as its execution is within the ``fsm'' context of the grammar and not the reducing rule.
+The calling grammar can then field this returned T specifically
+or use the two meta-terminal \QUEshift{} or \ALLshift{} to deal with them.
+They are allowed in any subrule symbol string  context: 
+thread calls where its returned T can be one of these symbols, and the
+regular subrule symbol string.
+\fbreak
+\fbreak
+Pinpointing where the error occured in the source file\fbreak
+Built into \O2 is the facility to tag each T with its approriate source file's GPS --- 
+filename, line number, and character position. These co-ordinates
+are used to print out the errant source line
+ with an arrow underlining the errant source token.
+So when an error T is created, use of the |set_rc| and variants allows one to pinpoint the 
+error T against the GPS's source file T. 
+Have a read on ``Abstract symbol class for all symbols'' --- |CAbs_lr1_sym|.
+\fbreak
+\fbreak
+Some subtleties on making the errant T fire off the error
+catching syntact directed code.\fbreak
+Let me pose a question: What happens when the errant T is not in the lookahead set
+to reduce that subrule? Well it will not get executed! Ugh. This is just 
+not acceptable Dave.
+Well to the rescue is the \QUEshift{} symbol.
+It is not in the token stream but represents an errant situation.
+So where is this errant T placed?
+When one enters the subrule's syntact directed code segment, 
+all its subrule's elements have been shifted onto the parse stack
+where this last errant symbol is  represented by \QUEshift{}.
+But the \QUEshift{} symbol {\bf  does not advance past the errant T}
+ as in regular parsing. 
+So what does this mean?
+The current errant T is also the lookahead symbol for the reduction.
+But wait what if this T is not in the lookahead set to reduce this subrule.
+Well i made this type of reduce a lr(0) context: no lookahead symbol required
+to reduce the subrule.
+
+To get at the current elements on the parse stack,
+\O2 emits within each subrule's c++ code the stack frame with each 
+subrule's symbol string assigned to ``|sf->pxx__|'' where xx is the symbol's
+string position.
+This is the difference to \ALLshift: \ALLshift{} depends on the lookahead set to reduce.
+Now what then is the advantage to using \ALLshift?
+One can test its under-its-hood T's enumerate value and then take error action or
+stop use of the \ALLshift{} facility that allows the grammar
+to continue parsing up to the ``start rule''.
+As it's a wild symbol shifter, it really lowers the grammar's parse tables sizes
+and eases the grammar writer's typing.
+\fbreak
+\fbreak
+Dictate no 2: Games on returning the new lookahead T back to the calling grammar\fbreak
+U can play games with resetting the new lookahead T that is passed back
+ with its |RSVP| T companion within the accept queue.
+This is what happens when 
+just 1 T is returned: the lookahead T is  the parse stream continue point
+and  also its  contents to set the calling parser's current token to continue with. 
+As an aside why use the returned lookahead's T  contents
+instead of just resetting the continue T from the token stream's
+container 
+using the lookahead token position?
+Well u could also remap the current token into another T type
+due to say a symbol table remapping --- like Pascal and its ``const-id'', ''function-id''
+as  described in the railroad diagrams of ``The Pascal Reference Manual''.
+The remapping facility is open for use via the ``Table lookup functor'' facility.
+The following methods adjust the parser's token stream:\fbreak
+\INDENT{.4in}{|override_current_token_pos(symbol,position)|}
+\INDENT{.4in}{|override_current_token(symbol)|}
+\INDENT{.4in}{|reset_current_token(position)|}
+
+In a dual competing threads situation
+where  each grammar have accepted their 
+parse and are returning their booty to the calling grammar,
+the calling grammar must use arbitration to select the T gift and
+sets its parse stream accordingly and the balance in the ``accept queue''
+are so-to-speak thrown away. 
+Of course the {\bf arbitration} facility 
+is programmed by the compiler writer when 2 or more successfull threads are
+returning their booty back to the calling grammar.
+Normally this does not occur as there is just one thread that will
+report its findings but this city is built on rock and nondeterminism.
+So a subset / superset competition, or an accept and error combo
+ is quite acceptable and for the arbitrator's
+choosing. Forgotten arbitration code  will be regurgitated by the \O2 library
+in message form
+for your fixing.
+
+The one caveat to watch for is: What is the current 
+token and its position in the parse stream 
+when it enters the subrule's syntax directed code?
+\QUEshift{} still has the errant T as its current T and to reset back
+to the previous T u only subtract 1 from the current token position.
+\ALLshift{} demands 2 be subtracted as the current T is the new lookahead T.
+So u've been warned.
+\fbreak
+\fbreak
+Some comments on stopping a parse by syntax directed code:\fbreak
+Apart from the don't do anything approach, the grammar writer can talk to the parser
+and dictate
+his intentions. The 2 methods open are abort-the-parse or stop-parsing.
+The abort-the-parse action allows the thread to stop without any T returned to 
+the caller grammar
+or  
+ use the {\bf failed} directive to last-chance return an  error T back to the caller.
+The stop-parsing approach returns a T back to the user but does not want
+to continue the complete parse through to its ``start rule''.
+It just short-circuits the overall grammar's parsing action.
+Remember that if the parse has been successfull ``why complete the parsing 
+thru to start-rule?''.
+Depending on your local grammar logic this might be the most expedient way to program.
+Here are the 2 methods to do this:\fbreak
+\INDENT{.4in}{|set_abort_parse(true)|}
+\INDENT{.4in}{|set_stop_parse(true)|}
+What about the reducing of this subrule?
+Well it occurs, as entry into the syntax directed code that
+contains the grammar writer's code to execute these  statements 
+are kosher reducing conditions.
+So why the ``abort-parse'' versus ``stop-parse'' difference.
+``stop-parse'' should contain the |RSVP| macro that
+enters the returned T into the calling grammar's ``accept-queue''.
+The ``abort-parse'' normally does not contain this action.
+\fbreak
+\fbreak
+Warning no 3: if \ALLshift{} being used, don't forget to turn it off.\fbreak
+This symbol is voracious: eats and  eats everything in its path.
+So u can arrive at trying to eat the ``end-of-the-parse-stream'' ``|eog|'' symbol
+forever...
+\O2 guards against this but is rather abrupt in its message to the grammar writer
+and stopping
+of the parse immediately. 
+So u'll see in some the suggested grammars  |set_use_all_shift_off| method being 
+called to get out of this perpetual motion and possiblely continue up the parse chain to the 
+``start rule''.
+Here is  a list of some \O2 grammars having error handling and premature
+ stopping of a parse to learn from.
+\INDENT{.4in}{1) |o2_lcl_opts.lex| and called thread |o2_lcl_opt.lex| --- command line parser}
+\INDENT{.4in}{2) |la_express.lex| --- |set_abort_parse(true)| thread's la expression parser}
+\INDENT{.4in}{3) |c_string.lex| --- semantic example stopping a parse and programmed fsa}
+Point 1 gives an example of how the ``failed'' directive in the called thread |o2_lcl_opt.lex|
+is programmed  and ``|set_stop_parse(true)|'' use in the calling grammar |o2_lcl_opts.lex|
+of a monolitic grammar. |pass3.lex| and point 2 give more examples on monolithic use
+to aborting. Point 3 also shows programming use of the ``|set_abort_parse(true)|''.
+For the really curious, why not use the find/grep/xargs combo to settle your appetite against
+\O2's grammars.
+\fbreak
+\fbreak
+The last word, amen and  happy parsing.\fbreak
+Remember that the normal flow of errors should be placed into the ``error queue'' and
+then post processed to report its findings. 
+|ADD_TOKEN_TO_ERROR_QUEUE| and its variant |FSM_ADD_TOKEN_TO_ERROR_QUEUE| 
+allow u to do this. |pass3.lex| gives lots of examples and \O2's program shows its
+way of post-verbing the troubles. 
+And with all this error stutter, each grammar does a post-execution  grammar  
+cleanup on current parsing
+for the next round of their calling. Again what does this mean?
+A semi-abort was done just to stop its execution leaving the grammar in an abort state. 
+But
+each grammar does a resetting to a clean slate for its next round of
+calling either by ``procedure call'' if no nesting calls of itself is occuring
+or by the heavy thread call.
+Hygiene is important so the cat washes itself for the next eating.
+