Changing the tokenizer to get rid of IDENT, KEYWORD and METAVAR

token categories, replaced by a single one called SYMTOK.
Package prefixes are now recognized and processed in tokens.

* lib.c (delete_package): Fix problem in no-such-package
error case: it would always report nil as the name.
(intern): Fix nonsensical error message: in the no-such-package case it
would report that the symbol exists already.

* parser.l (grammar): Occurences of KEYWORD, METAVAR, and IDENT
scrubbed. All rules reporting any of these now return
SYMTOK. The main one of these is greatly simplified.

* parser.y (sym_helper): New function.
(char_from_name): const qualifier inside param's type declaration.
(grammar): IDENT, KEYWORD and METAVAR tokens are gone.
New token SYMTOK. Grammar refactored around SYMTOK and using
the new sym_helper function.
(char_from_name): Updated.

1 files changed, 5 insertions, 14 deletions

diff --git a/parser.l b/parser.l
index 78e29408..ef92db03 100644
--- a/parser.l
+++ b/parser.l
@@ -260,17 +260,8 @@ UONLY   {U2}{U}|{U3}{U}{U}|{U4}{U}{U}{U}
       || yy_top_state() == QSILIT)
     yy_pop_state();
 
-  switch (yytext[0]) {
-  case ':':
-    yylval.lexeme = utf8_dup_from(yytext + 1);
-    return KEYWORD;
-  case '@':
-    yylval.lexeme = utf8_dup_from(yytext + 1);
-    return METAVAR;
-  default:
-    yylval.lexeme = utf8_dup_from(yytext);
-    return IDENT;
-  }
+  yylval.lexeme = utf8_dup_from(yytext);
+  return SYMTOK;
 }
 
 <BRACED,NESTED>: {
@@ -278,7 +269,7 @@ UONLY   {U2}{U}|{U3}{U}{U}|{U4}{U}{U}{U}
       || yy_top_state() == QSILIT)
     yy_pop_state();
   yylval.lexeme = utf8_dup_from("");
-  return KEYWORD;
+  return SYMTOK;
 }
 
 <SPECIAL>\({WS}all{WS}\) {
@@ -732,12 +723,12 @@ UONLY   {U2}{U}|{U3}{U}{U}|{U4}{U}{U}{U}
 
 <CHRLIT>{SYM} {
   yylval.lexeme = utf8_dup_from(yytext);
-  return IDENT;
+  return SYMTOK;
 }
 
 <CHRLIT>[^ \t\n] {
   yylval.lexeme = utf8_dup_from(yytext);
-  return IDENT; /* hack */
+  return SYMTOK; /* hack */
 }
 
 <STRLIT>\n {