[TheHeadFL]

That would probably be a good way to structure the core of the lexical analyzer.

Just to clean up your idea since he is using C++, he can use it like:

enum myTokens {RESERVED_WORD, OPERATOR, VARIABLE, STRING, INVALID, NUMBER};

enum myTokens token_type;
char* token;

while(token = getNextToken())  // Your token function, whatever it is
{

  token_type = getTokenType(token);  // Your token examining function

  switch (token_type)
  {

      case RESERVED_WORD:  // some code here
           break;
      case OPERATOR:  // some code here
           break;
      /// etc, etc
      default:  // in case you got some kind of invalid data
  }

}