Puma Reference Manual | Puma::Token Class Reference |
Abstraction of a language token. More...
#include <Puma/Token.h>
Public Types | |
enum | { ID_END_OF_FILE = -1, ID_UNKNOWN = -2, ID_ERROR = -3, ID_WARNING = -4 } |
Special token types. More... | |
Public Member Functions | |
Token (int type=0, LanguageID lang=LanguageID(0), const char *text="") | |
Constructor. More... | |
Token (const Token ©) | |
Copy-constructor. More... | |
virtual | ~Token () |
Destructor. More... | |
void | print (ostream &os) const |
Print the token text on the given stream. More... | |
virtual ListElement * | duplicate () |
Duplicate this token. More... | |
Unit * | unit () const |
Get the unit this token belongs to. More... | |
void | reset (int type=0, const char *text=0, LanguageID lang=LanguageID(0)) |
Reset the token object. More... | |
void | location (const Location &loc) |
Set the location of the token (usually the line and column in a file). More... | |
void | cont_lines (Array< int > *cl) |
Set the continues lines appeared in this token. More... | |
int | line_breaks () const |
Count the number of line breaks in the text of this token. More... | |
void | macro_call (bool is_call=true) |
Set the token as being a macro call. More... | |
const char * | text () const |
Get the token text. More... | |
DString & | dtext () const |
Get the token text. More... | |
char * | get_static_text () const |
Get the static token text. More... | |
int | type () const |
Get the type of the token (see Puma::CTokens). More... | |
const Location & | location () const |
Get the location of the token (usually the line and column in a file). More... | |
bool | is_macro_generated () const |
Check if the token is macro generated. More... | |
bool | is_macro_call () const |
Check if the token is a macro call. More... | |
bool | is_core () const |
Check if this is a core language token. More... | |
bool | is_identifier () const |
Check if the token is an identifier. More... | |
bool | is_keyword () const |
Check if the token is a keyword. More... | |
bool | is_wildcard () const |
Check if this is a wildcard token. More... | |
bool | is_preprocessor () const |
Check if this is a preprocessor token. More... | |
bool | is_directive () const |
Check if this is a compiler directive. More... | |
bool | is_whitespace () const |
Check if this is a white-space token. More... | |
bool | is_comment () const |
Check if this is a comment. More... | |
bool | is_macro_op () const |
Check if this is function-like macro operand. More... | |
bool | is_open () const |
Check if this is a left parenthesis. More... | |
bool | is_comma () const |
Check if this is a comma. More... | |
bool | is_close () const |
Check if this is a right parenthesis. More... | |
void * | operator new (size_t) |
Own operator new reusing memory. More... | |
void | operator delete (void *) |
Own delete operator. More... | |
![]() | |
ListElement () | |
Constructor. More... | |
virtual | ~ListElement () |
Destructor. More... | |
List * | belonging_to () const |
Get the list this element belongs to. More... | |
Static Public Member Functions | |
static char * | get_static_text (int token_type) |
Get the static token text for the given token type. More... | |
Static Public Attributes | |
static LanguageID | comment_id |
Comment tokens. More... | |
static LanguageID | pre_id |
Preprocessor tokens. More... | |
static LanguageID | dir_id |
Compiler directives. More... | |
static LanguageID | cpp_id |
C/C++ core language tokens. More... | |
static LanguageID | white_id |
White-spaces. More... | |
static LanguageID | wildcard_id |
Wildcards. More... | |
static LanguageID | keyword_id |
Keywords. More... | |
static LanguageID | open_id |
Left parenthesis. More... | |
static LanguageID | macro_op_id |
Function-like macro operands. More... | |
static LanguageID | identifier_id |
Identifiers. More... | |
static LanguageID | comma_id |
Comma. More... | |
static LanguageID | close_id |
Right parenthesis. More... | |
Abstraction of a language token.
A token is created by a scanner (see Puma::CScanner) as part a token chain (Puma::Unit). It encapsulates a lexical unit of the scanned text.
The information provided about a lexical unit is
anonymous enum |
Puma::Token::Token | ( | int | type = 0 , |
LanguageID | lang = LanguageID(0) , |
||
const char * | text = "" |
||
) |
Constructor.
type | The token type. |
lang | The language of the token. |
text | The optional token text (defaults to the empty string). |
Puma::Token::Token | ( | const Token & | copy) |
Copy-constructor.
copy | The token to copy. |
|
virtual |
Destructor.
Destroys the token text.
|
inline |
Set the continues lines appeared in this token.
cl | The continues line array. |
DString& Puma::Token::dtext | ( | ) | const |
Get the token text.
|
inlinevirtual |
Duplicate this token.
Implements Puma::ListElement.
char* Puma::Token::get_static_text | ( | ) | const |
Get the static token text.
Only for keywords, operators, and so on with a constant token text.
|
static |
Get the static token text for the given token type.
Only for keywords, operators, and so on with a constant token text.
|
inline |
Check if this is a right parenthesis.
|
inline |
Check if this is a comma.
|
inline |
Check if this is a comment.
|
inline |
Check if this is a core language token.
|
inline |
Check if this is a compiler directive.
|
inline |
Check if the token is an identifier.
|
inline |
Check if the token is a keyword.
|
inline |
Check if the token is a macro call.
bool Puma::Token::is_macro_generated | ( | ) | const |
Check if the token is macro generated.
|
inline |
Check if this is function-like macro operand.
|
inline |
Check if this is a left parenthesis.
|
inline |
Check if this is a preprocessor token.
|
inline |
Check if this is a white-space token.
|
inline |
Check if this is a wildcard token.
int Puma::Token::line_breaks | ( | ) | const |
Count the number of line breaks in the text of this token.
|
inline |
Set the location of the token (usually the line and column in a file).
loc | The location. |
|
inline |
Get the location of the token (usually the line and column in a file).
|
inline |
Set the token as being a macro call.
void Puma::Token::operator delete | ( | void * | ) |
Own delete operator.
void* Puma::Token::operator new | ( | size_t | ) |
Own operator new reusing memory.
void Puma::Token::print | ( | ostream & | os) | const |
Print the token text on the given stream.
os | The output stream. |
void Puma::Token::reset | ( | int | type = 0 , |
const char * | text = 0 , |
||
LanguageID | lang = LanguageID(0) |
||
) |
Reset the token object.
type | The new token type. |
text | The new token text. |
lang | The new token language. |
const char* Puma::Token::text | ( | ) | const |
Get the token text.
|
inline |
Get the type of the token (see Puma::CTokens).
Unit* Puma::Token::unit | ( | ) | const |
Get the unit this token belongs to.
|
static |
Right parenthesis.
|
static |
Comma.
|
static |
Comment tokens.
|
static |
C/C++ core language tokens.
|
static |
Compiler directives.
|
static |
Identifiers.
|
static |
Keywords.
|
static |
Function-like macro operands.
|
static |
Left parenthesis.
|
static |
Preprocessor tokens.
|
static |
White-spaces.
|
static |
Wildcards.