Classes | Typedefs | Enumerations | Functions

Token extraction and manipulation

The routines in this group provide access to the tokens within a translation unit, along with a semantic mapping of those tokens to their corresponding cursors. More...

Collaboration diagram for Token extraction and manipulation:

[フレーム]

Classes

struct CXToken

Describes a single preprocessing token. More...

Typedefs

typedef enum CXTokenKind CXTokenKind

Describes a kind of token.

Enumerations

enum CXTokenKind {
CXToken_Punctuation , CXToken_Keyword , CXToken_Identifier , CXToken_Literal ,
CXToken_Comment
}

Describes a kind of token. More...

Functions

CINDEX_LINKAGE CXToken * clang_getToken (CXTranslationUnit TU, CXSourceLocation Location)

Get the raw lexical token starting with the given location.

CINDEX_LINKAGE CXTokenKind clang_getTokenKind (CXToken)

Determine the kind of the given token.

CINDEX_LINKAGE CXString clang_getTokenSpelling (CXTranslationUnit, CXToken)

Determine the spelling of the given token.

CINDEX_LINKAGE CXSourceLocation clang_getTokenLocation (CXTranslationUnit, CXToken)

Retrieve the source location of the given token.

CINDEX_LINKAGE CXSourceRange clang_getTokenExtent (CXTranslationUnit, CXToken)

Retrieve a source range that covers the given token.

CINDEX_LINKAGE void clang_tokenize (CXTranslationUnit TU, CXSourceRange Range, CXToken **Tokens, unsigned *NumTokens)

Tokenize the source code described by the given range into raw lexical tokens.

CINDEX_LINKAGE void clang_annotateTokens (CXTranslationUnit TU, CXToken *Tokens, unsigned NumTokens, CXCursor *Cursors)

Annotate the given set of tokens by providing cursors for each token that can be mapped to a specific entity within the abstract syntax tree.

CINDEX_LINKAGE void clang_disposeTokens (CXTranslationUnit TU, CXToken *Tokens, unsigned NumTokens)

Free the given set of tokens.

Detailed Description

The routines in this group provide access to the tokens within a translation unit, along with a semantic mapping of those tokens to their corresponding cursors.

Typedef Documentation

◆ CXTokenKind

typedef enum CXTokenKind CXTokenKind

Describes a kind of token.

Enumeration Type Documentation

◆ CXTokenKind

enum CXTokenKind

Describes a kind of token.

Enumerator
CXToken_Punctuation	A token that contains some kind of punctuation.
CXToken_Keyword	A language keyword.
CXToken_Identifier	An identifier (that is not a keyword).
CXToken_Literal	A numeric, string, or character literal.
CXToken_Comment	A comment.

Definition at line 4995 of file Index.h.

Function Documentation

◆ clang_annotateTokens()

CINDEX_LINKAGE void clang_annotateTokens ( CXTranslationUnit TU,

CXToken * Tokens,

unsigned NumTokens,

CXCursor * Cursors )

Annotate the given set of tokens by providing cursors for each token that can be mapped to a specific entity within the abstract syntax tree.

This token-annotation routine is equivalent to invoking clang_getCursor() for the source locations of each of the tokens. The cursors provided are filtered, so that only those cursors that have a direct correspondence to the token are accepted. For example, given a function call f(x), clang_getCursor() would provide the following cursors:

when the cursor is over the 'f', a DeclRefExpr cursor referring to 'f'.
when the cursor is over the '(' or the ')', a CallExpr referring to 'f'.
when the cursor is over the 'x', a DeclRefExpr cursor referring to 'x'.

Only the first and last of these cursors will occur within the annotate, since the tokens "f" and "x' directly refer to a function and a variable, respectively, but the parentheses are just a small part of the full syntax of the function call expression, which is not provided as an annotation.

Parameters: TU the translation unit that owns the given tokens.

Tokens the set of tokens to annotate.

NumTokens the number of tokens in Tokens.

Cursors an array of NumTokens cursors, whose contents will be replaced with the cursors corresponding to each token.

References CINDEX_LINKAGE.

◆ clang_disposeTokens()

CINDEX_LINKAGE void clang_disposeTokens ( CXTranslationUnit TU,

CXToken * Tokens,

unsigned NumTokens )

Free the given set of tokens.

References CINDEX_LINKAGE.

◆ clang_getToken()

CINDEX_LINKAGE CXToken * clang_getToken ( CXTranslationUnit TU,

CXSourceLocation Location )

Get the raw lexical token starting with the given location.

Parameters: TU the translation unit whose text is being tokenized.

Location the source location with which the token starts.

Returns: The token starting with the given location or NULL if no such token exist. The returned pointer must be freed with clang_disposeTokens before the translation unit is destroyed.

References CINDEX_LINKAGE.

◆ clang_getTokenExtent()

CINDEX_LINKAGE CXSourceRange clang_getTokenExtent ( CXTranslationUnit ,

CXToken )

Retrieve a source range that covers the given token.

References CINDEX_LINKAGE.

◆ clang_getTokenKind()

CINDEX_LINKAGE CXTokenKind clang_getTokenKind ( CXToken )

Determine the kind of the given token.

References CINDEX_LINKAGE.

◆ clang_getTokenLocation()

CINDEX_LINKAGE CXSourceLocation clang_getTokenLocation ( CXTranslationUnit ,

CXToken )

Retrieve the source location of the given token.

References CINDEX_LINKAGE.

◆ clang_getTokenSpelling()

CINDEX_LINKAGE CXString clang_getTokenSpelling ( CXTranslationUnit ,

CXToken )

Determine the spelling of the given token.

The spelling of a token is the textual representation of that token, e.g., the text of an identifier or keyword.

References CINDEX_LINKAGE.

◆ clang_tokenize()

CINDEX_LINKAGE void clang_tokenize ( CXTranslationUnit TU,

CXSourceRange Range,

CXToken ** Tokens,

unsigned * NumTokens )

Tokenize the source code described by the given range into raw lexical tokens.

Parameters: TU the translation unit whose text is being tokenized.

Range the source range in which text should be tokenized. All of the tokens produced by tokenization will fall within this source range,

Tokens this pointer will be set to point to the array of tokens that occur within the given source range. The returned pointer must be freed with clang_disposeTokens() before the translation unit is destroyed.

NumTokens will be set to the number of tokens in the *Tokens array.

References CINDEX_LINKAGE.

Generated on for clang by doxygen 1.14.0