TokenStream in proc_macro - Rust
Description: The main type provided by this crate, representing an abstract stream of tokens, or, more specifically, a sequence of token trees. The type provides interfaces for iterating over those token trees and, conversely, collecting a number of token trees into one stream.

## TokenStream

# Struct TokenStream

1.15.0 · Source

    pub struct TokenStream(/* private fields */);

Expand description

The main type provided by this crate, representing an abstract stream of tokens, or, more specifically, a sequence of token trees. The type provides interfaces for iterating over those token trees and, conversely, collecting a number of token trees into one stream.

This is both the input and output of `#[proc_macro]`, `#[proc_macro_attribute]` and `#[proc_macro_derive]` definitions.

## Implementations§

Source§

### impl TokenStream

1.29.0 · Source

#### pub fn new() -> TokenStream

Returns an empty `TokenStream` containing no token trees.

1.29.0 · Source

#### pub fn is_empty(&self) -> bool

Checks if this `TokenStream` is empty.

Source

#### pub fn expand_expr(&self) -> Result<TokenStream, ExpandError>

🔬This is a nightly-only experimental API. (`proc_macro_expand` #90765)

Parses this `TokenStream` as an expression and attempts to expand any macros within it. Returns the expanded `TokenStream`.

Currently only expressions expanding to literals will succeed, although this may be relaxed in the future.

NOTE: In error conditions, `expand_expr` may leave macros unexpanded, report an error, failing compilation, and/or return an `Err(..)`. The specific behavior for any error condition, and what conditions are considered errors, is unspecified and may change in the future.

## Trait Implementations§

1.15.0 · Source§

### impl Clone for TokenStream

Source§

#### fn clone(&self) -> TokenStream

Returns a duplicate of the value. Read more
1.0.0 · Source§

#### fn clone_from(&mut self, source: &Self)

Performs copy-assignment from `source`. Read more
1.15.0 · Source§

### impl Debug for TokenStream

Prints tokens in a form convenient for debugging.

Source§

#### fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
1.45.0 · Source§

### impl Default for TokenStream

Source§

#### fn default() -> Self

Returns the “default value” for a type. Read more
1.15.0 · Source§

### impl Display for TokenStream

Prints the token stream as a string that is supposed to be losslessly convertible back into the same token stream (modulo spans), except for possibly `TokenTree::Group`s with `Delimiter::None` delimiters and negative numeric literals.

Note: the exact form of the output is subject to change, e.g. there might be changes in the whitespace used between tokens. Therefore, you should not do any kind of simple substring matching on the output string (as produced by `to_string`) to implement a proc macro, because that matching might stop working if such changes happen. Instead, you should work at the `TokenTree` level, e.g. matching against `TokenTree::Ident`, `TokenTree::Punct`, or `TokenTree::Literal`.

Source§

#### fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
1.92.0 · Source§

### impl Extend<Group> for TokenStream

Source§

#### fn extend<T: IntoIterator<Item = Group>>(&mut self, iter: T)

Extends a collection with the contents of an iterator. Read more
Source§

#### fn extend_one(&mut self, item: A)

🔬This is a nightly-only experimental API. (`extend_one` #72631)
Extends a collection with exactly one element.
Source§

#### fn extend_reserve(&mut self, additional: usize)

🔬This is a nightly-only experimental API. (`extend_one` #72631)
Reserves capacity in a collection for the given number of additional elements. Read more
1.92.0 · Source§

### impl Extend<Ident> for TokenStream

Source§

#### fn extend<T: IntoIterator<Item = Ident>>(&mut self, iter: T)

Extends a collection with the contents of an iterator. Read more
Source§

#### fn extend_one(&mut self, item: A)

🔬This is a nightly-only experimental API. (`extend_one` #72631)
Extends a collection with exactly one element.
Source§

#### fn extend_reserve(&mut self, additional: usize)

🔬This is a nightly-only experimental API. (`extend_one` #72631)
Reserves capacity in a collection for the given number of additional elements. Read more
1.92.0 · Source§

### impl Extend<Literal> for TokenStream

Source§

#### fn extend<T: IntoIterator<Item = Literal>>(&mut self, iter: T)

Extends a collection with the contents of an iterator. Read more
Source§

#### fn extend_one(&mut self, item: A)

🔬This is a nightly-only experimental API. (`extend_one` #72631)
Extends a collection with exactly one element.
Source§

#### fn extend_reserve(&mut self, additional: usize)

🔬This is a nightly-only experimental API. (`extend_one` #72631)
Reserves capacity in a collection for the given number of additional elements. Read more
1.92.0 · Source§

### impl Extend<Punct> for TokenStream

Source§

#### fn extend<T: IntoIterator<Item = Punct>>(&mut self, iter: T)

Extends a collection with the contents of an iterator. Read more
Source§

#### fn extend_one(&mut self, item: A)

🔬This is a nightly-only experimental API. (`extend_one` #72631)
Extends a collection with exactly one element.
Source§

#### fn extend_reserve(&mut self, additional: usize)

🔬This is a nightly-only experimental API. (`extend_one` #72631)
Reserves capacity in a collection for the given number of additional elements. Read more
1.30.0 · Source§

### impl Extend<TokenStream> for TokenStream

Source§

#### fn extend<I: IntoIterator<Item = TokenStream>>(&mut self, streams: I)

Extends a collection with the contents of an iterator. Read more
Source§

#### fn extend_one(&mut self, item: A)

🔬This is a nightly-only experimental API. (`extend_one` #72631)
Extends a collection with exactly one element.
Source§

#### fn extend_reserve(&mut self, additional: usize)

🔬This is a nightly-only experimental API. (`extend_one` #72631)
Reserves capacity in a collection for the given number of additional elements. Read more
1.30.0 · Source§

### impl Extend<TokenTree> for TokenStream

Source§

#### fn extend<I: IntoIterator<Item = TokenTree>>(&mut self, trees: I)

Extends a collection with the contents of an iterator. Read more
Source§

#### fn extend_one(&mut self, item: A)

🔬This is a nightly-only experimental API. (`extend_one` #72631)
Extends a collection with exactly one element.
Source§

#### fn extend_reserve(&mut self, additional: usize)

🔬This is a nightly-only experimental API. (`extend_one` #72631)
Reserves capacity in a collection for the given number of additional elements. Read more
1.29.0 · Source§

### impl From<TokenTree> for TokenStream

Creates a token stream containing a single token tree.

Source§

#### fn from(tree: TokenTree) -> TokenStream

Converts to this type from the input type.
1.15.0 · Source§

### impl FromIterator<TokenStream> for TokenStream

A “flattening” operation on token streams, collects token trees from multiple token streams into a single stream.

Source§

#### fn from_iter<I: IntoIterator<Item = TokenStream>>(streams: I) -> Self

Creates a value from an iterator. Read more
1.29.0 · Source§

### impl FromIterator<TokenTree> for TokenStream

Collects a number of token trees into a single stream.

Source§

#### fn from_iter<I: IntoIterator<Item = TokenTree>>(trees: I) -> Self

Creates a value from an iterator. Read more
1.15.0 · Source§

### impl FromStr for TokenStream

Attempts to break the string into tokens and parse those tokens into a token stream. May fail for a number of reasons, for example, if the string contains unbalanced delimiters or characters not existing in the language. All tokens in the parsed stream get `Span::call_site()` spans.

NOTE: some errors may cause panics instead of returning `LexError`. We reserve the right to change these errors into `LexError`s later.

Source§

#### type Err = LexError

The associated error which can be returned from parsing.
Source§

#### fn from_str(src: &str) -> Result<TokenStream, LexError>

Parses a string `s` to return a value of this type. Read more
1.29.0 · Source§

### impl IntoIterator for TokenStream

Source§

#### type Item = TokenTree

The type of the elements being iterated over.
Source§

#### type IntoIter = IntoIter

Which kind of iterator are we turning this into?
Source§

#### fn into_iter(self) -> IntoIter ⓘ

Creates an iterator from a value. Read more
Source§

### impl ToTokens for TokenStream

Source§

#### fn to_tokens(&self, tokens: &mut TokenStream)

🔬This is a nightly-only experimental API. (`proc_macro_totokens` #130977)
Write `self` to the given `TokenStream`. Read more
Source§

#### fn into_token_stream(self) -> TokenStream

🔬This is a nightly-only experimental API. (`proc_macro_totokens` #130977)
Convert `self` directly into a `TokenStream` object. Read more
Source§

#### fn to_token_stream(&self) -> TokenStream

🔬This is a nightly-only experimental API. (`proc_macro_totokens` #130977)
Convert `self` directly into a `TokenStream` object. Read more
1.15.0 · Source§

### impl !Send for TokenStream

1.15.0 · Source§

### impl !Sync for TokenStream

## Auto Trait Implementations§

§

### impl Freeze for TokenStream

§

### impl RefUnwindSafe for TokenStream

§

### impl Unpin for TokenStream

§

### impl UnwindSafe for TokenStream

## Blanket Implementations§

Source§

### impl<T> Any for T
where T: 'static + ?Sized,

Source§

#### fn type_id(&self) -> TypeId

Gets the `TypeId` of `self`. Read more
Source§

### impl<T> Borrow<T> for T
where T: ?Sized,

Source§

#### fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

### impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

#### fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

### impl<T> CloneToUninit for T
where T: Clone,

Source§

#### unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (`clone_to_uninit` #126799)
Performs copy-assignment from `self` to `dest`. Read more
Source§

### impl<T> From<T> for T

Source§

#### fn from(t: T) -> T

Returns the argument unchanged.

Source§

### impl<T, U> Into<U> for T
where U: From<T>,

Source§

#### fn into(self) -> U

Calls `U::from(self)`.

That is, this conversion is whatever the implementation of `From<T> for U` chooses to do.

Source§

### impl<T> ToOwned for T
where T: Clone,

Source§

#### type Owned = T

The resulting type after obtaining ownership.
Source§

#### fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

#### fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

### impl<T> ToString for T
where T: Display + ?Sized,

Source§

#### fn to_string(&self) -> String

Converts the given value to a `String`. Read more
Source§

### impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

#### type Error = Infallible

The type returned in the event of a conversion error.
Source§

#### fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

### impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

#### type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

#### fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.