I've been learning and loving Rust for something like 9 months now, but I haven't yet gotten a good grip on its various macro systems. So in this post I'll be exploring and sharing what I learn about writing macros in Rust. I'll start with a brief exploration of the different types of macros Rust supports, a few random anecdotes and then get into some declarative macro tinkering!
What are we even looking at?
Macros in Rust can provide a lot of utility and they can help simplify the process of writing code. Built-in macros like print!, write!, vec! and others are sprinkled throughout my code. Many of the dependencies I often use provide handy macros as well like #[tokio::main], json! from serde or the derive and attribute macros provided by thiserror. I've looked with curiosity at how some of the macros I've used are implemented, but for the most part my experience with that has given me the same feelings I get when I look at my old Perl code... "wtf I am I even looking at?"
I've dabbled with enough complex systems to know that I learn better with a bottom-up approach, so I'll start with the very basic building blocks and work my way up to increasingly more useful and complex designs. I've read that there are different types of macros, so enumerating those types feels like the best place to start.
Looking at The Book and this post by LogRocket. These are the types of macros in Rust:
- Declarative macros via
macro_rules! - Procedural macros
- Derive macros
- Attribute-like macros
- Function-like procedural macros
Generally, macros are tools for code generation. Between declarative and procedural macros, the former seems to be more common and the latter more advanced, so I'll explore the declarative macro systems first.
Just before jumping into declarative macros, I also want to call out The Little Book of Rust Macros as another great resource to have on hand. Often when I see or hear mentions of this book, it's followed by a warning that the information is outdated. This version is a fork that's actively maintained though, so it is very much worth consideration.
I declare!
When I look at the example macros in the docs, references and posts, my first reaction has usually been a strong aversion to the syntax. Right from the start, declarative macros begin with macro_rules!, which certainly comes off as declarative expression to me but... is this really the syntax/keyword for this? I was glad to find I'm not alone in thinking this should be something more like macro!, and I'm hoping someday that'll be what makes this post outdated. (See also: Rust RFC #440 and Rust RFC #1584)
It's partially comforting to learn that this is a result of Scheme's influence on Rust and not just an odd choice (though it certainly foreshadows the Lispy business to come). I know there are people that have a strong adoration for Lisp and they definitely have good reasons to feel that way, but my personal opinion is that Lisp dialects are usually more academic than they are practical. I really like when non-programmers can look at my code and follow along and that just doesn't seem feasible to me with a Lisp dialect. When I see the 7 closing parenthesis in a row in a tiny example I feel like I'm looking at a syntax made for machines rather than human beings.
(define-syntax while
(syntax-rules ()
((_ condition . body)
(let loop ()
(cond (condition
(begin . body)
(loop)))))))
15% parenthesis by volume! (or 20% if you remove non-essential whitespace)
With that context in place, and the cathartic vent, I already feel more prepared to enter Parenthesis Hell. Don't fret though! I'm still talking about Rust, so at least we're going to the Elysian plain and not Tartarus.
Basic syntax and structure
I want to start with a very simple macro that uses minimum syntax and build up from there. For this, I'm mostly referencing The Rust Reference - Macros By Example. I also ended up on Rust By Example - macro_rules! by accident at least twice thinking "I swear this page looked different 5 minutes ago", both are good to keep handy!
macro_rules! my_simplest_macro_yet {
() => {}
}
fn main() {
my_simplest_macro_yet!();
}
Alright, first goal achieved! It's technically a macro, and I'd say it certainly achieves the goal of doing nothing quite well.
While looking at the syntax tree for macro_rules! I noticed that you can also use parenthesis or square brackets instead of curly brackets as long as you also add a semicolon at the end:
macro_rules! my_simplest_macro_yet [
() => {}
];
macro_rules! my_simplest_macro_yet (
() => {}
);
As far as I can tell there is no real difference between these three. I don't think I've actually seen these last two formats in the wild... but hey, if you want to be the oddball here's one way to do it!
As described in The Book, you'll write declarative macros in Rust using a pattern matching syntax very similar to what is employed for match expressions. If the macro inputs match the thingies (matcher) to the left of the =>, apply the block of codestuffs (transcriber) from the right side.
The inputs you provide to macros like println! are not normal inputs. In the world of Rust macros, these inputs are lexical tokens of the language, so your actual code is the input to it. This is also the case on the output side as well. println! doesn't technically perform writing to standard out, but it generates code that will.
It'll be handy to see the actual results of the macros, so let's explore how to see their expanded outputs before dabbling with their inputs.
Show me the macro!
Since the macros aren't normal code, debugging them isn't quite normal either. I want some way to experiment and inspect the results, and cargo-expand currently looks like the simplest way to do that.
cargo install cargo-expand
To take a look at what's going on, I'll run cargo expand:
$ cargo expand
Checking rusty-macros v0.1.0
error: the option `Z` is only accepted on the nightly compiler
error: could not compile `rusty-macros`
Or not, I suppose! I'll run rustup default nightly to appease the deities of macro expansion and then try again.
$ cargo expand
Checking rusty-macros v0.1.0
Finished dev [unoptimized + debuginfo] target(s) in 0.08s
#![feature(prelude_import)]
#[prelude_import]
use std::prelude::rust_2021::*;
#[macro_use]
extern crate std;
fn main() {}
Hmm... well I guess I'm not sure what I actually expected here, but this does seem like a pretty reasonable implementation of my does nothing at all macro! Maybe it's a little more useful to consider what the default Rust "Hello World" expands to.
fn main() {
println!("Hello, world!");
}
cargo expand
#![feature(prelude_import)]
#[prelude_import]
use std::prelude::rust_2021::*;
#[macro_use]
extern crate std;
fn main() {
{
::std::io::_print(::core::fmt::Arguments::new_v1(&["Hello, world!\n"], &[]));
};
}
This is a bit more educational, as I can see the println! macro has indeed been replaced with code that will do the needful. I also get the same expansion if I modify by basic macro to wrap around the same functionality like this:
macro_rules! say_hello_world {
() => {
println!("Hello, world!")
}
}
fn main() {
say_hello_world!()
}
What if I add a touch of conditional logic to the matching block for this macro? I'm interested to see what happens (this is my kind of computer science right here: less math and more experimentation!)
macro_rules! say_hello_world {
() => {
if 1 > 0 {
println!("Hello, world!")
}
}
}
fn main() {
say_hello_world!()
}
What say you, cargo expand?
fn main() {
if 1 > 0 {
{
::std::io::_print(::core::fmt::Arguments::new_v1(&["Hello, world!\n"], &[]));
}
}
}
There are two things about this result I find interesting.
The first observation is that my always-true if condition survived expansion. I added this useless condition to test if/where it would be optimized away. Indeed, I don't see this condition actually implemented in the compiled program, which indicates to me that the optimization pass(es) occur sometime later in the compilation process. This makes total sense though, as optimization during macro expansion would likely be a bit premature. It feels like something useful to be aware of.
The second observation is not quite as useful: println! seems to be implemented in a way that that nests the output in its own block. This results in the expansion coming out as if 1 > 0 {{ print }}... (maybe (equal ((it) ("just that Lisp influence again"))))?
I'm also a little tickled that print! doesn't have these extra curlies:
macro_rules! say_hello_world {
() => {
if 1 > 0 {
print!("Hello, world!\n")
}
}
}
fn main() {
say_hello_world!()
}
cargo expand
fn main() {
if 1 > 0 {
::std::io::_print(::core::fmt::Arguments::new_v1(&["Hello, world!\n"], &[]))
}
}
Seeing the expanded macros should be pretty helpful for debugging, and with that it's time for some tinkering with macro inputs!
Charon's obol
Charon's obol?! Pardon me, I'm probably enjoying myself a little too much with the Parenthesis Hell and Hades references. I hope you don't mind though because it's time to visit the ferryman (he just wants your tokens, dammit!)
To start off as simply as possible, I'll first look at matching some simple literals in my macro.
macro_rules! say_what {
(true) => {
print!("Hello, world!\n")
};
(false) => {
print!("Goodbye world?\n")
};
(2) => {
print!("Hello, worlds!\n")
}
}
fn main() {
say_what!(2)
}
It does exactly what I'm expecting it to!
Hello, worlds!
Looking at how this expands, I can verify that this say_what! macro indeed acts a lot like a match expression (so far).
fn main() {
::std::io::_print(::core::fmt::Arguments::new_v1(&["Hello, worlds!\n"], &[]))
}
Post-expansion there are no indications left over of the original macro, the inputs to it, nor the un-matched branches. This macro isn't quite useful though, so it'll be much more interesting to implement something that's closer to a more realistic goal.
If at first you don't succeed... that makes me feel better because neither did I
The example I have in mind is a retry! macro. The scenario I'm giving myself is that I have some kind of network function that happens to use IPoAC. Since this protocol has pretty questionable reliability I'd like to retry the request a couple times before giving up.
Here's where I'm starting from:
use rand::prelude::*;
use anyhow::{Result,anyhow};
fn pigeon_post() -> Result<()> {
if random() {
Ok(())
} else {
Err(anyhow!("The pigeon got distracted in a grocery store parking lot"))
}
}
fn main() {
match pigeon_post() {
Ok(_) => println!("Success: The pigeon arrived!"),
Err(error) => println!("Error: {}", error)
}
}
I want my retry! macro to take 2 arguments: the identifier of the unreliable function to call and a numeric literal for the maximum amount of attempts to make.
The first couple ways I tried to implement this macro weren't going very well, even when I could see the problems pretty clearly using cargo expand. What ended up working for me was to go backwards: writing the result I want from the macro, moving that into the new macro, setting up the inputs then refactoring those inputs into the transcribed code.
So starting off in main(), I wrote what I wanted out of the macro:
fn main() {
for _ in 0..3 {
match pigeon_post() {
Ok(_) => {
println!("Success: The pigeon arrived!");
return
}
Err(error) => {
println!("Error: {}", error)
}
}
}
println!("We should probably hire a better pigeoneer")
}
To setup the inputs, I'll need to use metavariables. These are written in the format $<NAME>:<FRAGMENT-SPECIFIER>. The fragment specifiers declare what kind of Rust syntax we're looking to match on and capture. Picking the right fragment specifier seems like one of those things you generally get better at with time and practice. In my case, I'm using ident to pass in the identifier of the function to call and literal for the number of attempts to make.
macro_rules! retry {
($iffy_call:ident, $max_attempts:literal) => {
...
};
}
Now all I need is a bit of copy-pasta and to replace some syntax with my metavariables.
use rand::prelude::*;
use anyhow::{Result,anyhow};
macro_rules! retry {
($iffy_call:ident, $max_attempts:literal) => {
for _ in 0..$max_attempts {
match $iffy_call() {
Ok(_) => {
println!("Success: The pigeon arrived!");
return
}
Err(error) => {
println!("Error: {}", error)
}
}
}
println!("We should probably hire a better pigeoneer")
};
}
fn pigeon_post() -> Result<()> {
if random() {
Ok(())
} else {
Err(anyhow!("The pigeon got distracted in a grocery store parking lot"))
}
}
fn main() {
retry!(pigeon_post, 3);
}
Error: The pigeon got distracted in a grocery store parking lot
Error: The pigeon got distracted in a grocery store parking lot
Success: The pigeon arrived!
There it is, a working declarative retry macro! That's where I'll call it a wrap on this post. I hope you've found this entertaining and/or helpful in your own Rust adventures.
Originally I was planning to make this a little series on Rust macros, but as I researched my topic ideas I kept finding other good resources that did a great job of covering those topics. I'm not averse to reinventing the wheel, but only when I think I have a good idea! So instead, here's some other resources I thought could be pretty helpful:
- A few macro guides that re-implement
vec!: - Similarly, some resources that work to implement a
hashmap!macro: - Also pretty good: A Practical Intro to Macros in Rust 1.0