new trace_macros doesn't work if there's an error during expansion

[durka]

The new trace_macros has much better output, but it has regressed in functionality! The old trace_macros could be used to debug macro expansion. Example using rustc 1.6.0:

$ cargo +1.6.0 script -u trace_macros -e 'trace_macros!(true); macro_rules! m { (x $($t:tt)*) => { m!() } } m!(foo);'
   Compiling expr v0.1.0 (file:///Users/alex/.cargo/.cargo/script-cache/expr-634e13f197b6cae5)
m! { foo }
.cargo/.cargo/script-cache/expr-634e13f197b6cae5/expr.rs:11:81: 11:84 error: no rules expected the token `foo`
.cargo/.cargo/script-cache/expr-634e13f197b6cae5/expr.rs:11     match {trace_macros!(true); macro_rules! m { (x $($t:tt)*) => { m!() } } m!(foo);} {
                                                                                                                                            ^~~
Could not compile `expr`.

OK, you can see m!(foo) was invoked, and then it didn't match so there's an error.

Now with current rustc (all channels):

$ cargo script -u trace_macros -e 'trace_macros!(true); macro_rules! m { (x $($t:tt)*) => { m!() } } m!(foo);'
   Compiling expr v0.1.0 (file:///Users/alex/.cargo/.cargo/script-cache/expr-634e13f197b6cae5)
error: no rules expected the token `foo`
  --> .cargo/.cargo/script-cache/expr-634e13f197b6cae5/expr.rs:11:81
   |
11 |     match {trace_macros!(true); macro_rules! m { (x $($t:tt)*) => { m!() } } m!(foo);} {
   |                                                                                 ^^^

error: Could not compile `expr`.

You no longer get to see the chain of expansions leading up to the error. This typically occurs much deeper in a string of recursive macros, making debugging much harder.

cc #41520 @estebank @nikomatsakis

[Mark-Simulacrum]

Nominating for prioritization. Seems good to fix since this is very painful for macro authors.

[nikomatsakis]

triage: P-medium

Prioritizing as medium. @estebank -- any thoughts on how to include more information? @durka -- any suggestions for what the output should maybe look like?

[durka]

I think the output provided by #41520/#42103 is fine (modulo the verbosity issue #42223). It just needs to show up even when expansion doesn't finish due to an error. Something like this:

$ cargo script -u trace_macros -e 'trace_macros!(true); macro_rules! m { (x $($t:tt)*) => { m!() } } m!(foo);'
   Compiling expr v0.1.0 (file:///Users/alex/.cargo/.cargo/script-cache/expr-634e13f197b6cae5)
error: no rules expected the token `foo`
  --> .cargo/.cargo/script-cache/expr-634e13f197b6cae5/expr.rs:11:81
   |
11 |     match {trace_macros!(true); macro_rules! m { (x $($t:tt)*) => { m!() } } m!(foo);} {
   |                                                                                 ^^^
   = note: expanding `m! { foo }`

This is probably also the issue in #42170.

[jseyfried]

This underlying issue here is that macro errors cause aborts way too much. For a quick fix, we could make sure we always call trace_macros_diag in the fatal error handling methods.