Recent advances in machine learning have brought powerful new tools for predicting chemical reaction outcomes, especially template-free methods using GNNs and Transformers. Yet key questions remain: how do these models “reason” about chemistry? Can we trust and interpret their predictions in practice? In this article, we walk through a comparative case study of three approaches—GNN, Transformer, and semi-template—using the same reaction to explore not only prediction accuracy, but also how each model explains its outputs. If you're considering how to select or apply these tools in real-world chemical research, this piece offers grounded insights for informed decision-making.