Gathering and updating results scattered in journal articles over thirty years, this self-contained monograph gives a comprehensive introduction to the subject. Its goal is to: - motivate and explain the method for general Lie groups, reducing the proof of deep results in invariant analysis to the verification of two formal Lie bracket identities related to the Campbell-Hausdorff formula (the "Kashiwara-Vergne conjecture"); - give a detailed proof of the conjecture for quadratic and solvable Lie algebras, which is relatively elementary; - extend the method to symmetric spaces; here an obstruction appears, embodied in a single remarkable object called an "e-function"; - explain the role of this function in invariant analysis on symmetric spaces, its relation to invariant differential operators, mean value operators and spherical functions; - give an explicit e-function for rank one spaces (the hyperbolic spaces); - construct an e-function for general symmetric spaces, in the spirit of Kashiwara and Vergne's original work for Lie groups. The book includes a complete rewriting of several articles by the author, updated and improved following Alekseev, Meinrenken and Torossian's recent proofs of the conjecture. The chapters are largely independent of each other. Some open problems are suggested to encourage future research. It is aimed at graduate students and researchers with a basic knowledge of Lie theory.