I quite like this description from our docs (which are admittedly impossible to navigate)
PyTensor combines aspects of a computer algebra system (CAS) with aspects of an optimizing compiler. It can also generate customized code for multiple compiled languages and/or their Python-based interfaces, such as C, Numba, and JAX. This combination of CAS features with optimizing compilation and transpilation is particularly useful for tasks in which complicated mathematical expressions are evaluated repeatedly and evaluation speed is critical. For situations where many different expressions are each evaluated once, PyTensor can minimize the amount of compilation and analysis overhead, but still provide symbolic features such as automatic differentiation.
I personally think of Pytensor as a mix of sympy and <insert your favorite DL library here>