You probably need to specify an initial point initval=pt.exp(mu) as the default is zero. Otherwise you can specify a moment function to produce the initial point used for sampling.
If you are sampling with NUTS, you should also specify transform=pm.distributions.transforms.log so the sampler won’t propose invalid values.
BUT, in that case you might as well use a MvNormal directly with a log transform for the same result. Or, even better, a MvNormal that you manually exponentiate without the artificial transform. You only need custom distributions when they are used as likelihood or in distribution factories like Mixture or RandomWalk.