Hi Timothé,
The choice of prior often depends on use-case, especially for hierarchical models – the different levels and the link function (if any) can modify the scientific sense of the prior. So the best pratice to determine your priors is to do prior predictive checks, with the very handy pm.prior_predictive_checks function.
You can also just sample from your priors and plug the sampled parameters into your linear model: I wrote a NB to demonstrate how to do that in a multinomial regression – which is usually more complicated than simple linear regression so you should be able to adapt the idea.
Note however that the half Cauchy has fat tails and can therefore disrupt inference when strong regularization is needed – thus, the exponential distribution tends to be more approriate for std priors.
Finally, how does it impact inference? Well… it depends. So the best is to test several priors and see how inference about your problem changes! Priors are not an oath – just a hypothesis that you can test however you want.
Hope this helps! PyMCheers 