Is there a way to modify the objective function in the AEVB example to add L1/L2 regularization for the encoder parameters?
I guess you can create a new Operator class similar to KL and add the regulation term regularizer_func(weight) to the loss function:
However, you need to scaled it properly - not sure what is the correct way to do so.
Answering my own question if someone else is trying to do a similar thing: you can simply add a potential term with pm.Potential and put a regularizing penalty there. Something like pm.Potential("regularizer",-(w**2).sum()) should do. The potential simply adds an arbitrary term to the loss function.
1 Like