Regularization for AEVB

Is there a way to modify the objective function in the AEVB example to add L1/L2 regularization for the encoder parameters?

I guess you can create a new Operator class similar to KL and add the regulation term regularizer_func(weight) to the loss function:

However, you need to scaled it properly - not sure what is the correct way to do so.

Answering my own question if someone else is trying to do a similar thing: you can simply add a potential term with pm.Potential and put a regularizing penalty there. Something like pm.Potential("regularizer",-(w**2).sum()) should do. The potential simply adds an arbitrary term to the loss function.

1 Like