Can NUTS help me to find a set of parameters with largest likelihood if my model is identifiable only up to rearrangement?
One of the parameter is a matrix \mathcal{A} . Let’s say matrix A_{1} is a possible value of \mathcal{A} .
I can form a new matrix A_{2} by the following two steps:
swap i^{\text{th}} and j^{\text{th}} row of A_{1} to form B
swap i^{\text{th}} and j^{\text{th}} column of B to form A_{2}
Likelihood of A_{1} is same as likelihood of A_{2} .
I’m using a Dirichlet prior for \mathcal{A} .
Something like transform.ordered might work. But I need to sort both row and columns by the ordering of row sums.
TL;DR: I’m attempting to perform non-parametric clustering of circular data using a Dirichlet process mixture of von Mises distributions. I’m trying to fix label-switching using transforms.ordered, but my chains contain only divergent samples.
Details:
The observed data are stimulus values (colour) from a cognitive psychology experiment. Each trial of the experiment used 6 stimuli sampled from a uniform distribution. For our analysis we would like to characterize possible clusters of stimuli t…
Nope, most likely NUTS will try to do sample all the space and failed, see eg Identifying Bayesian Mixture Models
One solution I can think of is to port the permutation bijector from TFP into a transformer in pymc3.
# Copyright 2018 The TensorFlow Probability Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
"""Permutation bijectors."""
# Dependency imports
import numpy as np
import tensorflow.compat.v2 as tf
show original
One solution
Is the following post-processing of the trace enough to fix this problem?
For each value of matrix \mathcal{A} in the trace,
Calculate indices = np.argsort(np.sum(A, axis=0))
Reorder the matrix by A = A[indices, :][:, indices]
Well, it is certainly one way to do it, but if you have within trace mode exploring than it would be quite difficult.
Would the identifiability problem mess up the tuning of the mass matrix and NUTS?
I would say no as the mass matrix estimation is over the whole posterior, so label switching wont effect that
I meant mode switching within chain
Would the permutation bijector avoid the mode switching problem?
I think mode switching within chain would affect estimation of the distribution.