So what you want to know is the posterior of the coefficient (beta) of the linear function y = X*beta, but if you dont have per row data but just the aggregated count, you cannot only infer the information related to sufficient statistics (e.g., the mean) of y.
I think the easiest to see this is simulate some data and try modeling it.