[{"id":247336,"date":"2026-07-12T14:26:29","date_gmt":"2026-07-12T19:26:29","guid":{"rendered":"https:\/\/www.johndcook.com\/blog\/?p=247336"},"modified":"2026-07-12T14:26:29","modified_gmt":"2026-07-12T19:26:29","slug":"posterior-variance","status":"publish","type":"post","link":"https:\/\/www.johndcook.com\/blog\/2026\/07\/12\/posterior-variance\/","title":{"rendered":"Posterior variance"},"content":{"rendered":"

A few days ago I wrote a post entitled Does additional data always reduce posterior variance?<\/a>. In a nutshell, the answer is no, not always.<\/p>\n

That led the previous post<\/a> which looked at posterior means for three Bayesian models, showing how the posterior mean is a weighted average of the prior mean and the mean of the new data. The weights are precisions<\/em>, which means something different for each model.<\/p>\n

For the beta-binomial model, variance may increase when seeing unexpected data (details here<\/a>), but precision always increases.<\/p>\n

For the normal-normal model precision is the reciprocal of variance. Every new data point makes precision go up and posterior variance go down.<\/p>\n

The Poisson-gamma model may be the most interesting. As stated in the previous post, if data has a Poisson distribution with parameter \u03bb, and \u03bb has a gamma(\u03b10<\/sub>, \u03b20<\/sub>) prior distribution, then the posterior distribution on \u03bb after observing k<\/em> events over time t<\/em> has a gamma(\u03b10<\/sub> + k<\/em>, \u03b20<\/sub> + t<\/em>) posterior distribution. Therefore the posterior variance is<\/p>\n

(\u03b10<\/sub> + k<\/em>) \/ (\u03b20<\/sub> + t<\/em>)\u00b2.<\/p>\n

Note the posterior variance is an increasing function of\u00a0k<\/em> and a decreasing function of\u00a0t<\/em>. This means that the posterior variance increases\u00a0every time<\/em> an event is observed, and it decreases quadratically between observations.<\/p>\n

Here’s an illustration. I simulated data from a Poisson process with \u03bb and used a gamma(1, 1) prior on \u03bb. Here’s a plot of the posterior variance.<\/p>\n","protected":false},"excerpt":{"rendered":"

A few days ago I wrote a post entitled Does additional data always reduce posterior variance?. In a nutshell, the answer is no, not always. That led the previous post which looked at posterior means for three Bayesian models, showing how the posterior mean is a weighted average of the prior mean and the mean […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[17],"tags":[25,106],"class_list":["post-247336","post","type-post","status-publish","format-standard","hentry","category-statistics","tag-bayesian","tag-probability-and-statistics"],"acf":[],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"Posterior variance in conjugate models | Poisson-gamma","description":"Posterior variance for three conjugate Bayesian models. The posterior variance in the Poisson-gamma model jumps every time there's an observation.","canonical_url":"https:\/\/www.johndcook.com\/blog\/2026\/07\/12\/posterior-variance\/","robots":"max-image-preview:large","keywords":"bayesian,probability and statistics","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"en_US","og:site_name":"John D. Cook | Applied Mathematics Consulting","og:type":"article","og:title":"Posterior variance in conjugate models | Poisson-gamma","og:description":"Posterior variance for three conjugate Bayesian models. The posterior variance in the Poisson-gamma model jumps every time there's an observation.","og:url":"https:\/\/www.johndcook.com\/blog\/2026\/07\/12\/posterior-variance\/","article:published_time":"2026-07-12T19:26:29+00:00","article:modified_time":"2026-07-12T19:26:29+00:00","twitter:card":"summary","twitter:title":"Posterior variance in conjugate models | Poisson-gamma","twitter:description":"Posterior variance for three conjugate Bayesian models. The posterior variance in the Poisson-gamma model jumps every time there's an observation.","twitter:image":"https:\/\/www.johndcook.com\/blog\/wp-content\/uploads\/2022\/05\/twittercard.png"},"aioseo_meta_data":{"post_id":"247336","title":"Posterior variance in conjugate models | Poisson-gamma","description":"Posterior variance for three conjugate Bayesian models. The posterior variance in the Poisson-gamma model jumps every time there's an observation.","keywords":null,"keyphrases":{"focus":{"keyphrase":"","score":0,"analysis":{"keyphraseInTitle":{"score":0,"maxScore":9,"error":1}}},"additional":[]},"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":"","og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"Article","isEnabled":true},"graphs":[]},"schema_type":"default","schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":"-1","robots_max_videopreview":"-1","robots_max_imagepreview":"large","priority":null,"frequency":"default","location":null,"local_seo":null,"breadcrumb_settings":null,"limit_modified_date":false,"created":"2026-07-12 18:51:35","updated":"2026-07-12 19:26:29","ai":{"faqs":[],"keyPoints":[],"schemas":[],"titles":[],"descriptions":[],"socialPosts":{"email":{"subject":"","preview":"","content":""},"linkedin":[],"twitter":[],"facebook":[],"instagram":[]}},"seo_analyzer_scan_date":null},"aioseo_breadcrumb":"

\n\t\t\tHome<\/a>\n\t\t<\/span>»<\/span>\n\t\t\tStatistics<\/a>\n\t\t<\/span>»<\/span>\n\t\t\tPosterior variance\n\t\t<\/span><\/div>","aioseo_breadcrumb_json":[{"label":"Home","link":"https:\/\/www.johndcook.com\/blog"},{"label":"Statistics","link":"https:\/\/www.johndcook.com\/blog\/category\/statistics\/"},{"label":"Posterior variance","link":"https:\/\/www.johndcook.com\/blog\/2026\/07\/12\/posterior-variance\/"}],"_links":{"self":[{"href":"https:\/\/www.johndcook.com\/blog\/wp-json\/wp\/v2\/posts\/247336","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.johndcook.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.johndcook.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.johndcook.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.johndcook.com\/blog\/wp-json\/wp\/v2\/comments?post=247336"}],"version-history":[{"count":1,"href":"https:\/\/www.johndcook.com\/blog\/wp-json\/wp\/v2\/posts\/247336\/revisions"}],"predecessor-version":[{"id":247338,"href":"https:\/\/www.johndcook.com\/blog\/wp-json\/wp\/v2\/posts\/247336\/revisions\/247338"}],"wp:attachment":[{"href":"https:\/\/www.johndcook.com\/blog\/wp-json\/wp\/v2\/media?parent=247336"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.johndcook.com\/blog\/wp-json\/wp\/v2\/categories?post=247336"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.johndcook.com\/blog\/wp-json\/wp\/v2\/tags?post=247336"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}},{"id":247322,"date":"2026-07-12T12:48:14","date_gmt":"2026-07-12T17:48:14","guid":{"rendered":"https:\/\/www.johndcook.com\/blog\/?p=247322"},"modified":"2026-07-12T13:21:06","modified_gmt":"2026-07-12T18:21:06","slug":"posterior-mean","status":"publish","type":"post","link":"https:\/\/www.johndcook.com\/blog\/2026\/07\/12\/posterior-mean\/","title":{"rendered":"Posterior mean"},"content":{"rendered":"

Common sense says that what you believe after seeing new data should be some sort of compromise between what you believed before and what the new data says. You don’t want to ignore previous information or new information.<\/p>\n

How much should new data change your prior beliefs? When prior judgment and new information are in conflict, which one should be given the benefit of the doubt?<\/p>\n

Bayesian data models provide a framework for making such decisions quantitative and objective. The choice of a data model is somewhat subjective\u2014whether it’s a Bayesian model or not\u2014but given a Bayesian model, the rules for updating the representation of your beliefs are objective. As some put it, you “turn the Bayesian crank.” A likelihood model and a prior on parameters together specify how new data changes the prior distribution into a posterior distribution.<\/p>\n

We will make this more concrete with three examples.<\/p>\n

Normal-normal model<\/h2>\n

Suppose that data X<\/em> has a normal distribution with unknown mean μ and known variance \u03c3\u00b2, and we assume that a priori<\/em> μ has a normal distribution with mean \u03bc0<\/sub> and variance \u03c30<\/sub>\u00b2.<\/p>\n

After observing x<\/em>, the posterior distribution on μ also has a normal distribution, but with a different mean and variance. Its mean is somewhere between the prior mean and x<\/em>. We will ignore the change in the variance for this post.<\/p>\n

The posterior mean of μ is<\/p>\n

\\mu_{\\text{post}} = \\frac{\\dfrac{\\mu_0}{\\sigma_0^2} + \\dfrac{x}{\\sigma^2}}{\\dfrac{1}{\\sigma_0^2} + \\dfrac{1}{\\sigma^2}}<\/p>\n

This equation becomes more understandable when we introduce precisons \u03c4 = 1\/\u03c3\u00b2 and \u03c40<\/sub> = 1\/\u03c30<\/sub>\u00b2.<\/p>\n

Then we have<\/p>\n

\\mu_{\\text{post}} = \\frac{\\mu_0 \\cdot \\tau_0 + x \\cdot \\tau}{\\tau_0 + \\tau}<\/p>\n

which you can read as saying the posterior mean is the weighted average of the prior mean and x<\/em>, with the weights given by the precision. Intuitively, you take the weighted mean of your conclusions from previous data and new data, weighting the mean according to how much confidence you have in each.<\/p>\n

Beta-binomial model<\/h2>\n

Now let’s switch over to a different data model. Now assume\u00a0X<\/em> is a binary random variable, with probability of success\u00a0p<\/em> and probability of failure 1 \u2212 p<\/em>, and we assume p<\/em> has a beta(a<\/em>, b<\/em>) distribution.<\/p>\n

After observing s<\/em> successes and f<\/em> failures, the posterior mean of the distribution on p<\/em> becomes<\/p>\n

p_{\\text{post}} = \\frac{a + s}{a + b + s + f}<\/p>\n

We can rewrite this as<\/p>\n

p_{\\text{post}} = \\frac{(a + b) \\dfrac{a}{a+b} + (s + f) \\dfrac{s}{s+f}}{(a + b) + (s +f)}<\/p>\n

This says that the posterior mean is the weighted average of the prior mean a<\/em>\/(a<\/em> + b<\/em>) and the mean of the data s<\/em>\/n<\/em>. The weights are the prior effective sample size a<\/em> + b<\/em> and the sample size of the new data n<\/em>. In this example (effective) sample size is playing the role that precision played in the normal-normal model above.<\/p>\n

Gamma-Poisson model<\/h2>\n

Suppose data have a Poisson distribution with parameter λ, and λ has a gamma(α0<\/sub>, β0<\/sub>) prior distribution [1]. And suppose you observe k<\/em> events over time t<\/em>. Then the posterior distribution of λ given the data has a gamma(α0<\/sub> + k<\/em>, β0<\/sub> + t<\/em>) prior distribution and the mean of the posterior distribution is given by<\/p>\n

\\lambda_{\\text{post}} = \\frac{\\alpha_0 + k}{\\beta_0 + t} = \\frac{\\beta_0 (\\alpha_0 \/ \\beta_0) + t (k \/ t)}{\\beta_0 + t}<\/p>\n

As before, the posterior mean is a weighted average of the prior mean and new data, and the weights are interpretable as some sort of measure of confidence, namely time. The variable t<\/em> is directly time and the parameter β0<\/sub> is sort of an effective time, just as a<\/em> + b<\/em> is an effective sample size for the beta distribution.<\/p>\n

Common thread<\/h2>\n

In each example the posterior mean is the weighted average of the prior mean and the mean of the data, with the weights given by a precision. However, precision means something different in each example. In the normal-normal model, precision is the reciprocal of variance, but in the beta-binomial model precision is sample size and in the Poisson-gamma model precision is time.<\/p>\n

What all three examples have in common is that they are conjugate models using distributions from the “exponential family” of probability distributions. In technical terms, precision is the multiplicative factor on the sufficient statistic in the exponent of the posterior kernel.<\/p>\n

Related posts<\/h2>\n