Comments on: Probability distributions in SciPy

By: nick

nick — Tue, 21 Jul 2009 17:03:29 +0000

I’ve been meaning to see how stat packages organize their distribution families. I thought it would be exactly this, but didn’t want to be hasty. Thanks so much, now I know the right way to proceed in my application!

By: John

John — Mon, 20 Jul 2009 20:27:21 +0000

Omer: You can calculate the sample mean and sample variance no matter what distribution your data comes from. However, if your data comes from a normal distribution, the sample mean and sample variance are the best estimators for the population parameters in a technical sense.

Does it look like your data follow a normal distribution? If not, what kind of distribution do they follow? That seems to be the first thing to settle. If you have questions as to the appropriate distribution for your data, perhaps we should continue this discussion via email. See the “About” tab at the top for my contact info.

By: Ralf Gommers

Ralf Gommers — Mon, 20 Jul 2009 19:48:57 +0000

Actually, I just realized what is going on with python help(). The distributions are class instances, and their docstrings are updated at import time to be instance-specific. IPython is smart enough to then do the right thing, the Python interpreter is not – it just displays the class docstring.

By: Omer Khalid

Omer Khalid — Mon, 20 Jul 2009 19:48:43 +0000

If I understood correct; scipy.stats.mean and scipy.stats.std will be sample’s mean and standard deviation but this is not normal distribution yet. right? Because in your examples you are referring to scipy.stats.norm.* ?

I suppose normal-distribution mean/std.deviation could be calculated through these functions: scipy.stats.norm.mean and scipy.stats.norm.std.

Probably it would be best I guess to first fit the sample set to a distribution and get a mean/std.deviation for it. If this could be the right approach then, would fit function will return a distribution type object which one can use for mean, std and probability?

By: John

John — Mon, 20 Jul 2009 19:05:50 +0000

Omer: If you just want to compute sample mean and sample standard deviation, you could use the functions scipy.stats.mean and scipy.stats.std. But if you want to fit parameters for non-normal distributions, look at the fit method on the distribution you're using to describe your data.

By: Omer Khalid

Omer Khalid — Mon, 20 Jul 2009 18:53:49 +0000

A good step in the right direction…

It would be good John if you could expand more on certain use cases. i.e. if one have a sample space of X as list [], then what steps one should take to calculate the mean and standard deviation.

From that moment onwards, your article provides a clear way forward to get PDF/CDF probabilities..

Thanks,
Omer

By: Ralf Gommers

Ralf Gommers — Mon, 20 Jul 2009 18:25:41 +0000

The python help() facility does not do the right thing (not sure why) but ipython does:

In [4]: scipy.stats.norm?

gives the same as the html docs.