Posts Tagged ‘Mathematics’

Musical anniversaries in 2019

January 25, 2019

Every year I like to plan a few listening projects around composers who will be marking significant birthdays and memorials in the year ahead. From a very thorough list (Thanks, Osbert) I have culled the following set:

Memorials

25 years

  • Witold Lutosławski

150 years

  • Hector Berlioz

***

Birthdays

100 years

  • Mieczyslaw Weinberg

200 years

  • Clara Schumann
  • Jacques Offenbach

400 years

  • Barbara Strozzi

*

This is a pretty thin showing, and from this evidence I think we can confidently conclude that being born in a year ending 19 or 69 is a misfortune from a musicality point of view; the same holds true for those destined to die in such years. I expect there is a straightforward astrological explanation.

I have put Clara and Barbara on the list mainly because I thought it was interesting that two of the relatively few female composers were born in years for which the last two digits were the same. What were the chances? [*]

In a similar way, I mention Berlioz only because it seemed unduly audacious to leave him off. As far as I can tell, his principal redeeming quality is that Jacques Barzun admired him.

For me the highlight of the year is unquestionably the centenary of Mieczyslaw Weinberg, who has been mentioned here several times over the years. I’ve planned a big listening project that will take me through all those of his compositions which have been recorded and are available — roughly 100, I believe. I note that the blog Lines that have escaped destruction, which is dedicated to Weinberg’s music, has planned a series of 100 blog posts for this year about various aspects of his life and work; I will be following it with interest.

Happy listening! As an envoi, here is the adagio section of Weinberg’s Sinfonietta No.2, played by Gidon Kremer and Kremerata Baltica:

***

[*] This is a variant on the birthday problem. Suppose we have N female composers, and we want to find the probability that at least 2 of them share the final two digits of their birth year. There are Y = 100 possible such two-digit combinations.

It is easiest to calculate the probability that they do not share those digits. This probability is

\bar{P}(N, Y) = 1 \cdot \left(\frac{Y-1}{Y}\right) \cdot \left(\frac{Y-2}{Y}\right) \cdot \cdot \cdot \left(\frac{Y-(N-1)}{Y}\right)

assuming that N < Y and (of course) that each pair of year-digits are equally likely. This means that the probability that at least two of our ladies do share those digits in their birth years is

P(N, Y) = 1 - \prod^{N-1}_{\ell=0} \left(1-\frac{\ell}{Y}\right)

Since Y=100, the only variable is N, the number of female composers. It’s easy to find lists of more than 100, so the probability of having a collision on birth dates becomes 1. But my list above is limited to the cream of the crop, or at least the composers of no small repute, on both the male and female sides, and everyone knows that on those conditions there have been only a handful of female composers: St Hildegard, Barbara Strozzi, Fanny Mendelssohn, Clara Schumann, Lili Boulanger, and another few of your choosing — maybe Gloria Coates, Judith Weir, and Lera Auerbach, but feel free to swap in your favourites. Let’s say ten in total. Then the probability of a collision on one of these lists is

P(10, 100) = 0.37

which is higher than I expected! If we’re really strict and only admit those eminent five into the circle, the probability drops to about 10%. In other words, we are, quite possibly, witnessing this year an event that shall not be repeated for a decade!

Briggs: Uncertainty

November 4, 2016

Uncertainty
The Soul of Modeling, Probability & Statistics
William Briggs
(Springer, 2016)
278 p.

Being something of a beginner in the art of statistical analysis, I thought this book on the philosophical and conceptual underpinnings of statistical methods would be instructive, and I was right. I learned so much I’m not sure I want to learn any more.

In a nutshell: Briggs is critical of most of the standard apparatus of statistical methods, both technical and interpretive. Hypothesis testing, regression, data smoothing, quantification of everything, and, above all, p-values he condemns to perdition. The problem is not that such methods have no value, but that they are widely misunderstood and misapplied, with the result that the conclusions drawn from statistical analyses are often either simply wrong or the uncertainty in those conclusions is underestimated (and by an unknown amount). He gives many examples of ways in which standard techniques lead to spurious “significant” results.

By criticizing standard statistical methods, one might get the impression that Briggs’ is a lone voice crying in the wilderness, but he has plenty of citations to offer for most of his arguments. He belongs to an alternate, minority, but not negligible tradition.

Some of the important points he makes:

Probability is logical. Logic concerns relationships between propositions, and so does probability, except that in the latter case the logic is extended to propositions the truth of which is uncertain. This point was made lucidly and rather beautifully by Jaynes, and reading Briggs has made me want to return to that book to read more of it.

Probability is not a cause. Probability can tell us about correlations, but nothing at all about causes. The habit of inferring causes from statistical correlations, absent a corresponding causal model, is a bad habit that leads many astray. In general, uncertainty reflects our ignorance of causes rather than our knowledge of them.

Probability is conditional. Probability statements are always conditional on a set of premises. This is no such thing as Pr(X), but only Pr(X|Y) — that is, the probability of X given some set of premises Y. If the premises change, the probability of X will, in general, change. Thus Briggs, while not quite a Bayesian, does think the Bayesians have it over the frequentists when it comes to the debate over whether probability is objective (ie. out there) or subjective (ie. in the mind). Probabilities reflect the uncertainty in propositions given what we know; they do not exist outside our minds, and they change when our knowledge changes. A corollary is that one should never say, “X has a probability of Z”. Nothing has a probability. Probability does not exist. One should only say, “Given premises Y, the probability of X is Z.”

Probability is often not quantifiable. If we know “Most people like ice cream and Sue is a person”, the probability that Sue likes ice cream cannot be naturally or unambiguously quantified unless the meaning of “most” is clarified. Moreover, it is often a mistake to force probabilistic arguments into a quantified form. Briggs argues that the habit of doing so (as with “instruments” for assessing subjective attitudes about politics or emotional responses to stimuli, for instance) often leads to misleading results and promotes the vice of scientism.

Statistical significance is not objective. No probability model can tell one whether a given probability is significant or not. This is an extra-statistical, and often an extra-scientific, question. Whether it is judged significant is a matter of prudential judgment based on the specific question at issue and the decisions to be made about it. Thus he would like to disrupt the “turn the crank” model of statistical analysis in which “significant” results pop out of the sausage-maker, returning such questions to spheres of deliberation and judgment.

Probability models should be predictive. Briggs’ principal constructive suggestion (apart from shoring up our understanding of what probability is) is that statistical models should be predictive. They should state their premises in as much detail as possible, and should predict observations on the basis of those premises (taking into account uncertainties, of course). If the models fail to predict the observables, they are not working and should be amended or scrapped. As I understand it, he is proposing that fields which lean heavily on statistics should, by following his proposals, become more like the hard sciences. True, progress will be slower, and (acknowledged) uncertainties larger, but progress will be surer and causes better understood.

***

Briggs has some fun pointing out common fallacies in statistical circles. There is, for instance, the We-Have-To-Do-Something Fallacy, in which a perceived imperative to do something about something (usually something political) leads to the employment of some defective or fallacious statistical method, the defectiveness or fallaciousness of which is then ignored. Or the Epidemiologist’s Fallacy, in which a statistician claims “X causes Y” even though X was never measured and though statistical models cannot in any case discern causes. (This fallacy is so-called because without it “most epidemiologists, especially those in government, would be out of a job”.)  Or the False Dichotomy Fallacy, which is the foundational principle of hypothesis testing. Or the Deadly Sin of Reification, whereby statisticians mistake parameters in their statistical models for real things. And so on.

***

Much of this might seem rather obvious to the uninitiated. I’m not an adept of the standard techniques, so I was at times a little puzzled as I tried to discern the particular bad habit Briggs was criticizing. But, as is increasingly appreciated (here and here, for instance), the use and abuse of the standard techniques have led wide swathes of the scientific community into error, most commonly the error of over-certainty, which is actually an uncertainty about what is true. An audience for this book clearly exists.

Were his recommendations to be followed, he argues that the effects would be

a return to a saner and less hyperbolic practice of science, one that is not quite so dictatorial and inflexible, one that is calmer and in less of a hurry, one that is far less sure of itself, one that has a proper appreciation of how much it doesn’t know.

But, on the other hand, it would reduce the rate at which papers could be published, would make decisions about significance matters of prudential judgment rather than scientific diktat, and would make scientific conclusions more uncertain. He is fighting an uphill battle.

Briggs is an adjunct professor at Columbia, and has done most of his scientific work in climate science (and is, as you would expect, skeptical of the predictions of statistical climate models, which provide a few of his case studies). He seems to be something of an atypical academic: this book, for instance, includes approving reference to Aristotle, Thomas Aquinas, and even John Henry Newman (whose Grammar of Assent he cites as an example of non-quantitative probabilistic argumentation). It’s quite a rollicking read too. Briggs has a personality, and doesn’t try to hide it. Personally I found the tone of the book a little too breezy, the text sometimes reading almost as if it were transcribed lecture notes (I make no hypothesis), but overall the book is smart and clear-eyed, and I’m glad to have read it. Now back to Jaynes.

***

I found a good video which illustrates the problem with relying on p-values to determine statistical significance. When I consider that many of the findings of the social sciences are based on this criterion I’m not sure whether to cringe or weep. No wonder there is a replication crisis. Witness the dance of the p-values:

Here is a short video illustrating why it is reasonable to doubt the putative findings of many (and perhaps most) published research papers employing statistical methods. This argument and others are set forth in detail by Ioannidis.

The divisibility trick

January 7, 2014

You might have learned the “divisibility trick” in grade school. It says that if you want to know whether a number is divisible by 3, there is a shortcut: if the sum of its digits is divisible by 3 then the number itself is divisible by 3. For example, is 459 divisible by 3? Well, 4 + 5 + 9 = 18, which is divisible by 3, so 459 is divisible by 3 as well.

This trick also works with the number 9. Again, you can try it with 459.

A week or two ago I was reading Anthony Esolen’s “Word of the Day” blog in which he stated a result about the divisibility trick generalized to a base-X number system; namely, in a base-X number system the divisibility trick works for X-1 and its factors. I was intrigued, and, as I had given some thought to the divisibility trick a few years ago and had some notes on it, I sat down last night and came up with what I think is a sound proof of the claim.

I am sure there is a nice way to formulate the argument — my approach leans heavily on modular arithmetic, which is closely related to the elegant theory of cyclic groups — but I went about it in the most simpleminded way imaginable. You can read my argument here:

DivisibilityTrickBaseX [pdf, Updated]

An amusing application of this result is in a binary (base-2) number system. The claim simply says that any binary number for which the sum of its digits is divisible by 1 (which is all of them, since every positive integer is divisible by 1) is itself divisible by 1 (which is all of them, for the same reason). So the claim is almost empty in that case.