You are currently browsing the tag archive for the ‘expert testing’ tag.
Department of self-promotion: sequential tests, Blackwell games and the axiom of determinacy.
I am visiting the rationality center in the Hebrew University, and I am presenting some papers from the expert testing literature. Here are the lecture notes for the first talk. If you read this and find typos please let me know. The next paragraph contains the background story, and can be safely skipped.
A self-proclaimed expert opens a shop with a sign at the door that says `Here you can buy probabilities’. So the expert is a kind of a fortune-teller, he provides a service, or a product, and the product that the expert provides is a real number: the probability of some event or more generally the distribution of some random variable. You can ask for the probability of rain tomorrow, give the expert some green papers with a picture of George Washington and receive in return a paper with a real number between 0 and 1. The testing literature asks whether you can, after the fact, check the quality of the product you got from the expert, i.e. whether the expert gave you the correct probability or whether he just emptied your pocket for a worthless number.
So, let be a set of realizations. Nature randomizes an element from according to some distribution and an expert claims to know Nature’s distribution. A test is given by a function : the expert delivers a forecast and fails if the realization turned out to be in . A good test will be such that only `true’ experts, i.e. those who deliver the correct , will not fail. Read the rest of this entry »