TY - JOUR

T1 - How many parameters can a model have and still be testable?

AU - Bamber, Donald

AU - van Santen, Jan P.H.

N1 - Funding Information:
The first author’s work on this paper was supported by Veterans Administration Medical Research funds. The second author was supported by United States Air Force, Life Sciences Directorate, Grant AFOSR 80-0279. This paper has benefitted from extensive comments by Consulting Editor William Batchelder. Conversations with Richard Schweickert and James Townsend have helped to clarify some of the ideas presented in this paper. Also, careful reading by James Harper, and comments by James Neely and Peter Schoenemann, are appreciated. Morris W. Hirsch and Ralph Carr provided us with proof outlines for Appendix B and Example 4, respectively. Portions of this paper were presented at the Sixteenth Annual Mathematical Psychology Meeting, Boulder, Colorado, August 11, 1983.

PY - 1985/12

Y1 - 1985/12

N2 - A standard rule of thumb states that a model has too many parameters to be testable if and only if it has at least as many parameters as empirically observable quantities. We argue that when one asks whether a model has too many parameters to be testable, one implicitly refers to a particular type of testability, which we call quantitative testability. A model is defined to be quantitatively testable if the model's predictions have zero probability of being correct by chance. Next, we propose a new rule of thumb, based on the rank of the Jacobian matrix of a model (i.e., the matrix of partial derivatives of the function that maps the model's parameter values onto predicted experimental outcomes). According to this rule, a model is quantitatively testable if and only if the rank of the Jacobian matrix is less than the number of observables. (The rank of his matrix can be found with standard computer algorithms.) Using Sard's theorem, we prove that the proposed new rule of thumb is correct provided that certain "smoothness" conditions are satisfied. We also discuss the relation between quantitative testability and reparameterization, identifiability, and goodness-of-fit testing.

AB - A standard rule of thumb states that a model has too many parameters to be testable if and only if it has at least as many parameters as empirically observable quantities. We argue that when one asks whether a model has too many parameters to be testable, one implicitly refers to a particular type of testability, which we call quantitative testability. A model is defined to be quantitatively testable if the model's predictions have zero probability of being correct by chance. Next, we propose a new rule of thumb, based on the rank of the Jacobian matrix of a model (i.e., the matrix of partial derivatives of the function that maps the model's parameter values onto predicted experimental outcomes). According to this rule, a model is quantitatively testable if and only if the rank of the Jacobian matrix is less than the number of observables. (The rank of his matrix can be found with standard computer algorithms.) Using Sard's theorem, we prove that the proposed new rule of thumb is correct provided that certain "smoothness" conditions are satisfied. We also discuss the relation between quantitative testability and reparameterization, identifiability, and goodness-of-fit testing.

UR - http://www.scopus.com/inward/record.url?scp=0039029896&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0039029896&partnerID=8YFLogxK

U2 - 10.1016/0022-2496(85)90005-7

DO - 10.1016/0022-2496(85)90005-7

M3 - Article

AN - SCOPUS:0039029896

SN - 0022-2496

VL - 29

SP - 443

EP - 473

JO - Journal of Mathematical Psychology

JF - Journal of Mathematical Psychology

IS - 4

ER -