6/13/2019 Can someone advise on estimating item response theory parameters with bock aitkin EM and Metropolis-Hastings-Robbins-Monro (…

As regards the first question:

The short answer is the the MH-RM is faster in many cases.

At the risk of being misleading, I will try to give a simple explanation:

When we start out with the data in the scale, we don't know any of the parameters, so we have to estimate them
somehow. Unfortunately our IRT models express item parameters in terms of person parameters and
probabilities, and we don't know the person parameters either. Typically this is done with some kind of a
maximum likelihood approach. The likelihood is the probability of observing the parameters that are observed
(which we don't initially know) given some distribution underlying those parameters, which is why we want to
solve for the parameters which maximize the likelihood.

The Bock Aitkin approach, also called marginal maximum likelihood estimation, assumes that while we might not
know each person's ability, we know the distribution (typically the normal distribution) of abilities from which the
sample is drawn. This allows for a numerical solution to the likelihood maximization problem by integrating out
the person parameter but requires some computationally intensive methods since the integral does not have a
closed form solution. As the number of latent trait dimensions increases, there is a rapid increase in the
computational demands for this solution.

The MH-RM takes a different approach with no integration involved. Basically, the computer simply estimates
model parameters from an "observed" distribution. However, that "observed" distribution is not truly observed real
data; it comes from observations of simulated data. (That simulated data is arrived at via a Markov Chain -
wikipedia is a good place to start reading about this)

The reason to use the MH-RM method is because it is computationally faster for high dimensional latent traits.

To go to the literature:

I give a mix of papers here based on readability, convenience, and historical importance. However, I think that
these papers are not that clear by themselves. Some reading on Markov Chains and Metropolis Hastings will
probably be helpful.

There are two articles that describe the methods in which you are speaking, both of which are published in
Pyschometrika:

The first is the article by Bock and Aitkin, in which they introduce marginal maximum likelihood estimation:

Bock, R. Darrell, and Murray Aitkin. "Marginal maximum likelihood estimation of item parameters: Application of
an EM algorithm." Psychometrika 46.4 (1981): 443-459.

http://link.springer.com/article/10.1007/BF02293801

They discuss the generalization of the marginal maximum likelihood estimation towards the end of the article to
two dimensions. You get a hint of the computational difficulties that can be associated withe high dimensional
data towards the end when they discuss this two dimensional generalization, but in my opinion it is not quite so
obvious.

As such, I think that perhaps the single best resource is an original article by Li Cai in Psychometrika whom I
think is actually one of the developers of IRTPro available here (notably it is published as an open source
article): http://link.springer.com/article/10.1007/s11336-009-9136-x

This paper is about using the MH-RM algorithm, and its introduction does a pretty good job laying out in plain
English the role of the EM algorithm and the reason that an MH-RM algorithm has been developed. However, this
paper sort of starts out by assuming that the reader understands bayesian methods.

I hope that these are at least helpful to get you started.

As for the second question:

https://www.researchgate.net/post/Can_someone_advise_on_estimating_item_response_theory_parameters_with_bock_aitkin_EM_and_Metrop… 1/2
6/13/2019 Can someone advise on estimating item response theory parameters with bock aitkin EM and Metropolis-Hastings-Robbins-Monro (…

I am not sure what you mean by "invariant item parameters," but I assume that you are referring to mean
insensitivity of item parameter estimates to the sample in which they are estimated.

Let me point out that there is a difference between the parameters and the estimation of the parameters.

Now we don't have to prove that our model parameters are invariant. We don't have to prove anything about our
model - it is our model, so we can make whatever assumptions we would like in it. Whether or not our model has
anything to do with reality is another matter. The question is whether or not the model fits the data and whether or
not we can estimate the model's parameters.

The way that we estimate parameters is by obtaining some data from a sample. The question then is whether or
not the sample that we choose will affect the estimates of the parameters? Let me stress that we are referring to
sample selection within a single population. Two very different samples taken from the same population will
ideally give the same estimates, but two samples from two different populations might not.

Now, to answer the question:

If we use a marginal maximum likelihood estimation procedure, for example, we might find that changes in the
sample do change the estimated parameters, because this estimation methods relies on assumptions about
distribution of the observed participants. However, conditional maximum likelihood estimation in a sense allows
for sample independence. The most clearly laid out explanation of this that I am aware of was published in a
somewhat obscure journal (at least to the English speaking world):

http://erm.r-forge.r-project.org/ps_1_2007_26-43.pdf

It is about a software package implementing the CML estimation procedure and therefore explains why this
estimation method is the closest thing to "sample independent" that we have.

However, I don't think that multidimensional conditional maximum likelihood has been developed (unless one
considers the linear logistic models multidimensional models). Only IRT models in which a location parameter
has been estimated are suitable for CML estimation.

(There are other IRT estimation methods as well.)

Let me also add that IRT estimation methods are a contentious issue with both money and academic reputation
are at stake. Each IRT estimation program has an estimation method, and the developers of each program have
generally published many articles explaining why the estimation procedure that they use is best. You will notice
that the article I give you for CML estimation is by a group that has developed a software program using CML to
estimate the Rasch model. That being said, CML is probably the gold standard if there is such a thing, and other
estimation methods are sometimes compared against it to demonstrate that they yield good estimates.

So how confident can we be? Well, we can be absolutely confident that our model meets our assumptions, but
our model does whatever we want it to do. As for our confidence about parameter estimates..... This is debatable,
especially depending on what estimation method you use.

https://www.researchgate.net/post/Can_someone_advise_on_estimating_item_response_theory_parameters_with_bock_aitkin_EM_and_Metrop… 2/2