CS 146 Spring 2016

March 1, 2016

Homework 2: MT
Exercise 2.1
The alignment is < 2, 1, 4, 5, 3 >.

Exercise 2.2
We start of setting e,f = 1 for all English words e, French words f. Also, we start off the
algorithm saying ne,f = 0 for all e and f.
First iteration. The first iteration is easy to compute. From the first sentence:
pmange = She,mange + eats,mange + bread,mange = 1 + 1 + 1 = 3

(1)

With this we add 31 to neats,mange , which started off at 0. So right now it is at 13 .

The second sentence has a remarkably similar calculation:
pmange = He,mange + eats,mange + soup,mange = 1 + 1 + 1 = 3
1
3

(2)

2
.
3

Then again we need to add to neats,mange . So we have neats,mange =

Now at this point,
for all e, f where e and f are English and French words in our corpus respectively, ne,f = 31
- with the exceptions of neats,mange = neats,du = 32 .
We should note that
neats, = neats,elle + neats,mange + neats,du + neats,pain + neats,il + neats,boeuf
. In the maximization step, we then set eats,mange =
So, at the end of the first iteration, eats,mange =

1
4

neats,mange
neats,

2
3
8
3

(3)

= 14 .

and neats,mange = 23 .

Second iteration. First sentence:

pmange = She,mange + eats,mange + bread,mange =
So now, we add to neats,mange + =

1
4
1
2

1
1 1 1
+ + =
8 4 8
2

(4)

1 1 1
1
+ + =
8 4 8
2

(5)

1
2

The second sentence:

pmange = He,mange + eats,mange + soup,mange =
So now, we add to neats,mange + =

1
4
1
2

= 12 . neats,mange is now at 1.

What about ? Well, as neats,mange is now at 1, is just

1
.
neats,

neats, = neats,elle + neats,mange + neats,du + neats,pain + neats,il + neats,boeuf

(6)
1

Homework 2: MT
=

1
1 1 1
+1+1+ + +
3
3 3 3
10
=
3

Then eats,mange is the reciprocal of this, so

In summary:
1. After the first iteration, eats,mange =

CS 146 Spring 2016

March 1, 2016
(7)
(8)

3
.
10

1
4

2. After the second iteration, eats,mange =

and neats,mange = 32 .
3
10

and neats,mange = 1.

Exercise 2.3
In using the IBM Model 1, we make the assumption that each French word corresponds to
exactly one English word. This is not a one-to-one mapping: an alignment does not require
each English word to correspond to a French word. ne, is the number of times that an
English word is aligned to any French word, but if there is not a one-to-one correspondence
we cannot guarantee that this is equal to the number of times e occurs in the corpus.

Exercise 2.4
< 0, 0, 0 > adheres to the definition of an alignment and is therefore an entirely legal alignment. An alignment of this sort could occur with a particular expression or idiom that makes
no sense in a literal translation.
For example, in one section of Ulysses Leopold Bloom finds an old potato in his pocket as
hes closing his front door and realizes he left his keys inside. In the stream-of-consciousness
narrative, this sentence becomes Keys: not here. Potato I have. An Argentinian translator
of the novel interpreted the potato sentence as a turn of phrase (not as a literal potato)
and translated it with the phrase Lost my carrots, meaning Bloom was starting to lose his
grip of reality. Here the colloquial Argentinian expression would have a < 0, 0, 0 > alignment
with the original English sentence (except if youre willing to take some liberties when it
comes to root vegetables.

Exercise 2.5
If all initial values for e,f are the same, it is easily shown that the subsequent iterations of
the EM-algorithm are not functions of this initial value.
2

Homework 2: MT

CS 146 Spring 2016

March 1, 2016

Where do our initial values of get used? Say we assign our e,f to k for all e, f, where
k is a positive integer. We see that for any m pm is the summation of these values of ,
so ck (where c is a positive integer). Now, for a particular ne,f , we will throughout the
algorithm increment by e,f /pf which, of course (as all our values have been initialized
k
= 1c .
equally, ck
Concretely, this means that our actual initial values for have cancelled out and all our
arithmetic in assigning the new values is no longer based in k. Hence, our future values of
are independent of this initial value.
We do see that the values of pk are dependent on these initial assignments of (in the
explanation above, we denoted it as ck.) By extension, the likelihood Ld () of the data will
also be affected by this initial assignment.

Exercise 2.6
The word the will align most with pain simply because the is the most common word in
the English language. Most sentences include at least one or two instances of the article, so
there will just be more alignments where it aligns with mange than any other word.
However, the fact that pain aligns often with the is not a fact exclusive to a particular
French word. is calculated as a proportion, not as an absolute count of alignment incidences.
the aligns with so many different words that for any f the,f will be comparatively low.

Exercise 2.7
We can mathematically illustrate the difference between these two equations by invoking
equation (2.14) from the textbook:
P (Ak = j|e, fk ) =

P (fk |Ak = j, e)
P (Ak = j|e)
P (fk |e)

(9)

This equation establishes the relationship between the two probabilities compared and contrasted in this question. What this equation shows is that knowledge of the French sentence
comes along with an ability to weight certain alignments more favorably than others. (This
is shown by the way the right hand side splits into the factor conditioned on e and the factor regarding the probabilities of fk . When the French sentence is known, for example, we
could weight alignment probabilities based on length, thereby making alignments unequal in
probability.
Lacking this information, though, as in equation (2.23), we can not discriminate between
various French sentences and by the IBM model must assign them all the same probability.
Therefore, each particular alignment, solely conditioned by the English sentence, is equally
likely.