Answer: The Rambam writes in one of his responses (Freiman edition, Chapter 20): “It is incorrect to pray in front of garments with designs on them, even if the designs are not protruding. We usually close our eyes when it happens that we must pray in front of a wall or garment adorned with designs.” We see clearly from the words of the Rambam that it is incorrect to pray in front of clothing or a curtain adorned with designs. The underlying reason for this prohibition is because these images can cause one’s concentration to be disturbed while praying. The Tur, Maran Ha’Shulchan Aruch (Chapter 90, Section 23), and Rabbeinu Yosef Haim in his Sefer Ben Ish Hai (Parashat Yitro) all rule likewise.
In the Sefer Ohr Le’Tzion (Volume 2, page 64), Hagaon Harav Ben-Zion Abba Shaul zt”l is quoted as saying that there is a distinction regarding this Halacha: If the designs are on a garment that is usually not in front of the person when he prays, for instance, if one happens to pray in a place where there is a colorful garment adorned with designs hanging which one is unaccustomed to, then one should abstain from praying in front of such a garment. However, if one is praying in front of a curtain covering the Aron Kodesh which one is always accustomed to having in front of him, we need not worry so much about a lack of concentration and he may in fact pray in front of such a curtain.
Nevertheless, Maran Rabbeinu Ovadia Yosef zt”l in his Responsa Yabia Omer (Volume 9, page 236) disagrees with the Ohr Le’Tzion on this point as it makes more sense to say that a curtain adorned with designs will always cause one’s concentration to be disturbed and we cannot make a distinction between a curtain and a garment which just happens to be in front of a person by chance, as the Poskim did not mention this distinction at all.
Besides for this, the Zohar, in addition to other works of the Mekubalim (such as the Sefer Charedim by Rabbeinu Elazar Ezkari), states that one who is not careful to shut his eyes during the Amida prayer is acting disrespectfully towards the holy presence of Hashem which faces one who prays. The following are the words of the holy Zohar about this matter: “One who opens one’s eyes during [the Amida] prayer, when his soul departs his body, the Angel of Death hurries to him and he will not merit seeing the presence of Hashem upon passing away, for he would always disregard the presence of Hashem by looking at it while praying. About him does the verse state, ‘For those who honor me,’ whoever honors Hashem’s presence, ‘I shall honor,’ Hashem shall honor him by allowing him to merit seeing His presence upon passing away, as another verse states, ‘For man cannot see me and live,’ one cannot see Hashem while he is alive, but after he has passed away, he can see. The verse continues, ‘And those who disregard Me,’ whoever disregards the presence of Hashem during prayer by allowing his eyes to roam around and look at what is in front of him, ‘Shall be treated with disdain,’ he shall also be treated disrespect fully when he passes on.” Bearing all of this in mind, one should not be lenient about this matter at all and every person should be very careful to either close his eyes while praying, even if one is in a place where there is nothing in front of him that would disrupt his concentration or focus one’s eyes on the Siddur in front of him, for this is also considered having one’s eyes closed.
Summary: One should be careful to either close one’s eyes or focus on one’s Siddur during the Amida prayer and one should not be looking forward or from side to side while praying.