Question: May a soldier or a citizen armed with a pistol or a rifle enter a synagogue with his weapon?
Answer: The Mechilta (end of Parashat Yitro) states regarding the verse, “And when you shall build Me a stone altar, you shall not build it of hewn stone; for you will have waved your sword over it and desecrated it”: “Rabbi Shimon ben Elazar said: The Altar was created to prolong the life of man. Thus, one may not wave the ‘item that shortens’ (the item that shortens man’s life, i.e. a knife) over the ‘item that lengthens’ (the item that prolongs man’s life, i.e. the Altar).”
Similarly, regarding our question, the Orchot Chaim (Hilchot Bet HaKenesset, Section 7) writes in the name of Maharam of Rottenberg that one may not enter a synagogue with a long knife, for prayer prolongs one’s life and a knife shortens it. Maran HaShulchan Aruch (Chapter 151) rules in accordance with the view of Maharam of Rottenberg that one may not enter a synagogue with a long knife.
Maran Harav Ovadia Yosef Shlit”a (in his Responsa Yechave Da’at Volume 5, Chapter 18) explains that the Maharam does not prohibit entering a synagogue with a long knife only because a knife shortens one’s life; rather, he has an additional reason for this which is due to the respect of the synagogue, for it is disrespectful to enter a synagogue when one is armed with a weapon. He proceeds to support his view.
The Turei Zahav asks: Why is it that it is only prohibited to enter a synagogue with a “long” knife; it would seem that the same reason would apply to a short knife as well and it should thus be prohibited to enter a synagogue even with a small knife? He explains that since removing one’s knife upon entering a synagogue is a great hassle, our Sages were only stringent regarding a long knife, as entering a synagogue with it is more disrespectful.
Based on this, Maran Harav Shlit”a writes that when there is a pressing need for one to carry his weapon and removing it proves to be a great hassle, it is not forbidden to enter a synagogue with it, similar to the law of a small knife. He proceeds to bring sources and proofs to the fact that as long as there is a great need for one to carry the weapon, there is no prohibition to enter the synagogue to pray while armed with it.
Summary: It is preferable that an armed soldier entering a synagogue to pray cover his weapon with his clothing. If this is not possible, for instance, because he is armed with a large rifle, and there is a great need for him to be carrying his weapon, such as due to security concerns and the like, he may be lenient and enter the synagogue to pray in this manner.