Head Coveringin the Jewish tradition it is a sign of piety to keep the head covered. The most familiar form of head covering is the yarmalke, a skullcap. The word "yarmalke" comes from the Aramaic phrase meaning "fear of the king." One is supposed to constantly be mindful of God's presence.
Source: Binyomin Kaplan