Jerusalem is important to Muslims because it is believed that the Prophet Muhammad ascended to the heavens from Jerusalem after being taken there from Mecca. Muslims also faced Jerusalem in prayer instead of Mecca because of its connection to Moses, Abraham, Jesus and other prophets.
According to Inside Islam, Muhammad did not believe that he was creating a new religion. Instead, he felt that he was furthering the religion created by earlier prophets. Because he felt that Islam shared the same theological lineage as Christianity and Judaism, Jerusalem symbolized a connection between those faiths. Muslims also believe that Muhammad met the past prophets and led them in prayer in Jerusalem, which helped to give it context within the broader tradition of monotheism.
Muhammad said that Muslims should visit the Sacred Mosque in Mecca, the mosque in Medina and Masjid al-Aqsa, which means the furthest Mosque, in Jerusalem. This has helped Muslims to continually hold Jerusalem in high esteem.
Jerusalem was captured by Muslims in 637, who made it the chief shrine of Islam after Mecca. The Dome of the Rock was constructed in the city between 688 and 691 on the site where Muslims believe that Muhammad ascended to the heavens. Wikipedia