The Orkhon valley was a centre of the Xiongnu, Göktürk and Uyghur empires. To the Göktürks, the nearby Khangai Mountains had been the location of the Ötüken, and the Uighur capital Karabalgasun was located close to where later Karakorum would be erected (downstream the Orkhon River 27 km north–west from Karakorum). This area is probably also one of the oldest farming areas in Mongolia.

In 1218–19, Genghis Khan rallied his troops for the campaign against the Khwarezm Empire in a place called Karakorum, but the actual foundation of a city is usually said to have occurred only in 1220. Until 1235, Karakorum seems to have been little more than a yurt town; only then, after the defeat of the Jin empire, did Genghis’ successor Ögedei erect walls around the place and build a fixed palace.

Ögedei Khan gave the decree to build the Tumen Amgalan Ord (Palace of Myriad Peace, Wan’an’gong in Chinese) in 1235 the year after he defeated the Jin Dynasty. It was finished in one year. In the Yuanshi (元史) it is written in the section for Taizong (太宗) Ögedei Khan: In the seventh year (1236), in the year of the blue sheep the Wanangong 萬安宫 was established in Helin 和林, Karakorum. One of Genghis Khan’s nine ministers the Khitan Yelü Chucai(1190–1244) said the following poem during the ridge raising ceremony of the Tumen Amgalan Ord: “Installed ridge well fit and stone foundation, The parallel placed majestic palace has been raised, When the bells and drums of the Lord and officials sound pleasantly, The setting sun calls the horses of war to itself from the mountain peaks.” The Mongolian version of the poem is as follows: “Tsogtslon tavih nuruu chuluun tulguur, Zeregtsen zogsoh surleg asriig bosgovoi, Ezen tushmediin honh hengereg ayataihan hanginan duursahad, Echih naran uuliin tolgoigoos dainii agtadiig ugtnam.

The name Karakorum or “Kharkhorin” literally translates to ‘black-twenty’. But linguists argue that the ‘khorin’ might have been a diversion of the word ‘khurem’, which means “castle” in Mongolian. Other translations vary.

Under Ögedei and his successors, Karakorum became a major site for world politics. Möngke Khan had the palace enlarged, and the great stupa temple completed. They had the Parisian goldsmith, Guillaume Bouchier, design the Silver Tree of Karakorum for the city centre. A large tree sculpted of silver and other precious metals rose up from the middle of the courtyard and loomed over the palace, with the branches of the tree extended into the building. Silver fruit hung from the limbs and it had four golden serpents braided around the trunk, while within the top of the tree was placed a trumpet angel, all as automata performing for the emperor’s pleasure. When the khan wanted to summon the drinks for his guests, the mechanical angel raised the trumpet to his lips and sounded the horn, whereupon the mouths of the serpents began to gush out a fountain of alcoholic beverages into the large silver basin arranged at the base of the tree.

William of Rubruck, a Flemish Franciscan missionary and papal envoy to the Mongols reached Karakorum in 1254. He has left one of the most detailed, though not always flattering, accounts of the city. He compared it rather unfavourably to the village of Saint-Denis near Paris, and was of the opinion that the royal abbey there was ten times as important as the Khan’s palace. On the other hand, he also described the town as a very cosmopolitan and religiously tolerant place, and the silver tree he described as part of Möngke Khan’s palace as having become the symbol of Karakorum. He described the walled city as having four gates facing the four directions, two quarters of fixed houses, one for the Saracenes and one for the Cathai, twelve pagan temples, two mosques, as well as a Nestorian church.

When Kublai Khan claimed the throne of the Mongol Empire in 1260—as did his younger brother, Ariq Böke—he relocated his capital to Shangdu, and later to Khanbaliq (Dadu, today’s Beijing). Karakorum was reduced to the administrative centre of a provincial backwater of the Yuan dynasty founded in China in 1271. Further to that, the ensuing Toluid Civil War with Ariq Böke and a later war with Kaidu hit the town hard. In 1260, Kublai disrupted the town’s grain supply, while in 1277 Kaidu took Karakorum, only to be ousted by Yuan troops and Bayan of the Baarin in the following year. In 1298–99 prince Ulus Buqa looted the markets and the grain storehouses. However, the first half of the 14th century proved to be a second time of prosperity: in 1299, the town had been expanded eastwards, then in 1311, and again from 1342 to 1346, the stupa temples were renewed.

After the collapse of the Yuan dynasty in 1368, Karakorum became the residence of Biligtü Khan in 1370. In 1388, Ming troops occupied and later razed the capital. According to Saghang Sechen‘s Erdeni-yin Tobči, in 1415 a khuriltai decided to rebuild it, but no archaeological evidence for such a venture has been found yet. However, Karakorum was inhabited at the beginning of the 16th century, when Batu-Möngke Dayan Khan made it a capital once again. In the following years, the town changed hands between Oirads and Chinggisids several times, and was consequently given up permanently.

The Erdene Zuu Monastery stands near Karakorum. Various construction materials were taken from the ruin to build this monastery. The actual location of Karakorum was long unclear. First hints that Karakorum was located at Erdene Zuu were already known in the 18th century, but until the 20th century there was a dispute whether or not the ruins of Karabalgasun, or Ordu-Baliq, were in fact those of Karakorum. In 1889, the site was conclusively identified as the former Mongol capital by Nikolai Yadrintsev, who discovered examples of the Orkhon script during the same expedition. Yadrintsev’s conclusions were confirmed by Wilhelm Radloff.

The first excavations took place in 1933–34 under D. Bukinich. After his Soviet-Mongolian excavations of 1948–49, Sergei Kiselyov concluded that he had found the remains of Ögödei’s palace. However, this conclusion has been put into doubt by the findings of the 2000–2004 German-Mongolian excavations, which seem to identify them as belonging to the great stupa temple rather than to Ögödei’s palace.

Excavation findings include paved roads, some brick and many adobe buildings, floor heating systems, bed-stoves, evidence for the processing of copper, gold, silver, iron (including iron wheel naves), glass, jewels, bones, and birch bark, as well as ceramics and coins from China and Central Asia. Four kilns have also been unearthed.

In 2004, then Prime Minister Tsakhiagiin Elbegdorj appointed a working group of professionals to develop a project to build a new city at the site of the ancient capital Karakorum. According to him, the new Karakorum was to be designed to be an exemplary city, with a vision of its becoming the capital of Mongolia. After his resignation and the appointment of Miyeegombyn Enkhbold as prime minister, the project was abandoned.