In 1541, four medieval houses on the site of the current palace were damaged during the Hradčany fire. One of them housed the workshop of Master Theoderic, the court painter of Charles IV, in the 14th century. Later, Johann IV Popel of Lobkowicz bought the plots and built a North Italian style Renaissance residence which was completed in 1567. Over the following decades, the palace had several owners and eventually was inherited by the Schwarzenbergs in the early 18th century.
In the years 1870–1874, it received its iconic sgraffito decoration based on the design of Josef Schulz. In 1909, it opened to the public as the National Technical Museum. During World War Two, the Wehrmacht turned it into a military museum, and after the confiscation by the Czechoslovak state it became the seat of the Military History Museum. The National Gallery Prague acquired the palace in 2002.
