Construction began on Pavilion X and its adjacent student rooms in the summer of 1820, and was undertaken by William B. Phillips, a brick mason, and John Neilson, a master carpenter. The design of Pavilion X is based on the Doric Theater of Marcellus shown in Roland Fréart de Chambray's Parallèlle de l'architecture antique avec la moderne. The Attic parapet comes from Palladio's drawings of the Temple of Nerva Trajan (Book IV, Plate VIII of Leoni's 1721 edition of Quattro Libri dell'Architettura), as it was the only example Jefferson could find with dimensions. However, he stripped most of the ornamentation shown in Palladio's drawing, writing on the building specifications, "...that temple is overloaded with ornaments and it's [sic] pilaster frittered away so minutely in it's [sic] mouldings as to lose it's [sic] effect. I have simplified these mouldings to suit our plainer style..."

Dr. Robley Dunglison, Professor of Anatomy and Medicine, was the first resident of Pavilion X beginning in 1825, and made the first minor renovations in 1832. These renovations included a stair to the attic and a small porch to the rear. Various residents of Pavilion X through 1854 also rented the student rooms to either side of the pavilion for extra space (doorways have been located in the walls between the pavilion and the rooms), when the Board of Visitors requested that professors surrender the rooms for student use; however, there is documentary evidence that student room 52 was still being used as a pantry in 1896. The first major renovation occurred sometime after the summer of 1878, when the east addition was built at the request of John Barbee Minor. Some limited renovations were made in the late 19th century as sewers and drainage came to the University. An additional small, two story addition was made in the southeast corner between the original building and east addition by 1896. With minor changes over the years, this is the building that can be seen on the Lawn today.

See also: