Construction began on Pavilion II in the summer of 1819 by Philadelphia carpenter Richard Ware. The design of Pavilion II is based on the Ionic order used at the Temple of Fortuna Virilis in Rome, illustrated in The Architecture of A. Palladio; in Four Books. Construction was complete on all of the pavilions and dormitories by October of 1822; however, the Ionic column capitals for Pavilion II were carved out of Carrara marble in Italy and were not installed until September of 1823. The pavilion's original plan was unusual even for Jefferson; it was the only pavilion to have the main entrance open into a small vestibule instead of a room or hall, and the only one to have a freestanding stairway to the second floor.

Pavilion II underwent many changes, beginning as early as 1829 with some minor interior alterations. The first major renovation did not occur until June of 1892 when Dr. William C. Dabney was assigned to the pavilion; a large two-story addition with a flat roof was added to the back, making the plan of Pavilion II L-shaped. Circa 1920, the 1892 addition was removed and another addition (a basement and first floor level) was built in its place. In 1953, Frederick C. Nichols supervised alterations to restore many interior Jeffersonian elements, but the exterior maintains its circa 1920 appearance.

See also: Pavilion II Photograph Gallery and Pavilion II and East Lawn Animation