Educated at the École des Beaux-Arts, architects John Carrère and Thomas Hastings employed location, building orientation, and symbolism in the design of Whitehall and its grounds in order to evoke the sense of a temple to Apollo. As homes for Apollo's Muses of literature and the arts, temples to Apollo were the world's first museums (the word museum literally means home of the Muses) and in the strictest sense of the word Whitehall was Florida's first museum.

While the fact that a very deliberate plan for the grounds was an integral feature of the overall project would have been apparent to nearly anyone with a nineteenth century eighth-grade education, it is not obvious to the majority of Whitehall's visitors in the twenty first century.

Built on landfill extending into Lake Worth, Whitehall's location serves as subtle reference to Delos, the island birthplace of Apollo. The Doric columns of Whitehall's main façade are typically associated with temples to Apollo, and Whitehall's east-facing orientation as well as the lion heads (ancient symbols of the sun) at the center its massive bronze doors are again references to Apollo, the sun god.

Because a natural or bacchanalian environment was classically associated with temples to Apollo, instead of a formal garden Whitehall was situated amid a coconut palm grove. The walkway approaching Whitehall was originally very broad, as would be appropriate to a classical building. However, during the period between 1925 and 1955 when Whitehall operated as a luxury hotel, a grass median was cut into the walkway. Flanking the end of the walkway closest to Whitehall are two large urns carved in high relief with bacchanalian scenes, further reinforcing the sense that the grounds around the building represented the natural world, as opposed to the highly ordered world of Apollo's Muses found inside a temple to Apollo.

Upon entering Whitehall, the natural or bacchanalian world is left behind as one enters the highly ordered world of Apollo and the Muses of literature and the arts. There are many references throughout the first floor of Whitehall to Apollo and the Muses of literature and the arts, the most obvious of which is the domed ceiling at the center of the Grand Hall, which depicts the Oracle of Apollo at Delphi.

Whitehall's classical floor plan incorporates an atrium and garden. Rather than a flowering or formal garden, the atrium garden is a classical reference to paradise and is lush with subtropical plants and  water. The two-tired fountain at the center of the garden is a copy of the Venus fountain in the Boboli Gardens of Florence.

As a part of Whitehall's original configuration there was briefly a formal rose garden off the back porch. However, roses did not do well in the subtropical climate and very little information has survived about that small garden. When a hotel tower was added to Whitehall in 1925, the area off Whitehall's back porch originally dedicated to a formal garden became the dining room of the new Whitehall Hotel.

Whitehall was opened to the public in 1960. During the succeeding years the Hotel tower was removed and the atrium garden was restored. Though the grounds today are somewhat informal and therefore more or less in keeping with the architects' original vision, the driveway that was added during the Hotel Era remains, and the Cocoanut Grove was restored in summer 2015.