When Betsy is talking with her riding companion outside of the stables, her chinstrap alternates between being under her chin and above the brim of her riding hat throughout the same conversation.
The handset of Mrs. Barrett's phone is connected to the base with a modular connector. These were not introduced until the mid-1960's and only on Trimline phones.
It is no later than 1962, because the next episode takes place on April 8, 1962, but the song being played is "Lollipops and Roses" which was released January 1, 1964.
While the series is set in New York state, a palm tree is briefly visible behind Betty Draper during the initial scene at the riding stables. The series is filmed in California.
When Betty talks to Jimmy Barrett at the dinner conversation about her horse riding, she mentions that she doesn't do dressage but rather jumping. At some previous scene at the tables where we can see her in the saddle, the saddle type and the stirrup length both are clearly for dressage, not for jumping.