The song (Big Yellow Joint) playing while Oscar (Jeffrey Tambor) smokes pot is a Banana Stand reference introduced in Season 1.
Mrs. Featherbottom (Tobias) (David Cross) says his car was shipped over from Blackstool. Blackpool is a place in England but Blackstool is not. Blackstool is a medical condition which human faeces are an unusual color, usually black (hence the name).
Buster (Tony Hale) says that his family is going to the Church of the Good Shepherd. In episode 1.15, Staff Infection (2004), the office workers boarded a Church of the Good Shepherd bus (meant for Lupe (B.W. Gonzalez)'s family). Also in that episode, the man in the stair car hit his head on a sign that said Church of the Good Shepard.
The Veals have a painting on the wall of a girl with no face. One of Maeby (Alia Shawkat)'s nicknames for Ann (Mae Whitman) is Annie McNoFace.
The song that plays when Franklin is introduced and subsequently appears has the lyrics "Come on!", which is Gob (Will Arnett)'s catchphrase.