The second song that Fyvush Finkel (Wambaugh) sings at O'Calley's Bar is entitled "The Vatican Rag" written and recorded by Tom Lehrer in 1965 and released on his album 'That Was The Year That Was'.
The show actually uses three songs from Tom Lehrer's 1965 album "That Was the Year That Was" in the space of just two episodes: In Mr. Dreeb Comes to Town (#1.3), Sheriff Brock (Tom Skerritt) and Steven Pankow (Tuck Milligan) sing "New Math", while in this episode, Douglas Wambaugh (Fyvush Finkel) sings both "Who's Next?" (about the proliferation of nuclear weapons) and "The Vatican Rag" at O'Calley's Bar.