The Doctor tells Donna he had been to Rome once before, and that he "had nothing to do with that fire." The First Doctor, William Hartnell, visited Rome in The Slave Traders (1965)/All Roads Lead to Rome (1965)/Conspiracy (1965)/Inferno (1965) ("The Romans"). Although The Doctor did not directly start the fire, he did hold a map a bit too close to an open flame, giving Nero an idea.
The Latin phrase the Doctor utters when Lucius tries to have him arrested is "Morituri te salutant," meaning "Those who are about to die salute you": it was the traditional salute of slaves expecting imminent death, especially said by gladiators to the head of the games just before combat began.
Karen Gillan, who appears here as the Soothsayer, would later be cast as Amy Pond, the new companion for series 5. Peter Capaldi, who plays Caecilius, would later be cast as the Twelfth Doctor. Both use English accents in this episode, but their natural Scottish accents in their later roles. The dual casting of Capaldi is explained in canon, while Gillan's is never mentioned.
When the Doctor and Donna see Vesuvius rumbling and realize that they have landed in Pompeii and not Rome, the Doctor says: "It's volcano day." This may be a reference to The Doctor Dances (2005). When explaining his con to the Doctor and Rose, Captain Jack says: "Pompeii's nice if you want to make a vacation of it, though. But you've got to set your alarm for Volcano Day." Considering the source of the Volcano Day quote, Captain Jack Harkness will be there, doing his self cleaning con.
Caecilius' purchase of the TARDIS in the belief that it's a work of modern art is a deliberate reference to City of Death: Part Four (1979), in which John Cleese and Eleanor Bron played a pair of art critics who discuss its artistic merits before being even more impressed when it dematerialises in front of them.