In April 2013, Carole Ann Ford revealed that producer John Nathan-Turner had initially insisted that Susan not refer to the Doctor as her grandfather: "You will not believe why. They said, 'We don't really want people to perceive him as having had sex with someone, to father a child.' I just screamed with hysterical laughter and said, 'In that case, I'm not doing it.'" The script was changed to include mentions of the characters' relationship.
Tom Baker declined to return as the Fourth Doctor because he felt there hadn't been a long enough gap between him leaving the role and shooting commencing. He was represented on screen with previously unseen excerpts from Doctor Who: Shada (1992), a story which had been cancelled during its production because of a BBC strike. Consequently, The Five Doctors was one of the few episodes to be helped by a BBC strike. For publicity shots featuring all five Doctors together, the BBC used Baker's mannequin from the famous Madame Tussauds waxwork museum in London.
In the March 2008 issue of Doctor Who Magazine, Russell T. Davies expressed interest in bringing back the Raston Warrior Robot in the new series of Doctor Who (2005), citing the battle between the Robot and the Cybermen in "The Five Doctors" as one of the finest in the show's history.
Louise Jameson (Leela) offered her services but there was no opportunity to insert her into the storyline.
Temperatures on location filming were so cold that at one point Elisabeth Sladen had to be asked to slap Jon Pertwee in the face, as he had begun to turn blue.