This is the first time Batman (i.e., the adult Bruce Wayne) appears, albeit only as a hallucination and entirely unidentifiable. This is also the first time Jim Gordon appears with his comic book trademark mustache, also as a part of a hallucination.
The Penguin's prison uniform shows the number 58. The character made his first appearance in Detective Comics #58 (December 1941).
Bruce Wayne has a bust of Shakespeare on his mantelpiece, almost identical to the one used in Batman (1966) to hide the Batpole controller.