Brian tells Kevin that he hadn't contacted social services to tell them he was making contact with Olivia. If he hadn't talked to social services, he shouldn't have known Olivia had been placed with Kevin and Scotty, or known where they live and work.