Originally apprenticed to Thomas Ivory, Baker worked with James Gandon after Ivory's death. He continued work on the King's Inns after Gandon resigned from the project. But his most important work was the laying out of Westmoreland and D'olier Street for the Wide Streets Commissioners with a co-ordinated design of houses over ground floor shop façades. He was also responsible for the restoration of St Audoen's Church of Ireland on High Street.