Baker's Cottage was built in 1865 as the residence of William Baker and his family. The cottage once stood in the middle of a well-kept garden and orchard. The cottage itself was once larger, with more rooms and a verandah surrounding it. The residence changed hands only twice, and was then appropriated by the Minister of Lands in 1938.