Brown was born in Irvine, the sixth child of Ann Brown and her husband, James Brown, a sea captain. John was raised in a family that placed high value on education—for both sexes. He attended Irvine Academy, then University of Edinburgh and University of Glasgow. He declined a Balliol scholarship in mathematics instead taking a Snell exhibition for Classics and philosophy.
In 1874 he was selected as professor of Classics and English, one of three at the newly established Canterbury College in Christchurch, then part of the University of New Zealand. In his new role, he worked 16 hours a day, covering English literature and composition, Latin and Greek; inspecting secondary schools and examining for teachers' certificates. After a professor of classics was appointed in 1879, he took on history and political economy.
Under his patronage Canterbury College admitted Helen Connon as a matriculated student in 1876, the first Australasian university institution to admit women to degree classes on an equal basis with men. Connon was second woman in the British Empire to graduate BA and the first to receive a degree with honours. Brown and Connon married on 9 December 1886. They had two children, Viola, and Millicent, who later married Archibald Baxter and was mother to the poet James K. Baxter.
In addition, Brown also helped establish the Macmillan Brown Library, a research library, archive, and art gallery which is associated with the University of Canterbury. The Macmillan Brown Library's collections focus on New Zealand and Pacific Islands history. Brown also allocated a substantial proportion of his fortune to the Macmillan Brown Library and the University of Canterbury's Macmillan Brown Centre for Pacific Studies.