Bury went to Eton, became an officer in the Scots Guards and in 1850-51 was private secretary to prime minister Lord John Russell. In 1854 he went to Canada as superintendent of Indian affairs and later wrote a two-volume history of north America which concentrated on Canada. He was Liberal MP for Norwich (1857-1860), for Wick Burghs (1860-65) and Berwick (1868-74). He was Treasurer of the Household, in effect a government whip from 1859 to 1866 under Palmerston and Russell. However he underwent political and religious conversions, joining the Conservatives in 1875, later serving twice at the War Office and to Roman Catholicism in 1879, following which he had a public row with Gladstone about the Vatican Council.