Born in London, the son of an industrial chemist and his wife, Rutter grew up living over the Globe pub on London's Marylebone Road. He was educated at Highgate School, where a fellow pupil was John Tavener, before reading music at Clare College, Cambridge, where he was a member of the choir. He served as direcor of mustc at Clare College from 1975 to 1979 and led the choir to international prominence.
In 1974, Rutter visited the United States at the invitation of choral musician Melvin (Mel) Olson and conducted the premiere of his cantata "Gloria" in Omaha, Nebraska, in the Witherspoon Hall of Joslyn Art Museum. The composition, commissioned by Olson's Voices of Mel Olson chorale, has become a much-performed favourite over the years.
In 1981, Rutter founded his own choir, the Cambridge Singers, which he conducts and with which he has made many recordings of sacred choral repertoire (including his own works), particularly under his own label Collegium Records. He resides at Duxford in Cambridgeshire and frequently conducts many choirs and orchestras around the world.
In 1980, he was made an honorary Fellow of Westminster Choir College, Princeton, and in 1988 a Fellow of the Guild of Church Musicians. In 1996, the Archbishop of Canterbury conferred a Lambeth Doctorate of Music upon him in recognition of his contribution to church music. In 2008, he was made an honorary Bencher of the Middle Temple while playing a significant role in the 2008 Temple Festival.
From 1985 to 1992, Rutter suffered severely from myalgic encephalomyelitis (ME, or chronic fatigue syndrome), which restricted his output; after 1985 he stopped writing music on commission, as he was unable to guarantee meeting deadlines.
Rutter also works as an arranger and editor. As a young man he collaborated with Sir David Willcocks on the extraordinarily successful Carols for Choirs anthology series.
He was inducted as a National Patron of Delta Omicron, an international professional music fraternity in 1985.
Rutter is also a Vice President of the Joyful Company of Singers.