Scherchen was one of the leading conductors in the middle part of the twentieth century, especially valued for his pioneering performances of the contemporary music of his time. He was essentially self-taught as a musician and became a violist in the Blüthner Orchestra and the Berlin Philharmonic when he was 16. In 1911 he was an assistant to Arnold Schoenberg in the preparation of Pierrot Lunaire for performance. Following its Berlin premiere, the piece was taken on a tour in which Scherchen conducted. He became the conductor of the Riga Symphony Orchestra in 1914, but was soon interned by the Russians as an enemy alien when World War I started. He returned to Germany after Russia left the war to found the Neue Musikgesellschaft and the Scherchen Quartet. In 1919 he founded a militant magazine Melos.
He succeeded Furtwängler as the director of the Frankfurt Museum Concerts in 1922 and in the same year began a long relationship with the Winterthur Musikkollegium in Switzerland. From 1928 to 1933 he was the Generalmusikdirektor in Königsberg. He frequently conducted contemporary music festivals, especially with the International Society for Contemporary Music, with which he was connected from its founding in 1923. Among his premieres in the 1920s and 1930s were the Three Fragments from Wozzeck by Berg and the quarter-tone opera Mother by Alois Haba. He left Germany immediately upon the rise of the Nazis to power in 1933, settling in Switzerland, where he became music director of the Zurich Radio Orchestra and also gave courses in conducting, which became a regular summer school in Switzerland in 1939. In the same year he founded the Ars Viva Orchestra. He married the Chinese composer Hsiao Shu-sien. They had a daughter, Tona Scherchen-Hsiao, born in 1937, who went back to China with her mother in 1949. She became a noted composer, especially after she moved to France in 1972.
Scherchen resumed his continent-wide activities after World War II ended. He was director of the Zürich Radio Orchestra (1944-1950) and in 1950, with the support of UNESCO, opened a studio for electroacoustic research in 1954 in Gravesano, the village where he lived. He continued his writing about new music in the Gravesano Blätter. Unlike many conductors of his generation his "new music" was not merely the new music of his youth, but the continuing evolution of new music. In the 1950s he conducted the premieres of such works as Dallapiccola's Il prigioniero, Dessau's Das Verhör des Lukullus, and Henze's König Hirsch. He was the first to play any music from Schoenberg's Aron und Moses in Darmstadt (1951), edited it for its first performance under his colleague Hans Rosbaud, and led its first performance in Berlin. He did not appear in the United States until 1964 when he conducted the Philadelphia Orchestra.
He was the author of a leading text on conducting and of many articles supporting modern music. He suffered a heart attack while conducting Malipiero's Orfeide in Florence and died four days later.