Born in 1954 in Pittsburgh, Nelson started playing the vibraphone at age 15 after being introduced to the instrument by a school friend's father, George A. Monroe. A steel worker by day, Monroe was also a talented vibraphonist in the Milt Jackson tradition. He gave Nelson crucial early instruction in jazz and even started him playing piano. Looking to focus on his jazz interests, Nelson dropped out of high school and by the early '70s was gigging regularly, honing his skills alongside such local Pittsburgh luminaries as drummers Roger Humphries and J.C. Moses, trumpeter Tommy Turrentine, saxophonists Eric Kloss and Nathan Davis, and guitarist Jerry Byrd. After several years in Pittsburgh, he moved to New Jersey where he lived with his brother while seeking out more opportunities around New York city. It was during this period that he sat-in on an outdoor concert at Rutgers University featuring professors pianist Kenny Barron, bassist Larry Ridley, and drummer Ted Dunbar. Impressed, the professors helped Nelson gain entrance to the school's then-fledgling jazz degree program, where he eventually graduated and earned a Master's degree. Along with the school, Nelson continued to find work. He made his recorded debut in 1976 on James Spaulding's Plays the Legacy of Duke Ellington, and found himself busy throughout the early '80s working with Barron, David "Fathead" Newman, Bobby Watson, and others.