Apart from guitarist Charlie Christian, Green’s primary influences were saxophonists, particularly Charlie Parker, and his approach was therefore almost exclusively linear rather than chordal. He thus rarely played rhythm guitar except as a sideman on albums led by other musicians. The simplicity and immediacy of Green’s playing, which tended to avoid chromaticism, derived from his early work playing rhythm and blues and, although at his best he achieved a synthesis of this style with bop, he was essentially a blues guitarist and returned almost exclusively to this style in his later career.

Green was born in St. Louis, Missouri. He first performed in a professional setting at the age of 13 as a member of a gospel music ensemble. His influences were Charlie Christian, Charlie Parker, Lester Young, and Jimmy Raney, he first played boogie-woogie before moving on to jazz. His first recordings in St. Louis were with tenor saxophonist Jimmy Forrest for the United label. The drummer in the band was Elvin Jones, later the powerhouse behind sax player John Coltrane. Green recorded with Elvin again in the early 1960s. Lou Donaldson discovered him playing in a bar in St. Louis. After touring together with Donaldson, Green arrived in New York around 1959–60.