Maphis was born in Suffolk, Virginia, United States. Joe’s family moved to Cumberland, Maryland, in 1926 when his father Robert Maphis landed a job with the B&O Railroad. Joe’s first band was called the Maryland Rail Splitters. He also played in the local (Cumberland) Foggy Mountain Boys as well as The Sonnateers before Maphis hit the road in 1939. He played across Virginia until he became a regular featured performer on the “Old Dominion Barn Dance,” broadcast live on radio WRVA-AM and aired in 38 states.

In 1944, Joe went into the U.S. Army. His musical skills landed him a gig entertaining the troops around the world. Maphis was discharged from the Army in 1946. On his return to the states, he began playing on WLS radio in Chicago. In the late 1940s, he returned to Richmond, Va. and the Old Dominion Barn Dance until the early 1950s.

During this period Maphis met many country music stars of the day who played the same circuit including Merle Travis. While in Virginia, he also met the musically talented Rose Lee (Doris) Schetrompf, his future wife. Maphis and Schetrompf, of Clear Spring, Md., were married in 1953. A talented vocalist and rhythm guitarist, Rose backed Joe onstage throughout the remainder of his career.