Leroy was employed in one of the many Milwaukee breweries when he enlisted in the Army in Milwaukee on October 6, 1942. 

He must have had some existing musical talent as he was first assigned to the 505th PIR band and was then transferred to the 508th's own fledgling unit band on January 27, 1943, just three months after his enlistment.

Zimmerman apparently remained with the band throughout its stint with the 508th until he and other band members were lost to the reorganization into the 112th Army Ground Forces Band in May 1944.