Mysterious Island (1961)

During the Civil War a group of Union soldiers, a Confederate and a civilian escape the stockade using a hot-air balloon and end up on a strange Pacific island.