Cathryn,
Like Julie, I am unable to find your Joshua Gloyn anywhere but upon his arrival at Castle Garden (the predecessor of Ellis Island) in New York. The U.S. census he would have been most likely to appear on after his arrival would have been the one taken in 1890. Unfortunately, that one was mostly destroyed by fire. I've searched the fragments available on ancestry.com as well as a few state census fragments from around that time but can find no sign of him.
Is it possible he left the U.S. for Canada or elsewhere?
Sorry to be so unhelpful,
Nuala