There are a few in Norfolk in 1861 but nothing conclusive. His lack of POB doesn't help
If he stayed in Mileham, this also looks likely and would match one of the deaths in post #14..
BI:
18 Apr 1856
James WRIGHT
80
Norfolk
Mileham, St John the Baptist
Geoff
Thanks Geoff.
Probably more likely it's that one.
I can't find a tree with James.