I am a Tormey and a descendent of the King Country Tormeys. My family came from Moate, in County Westmeath. I have been trying to complete my family tree for the last couple of years - it has been quite difficult doing it from New Zealand, but I have traced it back to Peter Tormey 1705.
A lot of Tormeys changed their name to Toomey. For instance, my uncle Owen changed his when he immigrated to USA. It could have been because of the accent and a phonetic derivative at Immigration, I donít know. Downy is also a Tormey derivative.
You might be able to find information through Irish heritage sites, such as, the Westmeath Genealogy Centre, in Moate, or Roots Ireland (Roscommon). The Church in Moate also holds records apparently - Monsignor Moonan (I think) is an expert in Tormey genealogy. A new-found cousin of mine visited him in the hope of filling in some gaps.
There was a James Tormy, born 1795 (possible father or uncle of your James), son of Thomas Tormy, 1765- 1830. He has 3 brothers, one called Patrick (1798 - 1870), birth registered in Kells, Meath. That could be a starting point.
If you wish I can send you some screen clips of my family tree, but I am not sure how accurate it is that far back.