Have you found them on the census in Gt Torrington,?
If you look at original census that they are on and follow the pages back to the page which describes the area covered by the enumerator, it will give names of roads nearby, and you might be able to work out whereabouts Rosemary Lane is.
I noticed that the son Stephen born 1855 died and buried in 1866 In Gt Torrington, so am guessing they were there in 1861.