I like VW's post and agree with it. If you love wrestling, officiating is a great way to stay involved/give back/ keep the love affair alive.
In 9 years as an official I have had very few negative encounters with fans, and have had many positive encounters. Coaches have given me an earful from time to time, but the best of them always approach you after the fact and sometimes apologize for their over-exuberance (really, the apology is seldom necessary, their job is to advocate for their wrestlers) but always listen and try to understand what you're thinking.
Now here is the hard part. VW nailed it by saying you will be your own worst critic. As an official you always want to get it 100% right, and you almost never will. On good days your mistakes will not affect the outcome of a match. On bad days you will kick yourself for a call that was (or even just "may have been") wrong.
But in between those (hopefully) rare events you will watch outstanding wrestling up close, you'll get to interact with some of the best peple on the planet, and you will keep your love of wrestling alive, fresh, and very real.
Do it. Take the plunge.