Go to paradise and play pasture pool. My guess is once you see the place, your thoughts of golf will long be gone. Too much beauty, too many friendly people and too much beer for some. Leave the clubs at home, I dout you will touch them once there.
I think the closest course(s) are in Ridgway, CO, about 10-15 miles north of Ouray. I don't know much about them (like if they are public or private), but if you search for Ridgway CO Golf Courses you'll quickly find that google is much smarter than I am.
I can understand the other's sentiments that you'll likely forget about your clubs once you get out on the trails, but I'm all for having your vacation filled with exactly what you like to do! If that's combining Wheelin' and Puttin' (or Drivin' and Drivin'), then definitely mix the two together.
(I'd personally like to mix 4-wheeling with some scuba diving, but the coral reefs in Lake Como are lousy at 12,000 feet)