In a crash, the doors are essential. While driving (including all wheeling), I don't believe they are any part of the structure whatsoever.
I want to do this very much, but there are many challenges that become more apparent when you really look at it. Wires, seatbelt, rear door or not, etc... also, considering the way my paint looks I don't think the seats would last a couple outings.
