Sounds like that LCA bushing is the culprit causing your issues, assuming the rest of the arm is in good shape. You shouldn't need to take the arm off to get at that bushing... just the bolt thru the bushing first, then two bolts into the chassis to remove the LCA bushing and mount. Then press a new bushing into the mount and re-install.
To remove the whole LCA, I don't bother with the axle/spindle. I just disconnect the sway bar, remove the upper half of the ball joint from the spindle, then remove the front bushing bolt and the two rear attachment chassis bolts from the LCA to take it off. Reverse the order to reinstall, but make sure to leave everything a little loose until it's all lined up. Then tighten in the order from the manual (which I don't remember off hand). I think the car is supposed to be on the ground when torquing things down.
I'm assuming the '02 WRX service manual procedure is the same on your impreza:
http://www.seccs.org/tech/