All the previous posts are good, especially Scubacruz's observations on perceptions of the impact of weight change.
Weight loss from the body or bike probably leads to similar improvement in performance. Loosing weight from the body has the performance advantages of lowering center of gravity and decreasing effort related to supporting that weight. Based on that, there may be a marginal advantage to body weight loss in most cases.
As for decreasing bike weight, people put higher value on decreasing unsprung and rotational weight. Unsprung mass is anything below the suspension; lowering it helps wheels and shocks react more quickly to bumps. Rotational mass is anything attached to the wheels; lowering weight here improves acceleration more than decreasing weight elsewhere because you are decreasing mass you must move forward and spin. In bikes, I suspect the benefits of decreasing unsprung or rotational weight are marginal and may only be important in competition or longer rides.
wind whistles in an empty ear