You don't need to 'remove the spikes' so much as 'give the right reading'.
There are at least two good possibilities for what's happening, and they require different software filters. Then there are bad possibilities, that may require a rethink of the mechanical arrangement.
The difference between the two that can be handled in software depends on the sample rate and the sensor bandwidth. If you are sampling above the Nyquist rate for a low-pass bandlimited sensor, and your sensor is linear, then the correct filter is the mean. The spikes are part of the correct reading, and are required to balance the low readings you get either side of the spikes.
If you are sampling well below Nyquist on a wideband sensor, and the correct reading is 'most of the readings', then you do indeed need to reject the spikes. As long as the number of spikes is well below 50% of the readings, the simplest filter to use is the median, that value for which 50% of the readings are above and below. This will be slightly biassed, but not as much as a mean filter.
If you can identify the spikes and remove them from your data set before filtering, then the median will be much less biassed, and still less sensitive to any errors in the spike classification process than the mean.
If you have a situation which is neither of these extremes, then it will be very difficult by straightforward software filtering to recover the true forces, as you have contaminated the measurement at the sensor.