Intermittent duty means a lot of high current operation, and sparking at the brushes as the motor operates at something other than its rated speed while accelerating.
Sparking occurs when the commutator breaks the current, and can be minimised by ensuring this happens when the induced voltage is zero, which happens at a different brush position depending on the motor's speed. So, the brush position on large motors is adjustable, and you have to set it to the neutral point to minimise sparking, (pdf) but possibly factory set for best operation at normal speed on your motor.
So operating the motor at other speeds, the neutral point is in a different place, and you will see sparking during starts. which will wear the brushes. Now sparks are small arc lights, hot plasma accelerating across the electrical field and colliding with the electrodes (in this case, the brush and the commutator at each end). And arcs eat anodes - twice as fast as they eat cathodes, in the linked article.
In 1876 Paul Jablochkoff, a Russian living in Paris, produced what was considered at the time to be a big breakthrough, [... in electric arc lighting.] His device was known as the "electric candle", since it involved two parallel carbon rods in an upright position separated from each other by a layer of Plaster of Paris. As the carbons were consumed, the positive rod was eaten away at twice the speed of the negative rod so alternating current was preferred to equalise the erosion. In other later arc lamps the positive carbon rod was made twice the size as the negative rod.
So at any one moment, one brush is positive while the commutator is negative, and at the other brush, the commutator is positive while the brush is negative. Now "arcs eat anodes" means the positive brush will sustain more damage than the other. (The copper commutator will also sustain some damage, but it is evenly spread as it rotates).
At full power, your motor operates off AC, so the brushes are taking turns (50 times per second) in being positive, and so the wear is even.
However a series diode upsets this fairness : one brush is always positive, and thus wears faster than the other.
So what you are probably seeing is a combination of two effects : unusually bad sparking by operating the motor through a lot of acceleration (high current and away from its neutral point) and differential erosion of the positive brush because your diode "speed controller" operates the motor off DC.
Either swap the brushes periodically, or reverse the diode every day (use a DPDT switch to make this easy), or install a kinder sort of speed control - like a triac based dimmer, though cheap ones may not be robust enough to drive inductive loads like motors.