I noticed that the amp-meter didn't jump up while driving. A quick test with a multi-meter learned that the generator supplied around 0.30 volts at full speed. Normally this should be a bit over 6V.
As the generator is pretty straight forward I removed it and measured the output while turning the generator manually. It was around 0.2 volts. I opened the cover and it turned out that one of the brushes was totally flat. As I have a rusty spare (unrestored) generator I swapped the brush from the unrestored generator and turned the generator by hand. Now it was already outputting 0.13V which is a great improvement. Bear in mind that these generators need a few RPM to get voltage so it's normal that the output is very low with low RPM. After installing the generator back into the car the amp-meter was back at full potential.
The old rusty generator; left shows a proper (but old) brush and right is the worn out brush.
This was a quick fix but I should take a better look at it soon. The quick worn out of the brush probably has a reason ...