The noise may come from the fiber.
I checked with a sound generator on my phone and put the speaker near the fiber. I see the peak at 4690Hz (random choice, but the same thing happen for other frequencies) on the spectrum (see pictures). Some acoustic noise is coupled to the fiber. So I fixed the fiber with some tape.

I put the OD2 filter to reduce the power, then a non-polarizing BS to send half of the beam on the in-loop PD.
I put the in-loop PD on a separate optical board standing on a periscope post because there was no space on the breadboard.
Before the in-loop PD, I put a f=50mm lens. The in-loop PD is mounted on an XY lens mount for a fine centering of the beam on the detector.
I put the out-of-loop PD on the translation stage and moves it to center the beam inside the PD.

Both PDs have a load of 7.5kOhm.
On the oscilloscope, the two signals (AC coupling) are very similar. Also, the coherence is high.
I closed the loop but the noise doesn't reduce on the out-of-loop PD.

I took the spectra with 400 peakhold averages, not linear averages, because I'm interested in reducing the nose peaks, the spikes due to the jumps.
I really need to understand where these "jumps" come from.