You can fix it with these steps:
Select the text frame with the standard cursor, or select the text itself with the text cursor and the press »CMD+A« (macOS) or »CTRL+A« (PC) to select ALL text in connected text frames
Go to -> Menu -> Windows -> Content Properties or just hit the »F3« key
Select your font, select next text frame, etc.
With this trick, I managed to eliminate the »default font« during the PDF generation