An update as I visited an indie shop nearby and an official Audi dealership.
The local mechanics had discovered it was the coolant shut-off valve 4H0 121 671D. A simple solenoid, which intermittently gets stuck in a semi-closed or fully closed position. As I understood the schematics, this valve is preventing the (cold) coolant from circulating through the heater to make the engine warm up faster. According to multiple other forums' threads, it is very often stuck but many times in the open position, which of course goes unnoticed by owners. The worst is when an older revision part (B at the end) starts leaking and pushing coolant through the socket onto the wires, but this didn't happen in my case. All electrics is intact.
Diagnostics (and temporary fix) are fairly simple: knock on it with something chunky such as a screwdriver handle a few times and observe the heat coming back.
Importantly, it wouldn't show in the errors (unless the electrics is damaged and the circuit is always open/short) because there's no flow sensor and the device itself is passive. The only error related to the AC/heater I had was B108C07 related to the motor controlling the left air flap. And I am very glad I went to an indie shop first because I suspect the dealership, even if things were covered by the warranty, would do unnecessary replacements of the heater core, pump, thermostat, etc eventually damaging the system's integrity.
To me, it was very confusing because it really reminded of a possible airlock, especially since there was a correlation between me adding coolant/revving up the engine and the heater getting back to life. Perhaps a stronger coolant flow would indeed push the valve into its place, who knows. The aux pump mentioned in the first message was suspicious too because of the outgoing hose being cold, but this was probably due to the closed valve, which also makes sense in retrospect.
Anyways, the dealership are now requesting if the replacement is covered by the warranty and will fix it, including flushing and re-filling the entire cooling system. I am a bit skeptical about the latter but I guess it's ok as soon as they know what they're doing.