When I bought mine @ 49k miles, it was not silent. I was prepared for that, before I bought the car.
I commute ~1400 mi./month, and over the next eleven months the miles racked up and the noisyness got louder, but never really highly-annoying to me.
The original warranty covers the DU through 60k, so at 59k I took it to my somewhat-local Toyota dealer (Toyota of Seattle). Because a) they'd serviced the car earlier for the wipers linkage recall, and b) they had had at least three RAV4EV DUs replaced on other local RAV4EVs before mine, they knew the process for DU replacement authorization, and they knew me as a customer (I, unfortunately, get oil change-special-coupons via mail every three months from them).
When I dropped it off, I complained about both low- (up to ~20 MPH) and high-speed (at around 60) noise. The repair ticket I later received declared the high-speed noise/whine to be within normal parameters, and the low-speed noise to be excessive: the DU replacement was authorized. My hearing is fairly poor, and I blast NPR on my commute, so I don't really mind the highway whine, but the low-speed noise -- esp. accelerating from stops -- was getting fairly annoying, esp. in the summer with the windows down, and is bad PR for EVs.
The dealer did not wait for the roving Toyota EV tech to come listen to it. I assume they now know to use some other metric or process (make a recording, use their own expertise, IDK) to confirm or deny that the noise is "loud enough" to authorize DU replacement. When I picked it up some weeks later, the service advisor commented that "they all have to be replaced", probably because every RAV4EV that he had seen had to have had it replaced.
It was pretty painless. But that's likely because other RAV4EV owners had already trained this dealer about this issue. I have heard about Toyota dealers declining a DU replacement due to noise, but that appears to be rare.
As a byproduct, they changed the DU coolant, which is a 50k service I declined earlier, knowing I was going to have the DU replaced soon.