This has been problematic for many that have tried before you. This problem is way, waaaay overdue for a resolution from Toyota. The charge timer controller in the RAV4 EV is apparently very, VERY buggy! Most likely the problem is with the software (i.e. firmware) that is used to "schedule" battery charging operations. This has caused various reoccurring problems for many here on this forum. The bottom line, Delayed Start Charging is extremely unreliable. I believe EVERY car has the same problem, and there is currently NO SOLUTION. Sadly, we all have been waiting patiently for Toyota to fix this problem for well over six months!
There are several related subject matter postings on this forum about the faulty charge timer issue, which you can read to gain a greater understanding of the problem. Unfortunately, there are more questions than answers, including several speculative theories about the root cause. It is completely ridiculous how long Toyota has been aware of this problem, or quite possibly in denial that such a problem even exists. Toyota has thus far issued to the dealer network one or more so-called RAV4 EV Battery Charger "updated" that have totally failed to resolve the problem. This has probably been the result of a massive finger pointing exercise between Toyota and Tesla.
Stay tuned . . .