The job scheduler right now really only handles the happy path, that is when everything succeeds. It needs to handle situations in which jobs fail. First and foremost, it needs to continue running and executing other jobs. It also needs to make sure that the job get rescheduled it it is a repeating job.
We also need to figure out what to do with failed single execution jobs. In the case of something like tenant deletion, we want it to run again until the job completes successfully.