自責 and 後悔: blaming oneself and regretting a past choice
Compare self-blame over a failure with continued regret about not apologizing. Preserve thought, degree, and duration without establishing responsibility or a later apology.
Assigning the blame to oneself
自責（じせき） means self-blame. 失敗をすべて自分のせいだと思い、自責の念に苦しんだ。 describes suffering from self-blame while believing the failure was entirely one's own fault.
自分のせいだと思い reports that belief; it does not independently establish responsibility. 過度な自責をやめ、次にできることを考えよう。 proposes stopping excessive self-blame and considering what can be done next, rather than reporting that it has all disappeared.
Still regretting an apology not made
後悔（こうかい） means regret. あの時謝らなかったことを今も後悔している。 says that the speaker still regrets not apologizing then. 謝らなかったこと identifies the past choice being reconsidered.
後悔のないよう、最後まで力を尽くした。 describes giving one's all to avoid regrets. The purpose of that effort does not guarantee that no regret remained.
Separate an emotion from a finding of fact
自責 can focus on assigning responsibility to oneself, while 後悔 can focus on looking back at an action or choice. Both reactions can concern the same failure, so do not make them exclusive categories.
Believing that one is to blame does not establish the actual cause. Regretting an apology not made also does not state that a later apology was offered or that forgiveness followed.
What is still regretted in あの時謝らなかったことを今も後悔している。?
Show the answer and why
Not apologizing at that time. The regret continues now; the sentence does not report a later apology or forgiveness.