Delete multiple AD users from Task Scheduler with PowerShell script

I'm hoping someone can assist with my issue. We have a termination script that will move users to a specific OU, disable the account, and set the expiration date to 60 days from now. That works great and we love it. What we need now is to execute a script via Task Scheduler on a daily basis to then delete those users that have Account Expires on that day's date. I have tried the following PowerShell script that launches from Task Scheduler:

From Task Scheduler

Program/Script: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe

Argument: -ExecutionPolicy Bypass -NoLogo -NonInteractive -File C:\scripts\DeleteTermUser.ps1

DeleteTermUser.ps1 Script

Import-Module ActiveDirectory

$TermDate = Get-Date

$TermUsers = Get-ADUser -Filter {AccountExpirationDate -le $TermDate} -SearchBase "OU=Terminated,OU=Function,OU=Groups,DC=columnit,DC=com"

ForEach ($TermUser in $TermUsers) {
 Remove-ADUser $TermUser -Confirm:$False
}

--------------------------------------------

If I manually run this script I get the following:

Remove-ADUser : The server is unwilling to process the request
At C:\Scripts\DeleteTermUser.ps1:8 char:15
+     Remove-ADUser <<<<  $TermUser -Confirm:$False
    + CategoryInfo          : NotSpecified: (CN=Paul Smith,...ourdomain,DC=com:ADUser) [Remove-ADUser], ADException
    + FullyQualifiedErrorId : The server is unwilling to process the request,Microsoft.ActiveDirectory.Management.Comm
   ands.RemoveADUser

If I use the -WhatIf at the end then it tells me the following:

PS C:\Scripts> .\DeleteTermUser.ps1
What if: Performing operation "Remove" on Target "CN=Jon Smith,OU=Terminated,OU=Groups,DC=ourdomain,DC=com".

What if: Performing operation "Remove" on Target "CN=Guy Johnson,OU=Terminated,OU=Groups,DC=ourdomain,DC=com".

What if: Performing operation "Remove" on Target "CN=Bob Jones,OU=Terminated,OU=Groups,DC=ourdomain,DC=com".

I have numerous scripts running from Task Scheduler with no problem (all with Highest Privileges), but I can't wrap my head around this one.

Thanks in advance.

~Rick