Remove certificates from Active Directory (but not all of them)

I have a customer who has piles of users that have more than one published certificate. There is never a situation where they should have more than one published in Active Directory. I can obtain the list of certificates and even identify the certificates I want to remove, but I cannot figure out how to write the good certificate back to Active Directory, or delete just one certificate. Below is my example. I am restricted to ADSI provider. RSAT is not an option.

I have tried two methods so far. I tried just writting the good certificate to the value after looking through all the certificates (not shown in the functional example)

$User.putex(1,"userCertificate",$CertificateObject)

I have also tried deleting the certificates that should no longer be published (Current attempt below)

Function CheckUserCertificate {
    [cmdletbinding(SupportsShouldProcess=$True)]
    Param(
        [Parameter(Mandatory=$True)][string]$Path
    )
    Begin{
        $ErrorActionPreference="SilentlyContinue"
    } #End Begin
    Process{
        $User = [adsi]$Path
        Try{
            $CertificateBlob = $User.getex("userCertificate")
        }
        Catch{}
        ForEach ($CertificateObject in $CertificateBlob){
            Write-Verbose "Getting Certificate from AD for $($User.name)"
            [Array]$Certs += new-object System.Security.Cryptography.X509Certificates.X509Certificate2(,$CertificateObject) | sort NotBefore

        }
        If ($Certs.count -gt 1){
            Write-Verbose "$($User.Name) has more than one certificate"
            $C = 1
            ForEach ($Cert in $Certs){
                #Exporting the certificate to a file in case something goes wrong.
                $Bytes = $Cert.Export("Cert")
                [system.IO.file]::Writeallbytes(($OutputPath + "$(get-date -Format yyyyMMddHHmmss)-$($User.name)-$C.cer"),$Bytes)
                
                #We sorted by NotBefore. Only the first one should be kept.
                If ($C -gt 1){
                    Write-Verbose "Deleting Certificate $Cert"
                    $ErrorActionPreference="Continue" #Just here for trouble shooting
                    $Error.clear()                    #Just here for trouble shooting
                    $User.putex(4,"userCertificate",$CertificateObject)
                    $User.SetInfo
                    $Error                            #Just here for trouble shooting
                }
                $C++
              
            } #End ForEach Loop through Certificates
        } #End of If Certs Greater than one.
    } #End process
    End{
        #Nothing here yet because I have not decided what to return
        #Work in progress.
    } # End End
} # End CheckUserCertificate Function

Can someone point me to what I am missing?