Just playing devil's advocate, but lets say you submitted a paper to a reputable journal and it had been accepted and published online, and the author refused to pay the agreed publication fee†. It would not be unreasonable for them to "threaten" to withdraw the paper and leave a note to explain why. What else should they do, given that they had published the work for you, as agreed?
Predatory publishers exists because they meet a need. Some academics need to get papers in print, whether competently peer reviewed or not, perhaps for career progression reasons. How is the publisher to know whether or not you agreed for them to publish your paper knowing what the journals standards are like.
At the end of the day, there is an element of "caveat emptor" (buyer beware). You seem to agree that you made a mistake in sending the paper to a predatory journal, so you should expect there to be some consequences to your mistake. I would send them a letter requesting them to withdraw your paper (or agreeing to the withdrawal). Call their bluff. If someone asked me why the paper was withdrawn, I would just say "yes, I discovered it was a predatory journal after I had submitted the paper". If they give a false reason for the withdrawal, then you should have the reviewer comments and email from the editor showing that it was not because of a problem with the content.
† It amazes me that commercial academic publishers still exist, give that reputable on-line journals, such as jmlr.org have demonstrated it is feasible to have an open access journal that is free to both reader and author, as most of the work is done pro-bono by academic anyway. It would be a much better use of taxpayers money for the funding bodies to subsidise similar academic-run on-line journals (the quality of the journal will depend on the quality of the editorial board).