How do I remove a database from a deleted site?

Hmm, so I’ve installed a site but have not been able to bring it up. Then uninstalled it.
But when I reinstall it I get an error in the ee.log stating:

The database for [domain.com] already exist
Cleaning up [domain.com]

Then it simply cleans up on site create.

How do I delete a database like this?