So I guess there are two points to think about:
1) What does your "local" government say? The reason being that tenancy deposit law many vary from state to state and country to country.
2) What does your contract actually say? In most cases, the funds in a pet deposit can only be used to cover very specific costs, and this should be outlined in your contract.
I ran a quick Google search and come across this site covering tenancy law in British Columbia.
The main line you'll be interested in is:
If a landlord is entitled to keep all or part of a pet damage deposit, the deposit may be used only for damage caused by the pet unless the tenant agrees otherwise.
Whilst this obviously only applies within the jurisdiction of British Columbia, I would be surprised if similar laws didn't apply in most jurisdictions.Finally, things like security and pet damage deposits should be arbitrated by an independent third party. It shouldn't be up to your landlord whether he returns your deposit or not, he'll have to make a claim to the tenancy deposit scheme and they will decide whether his claim is justified or not.