Legally, I think it all comes down to what is stipulated in the contract or lease you signed.
But personally, it does seem a bit unfair to prevent you spending time with your boyfriend based on hearsay.
However, try to understand things from her perspective: If she's convinced your boyfriend is a drug dealer - then she may be perfectly within her rights to not want him on her property.
My advice would be to try and come to an amicable solution. Is there nothing you can do to prove her wrong in a sensible and polite way?
If not, then maybe it's time you start looking for somewhere better (in your own time, don't feel pressured to leave).
You want your home to be somewhere relaxing and comfortable to retreat to, not somewhere you have to worry about who you can and can't invite back. Also, a live-in landlord just sounds like a bad idea.
No matter how nice they are, many of them will never accept that you anything more than a 'guest' in their house - despite paying them money for your own space.