There could be exceptions to this, though, and they could come into being depending on what your visitors do when they are visiting you. If they are creating damage to his property, he certainly has a right to refuse them access. He may also want to press charges for criminal damage, or expect you to pay for any necessary repairs.
If your visitors stay the night at your house or flat, you will probably find that that is fine, but if it becomes so that your visitors are never away from the place, your landlord could take the view that you are subletting. Depending upon the contract that you have with him, it is likely that this breaks the terms of your lease. You could find, if this were the case, that your landlord has a very good reason could be evicted.
There are many rules that both you and your landlord have to stick to, and they should be set out in your lease. For example, your landlord is allowed to come to the house to make an inspection or to check up on something but he must give you 24 hours notice that he is going to turn up.
It will certainly state in your lease that you are responsible for taking care of the property, but the landlord is unable to dictate to you that you keep it tidy; that is entirely up to you and different people have different standards.