Can you go to jail for stealing someone's purse?

I think you would get arrested and have to pay a fine. But that is only when you get caught. Why would you steal a purse anyways?