A man in a resturant thought I was talking about his son and threatend me, is that legal?

I was in a resturant talking to my frind about children in general being bad. His son was misbehaving and he thought I was talking about his son. He threatened me saying he was going to hit me and call the manager to have me move to the other side of the resturant. Is that legal? Should I have called the police?