Can i fight against the father of my child getting contact?

I have recently moved over 300 miles away to get away from my abusive ex partner who unfortunately happens to be the father of my child. We have been to many refuges and he has not been able to find my whereabouts for over 7months now. I and my daughter are finally happy and learning to move on from the trauma. (shes two by the way) for the three years on and off that i was with him, he never made any effort to play, take out, arrange times to see my daughter and whenever he came round, it would always be so late that my daughter would be in bed. So he has never really been that interested in her. & months after no contact at all, i get served court papers on my doorstep from his solicitors telling me that he has applied for contact with my daughter. I have decided tat its in my daughters best interest that she has no contact as he is violent, a drug user and a complete let down. He will mess her up emotionally. I cant guarantee this but i don't think its worth taking the risk as he has shown no emotion towards her in the time she was born. He was even abusive to me while i was pregnant With my baby! I love her so much and don't want her to suffer emotionally or physically because of him. There is nothing that he could contribute that would benefit her. I just want to know if anyone else has tried to fight the courts to stop the father getting Contact and what the outcome was. Please don't reply if you are just going to tell me its my fault for getting pregnant etc...... because that is not useful. I just want to hear from anyone who has experienced similar problems or from any legal persons who might advice me. My solicitor is not being very positive!