The court heard that Lee Yuk-ming, 35, met the woman who has an intellectual disability at a church several years ago.
In July 2019, the pair and a friend went to a restaurant in Mong Kok to have lunch. Lee dragged the victim to a toilet and raped her.
The woman became pregnant and had an abortion.
In sentencing, Deputy High Court judge Johnny Chan said he had taken into account a number of aggravating factors in the case, including that Lee had not worn a condom and the fact that the woman’s disability made her vulnerable to abuse.
Chan said the woman had been deeply hurt by the incident, with medical reports showing she had suffered from post-traumatic stress disorder.