The accused and the taxi driver are residents of Kabirnagar area in suburban Andheri.
The man, whose wife and children had gone to their native place, brought his girlfriend home at 11.45 on Friday night. They were spotted by the accused, who were roaming in the area, an officer at Vile Parle police station said.
At about 2am, the four barged into the house and threatened the victim and the taxi driver with a knife, police said.
Following preliminary probe, the four were arrested and booked for rape, molestation and assault. They were produced before a local court, which remanded them in police custody till March 20.