Can it be possible to build a house that has all its four walls facing towards the South?

Yes, it is possible.
If the house is built at the North Pole, all the walls will face towards South.