It is true that out of province purchases don't get taxed but by law you are supposed to report the purchase to your own provincial government and offer to pay them the tax. Nobody in the history of Canada has ever done this...
The GST is even more bizarre. If you buy something in the US you are charged GST in Canada even though it wasn't purchased here. However if an American buys something in Canada and takes it home they get the GST refunded. So despite the name, the government isn't really taking Canadian goods or services, its just taxing Canadians for spending money.