Theoretically you have to pay. If you don't, you have a tiny chance to be traced out. Also, if you buy something (like furniture from Hull) and asked them to delive it to Ottawa, you don't need pay provincial tax. Again you have the responsiblity to pay tax by yourself. If you don't, you take risk although it's very small.