The best souvenir for stamp collectors are those yearbooks, it comes with a high quality album, and an envelope of the whole year's stamps. The album has all the introduction of the stamps, with space left for stamps. It's cool and elegant. But it's a bit expensive, around $50 plus 7% tax.
If you just want to buy some stamps for collecting, check the Canada Post's magazine at any postal outlet (usually in Shoppers Drug Mart) and order from there. Or you can go to their main post office on Sparks street, they usually carry the most collections.