Inside the vault you can find the Founders Crown.
A way to glitch into the Vault is to equip a Grappling Hook (such as the Crossbow, Plunger, or Ice Pick) and spam-click the Vault. There's also another way to glitch inside, which is by using a stroller and let the baby inside the vault. Doing this will get you into a second chamber. This one has a large golden sphere, also with a lock. You cannot interact with this one. Nobody has glitched into the sphere yet, but you can stand on top of the Vault by spam-clicking a Grappling Hook.
There are also many Toys around the vault for sale.
In the '???' update, there was a new way into the vault. If you go to Burt, located between the red bridge and the hospital, you'll find an underground place underneath his fishing hook. Inside, you will be asked to look for the three parts of a key. This key, in turn, leads to the Vault in the Pet Store. One piece should be straight in the cove ahead. The next one should be in the stairway area, while the last one is on the top of the tallest tree. When you open the vault with this newly-assembled key, there will be a small crown inside, standing on a pedestal, this can be used as an accessory for your pet. Now, however, you can't use the key to open the vault anymore. After the Dress-Your-Pet update came out, the founder's key turned into a throwing toy.