As a person who occasionally works with mercury, it is the vapour that gets you - though ingesting it isn't good either.
I believe the main risk with touching the stuff is that if you are not 100% certain you removed it all, it can slowly get absorbed, however touching it has a lower risk (but there is still a health and safety issue) than breathing the vapour. Stick of bowl of mercury out in a warm room and stick a UV light beside it and observe the shadow cast by the vapour and it is obvious that it gives off a lot of vapour.
Do they still use it in gold mining anywhere in the world? I know got banned in most countries, but sometimes you hear of dodgy mining practices in developing countries.
Is it safe to come out of the bunker yet?