I don't like the combination of end float and an [apparently] noisy camshaft.
I would take off the primary chain cover and make sure the cush drive assembly is up tight. A loose one rattles on the splines and does sound like a dodgy camshaft [which it could be]. Mine did the same, it was tight, but it had been loose at some stage and rattled on the worn splines.
Also, 3 thou isn't an excessive end float, if I remember rightly, 3 thou is the maximum permissible in Roy's book during a rebuild so that in itself shouldn't be a problem..