My 2 cents worth.....my theory is that the cam and followers are meant to "wear together" - it seems to me that the tolerances and design make it unlikely that there will be a full (line) contact between cam and follower until they "wear together" - so one will almost always find evidence of wear on these parts but it is nothing to worry about provided it's not excessive.
So I reckon that as long it looks like the wear has not gone through the hardened surface of each part I'd put the worn looking parts back in and certainly not regrind followers that had "worn in" to suit the cam. This assumes it's known which follower goes with each lobe etc.
On my bike however I had to replace the cam as I wanted a 356 not the 334 it came with (eventually SRM were able to supply a new 356, after being out of stock for many months) so I felt I had to get the followers ground as the accepted wisdom is to always "replace/refurbish" both the cam and followers together and the followers had (obvious) wear marks where the old cam had been contacting them - luckily a local guy was set up to do the little job on the followers, and it only cost me a few dollars. he reckoned there was still plenty of depth left in the hardened surface of the followers.
After 600 miles I had to strip the engine
and there was no evidence of wear on these parts, absolutely none, so if my theory is correct it must take quite a few miles for the parts to wear together! Either that or I was lucky and I'm getting a good "line" contact between the new cam and reground/polished followers.