I once had a similar problem, which sounded a bit like excessive tappet clearances but different. I checked and re-checked them, I was running dural pushrods so at first I suspected something there, but in the end it turned out to be as follows. When I got the bike the camshaft (357) was worn, so I had a local cam-grinding company (Wades, Melbourne, remember them?) to regrind my camshaft, -they had a 357 grind pattern so all seemed ok. But what I hadn't realised was that, although they reground the cams correctly (I checked using a timing wheel), the grinding process had removed a touch too much height off the camshaft, resulting in the tappets not following the cam profiles at the lowest point of the cams, and the tappets were clattering against their 'stop' screws on the base circle positions. More to the point, I think I may also have had Wades regrind the tappets too (wish I could remember exactly what it was, sorry I can't be more precise, but I know it was one or both of these things).
All fixed when Bob Joyner sent me a shiny new 357 camshaft!