Single sided rear brake (drum and sprocket in one) are used on the early S/A models, brake pedal on left, brake rod linked directly to the brake drum. Swinging arm has open ended forks. Solid swinging arm spindle.
Ariel type are full width alloy hubs used from 1956 until 1958. Brake pedal on left, crossover shaft within the hollow swinging arm spindle, cable to rear drum brake lever on right hand side. From 1958, the alloy hubs are replaced by cast iron full width hubs, again with cross over shaft and cable to the rear drum. These later swinging arms have closed fork ends.
So, three basic variants of swinging arm. Some have mounts for a full chaincase, and lugs move to accommodate torque arms. Best to start with the hub assembly of choice and match the swinging arm. The wheel bearings are different on Early Single Sided, Ariel and Cast Iron hubs, as are the wheel spindles. So as you have found, variation in the size of the hole in the fork end.
Swinging arm has the same basic dimensions for all model years, so plenty of scope for mix'n match.
Model years and year of manufacture are not the same, and parts from earlier model years were fitted to later models to use up existing earlier type stock. Add to that the frame date is close to a major change in wheel type plus 60 odd years of other folks' mods and you can't assume any part is correct.
I measured a a late S/A, ( full width iron hub) and found slot width to be 17mm on chain (stub axle) side, 20mm on right hand side. Difference is to match the flat locating the stub axle. A further consideration is that while in theory swapping swing arms is a possible, doing it for real can involve major hassle. Plenty on this already on the Forum. Enlarging the slot is certainly possible, but because it affects the horizontal location of the spindle is not an angle grinder, hacksaw and file job. My view is that it is better to avoid modifying genuine OEM parts.
Swarfy.