I've had 2 diversions and they both had this feature. Only one solution and that is via bearings, either regrease and re torque the steering (ball)bearings or get "roller bearings" fitted. The wobble will be particularly noticable with loaded panniers/top box until the bearings are done
Best