Introduction Conversion of laparoscopic cholecystectomy to open is uncommon, but is associated with longer hospital stays and recovery. Prognosticating conversion may aid service planning and provision. We therefore aimed to assess the externally validity of the largest risk score for operative conversion. Methods CHOLENZ was a multicentre, prospective, national cohort study of cholecystectomy for benign biliary disease conducted by STRATA, a trainee-led collaborative network. Data were collected from patients undergoing cholecystectomy in New Zealand hospitals between 1 August and 30 October 2021 with 30-day follow-up. The Conversion from Laparoscopic to Open Cholecystectomy (CLOC) score from the CholeS study was assessed for external validity by interrogating its accuracy and calibration in the CHOLENZ dataset. Results Of 1162 cholecystectomies started laparoscopically, 20 (1.7%) were converted to open in the CHOLENZ dataset. The CLOC score predicted 2.9% (IQR 1.3% - 8.1%) would be converted. Area under the curve was 0.65 (95% 0.51 - 0.79) and calibration was acceptable with a Hosmer-Lemeshow p value of 0.45; with evidence of tendency to overestimate with interrogation of calibration across a continuous risk profile (intercept 1.27, slope 0.4). Sensitivity analysis with imputed data improved accuracy. Recalibration with the addition of body mass index, and preoperative bilirubin also improved accuracy to 0.86 (95% CI 0.78 - 0.95). Conclusions The CLOC score performance was considered moderate in the Aotearoa New Zealand setting to predict conversion of laparoscopic cholecystectomy to open and in its current form lacks the accuracy to be used in our setting.