Rigor Reviewed

Rigor-Reviewed Manuscript

An LLM chatbot to facilitate primary-to-specialist care transitions: a randomized controlled trial.

Name: Adcurare Rigor Record — An LLM chatbot to facilitate primary-to-specialist care transitions: a randomized controlled trial.
Item: An LLM chatbot to facilitate primary-to-specialist care transitions: a randomized controlled trial.
Rating: 96
Author: Adcurare

Tao X, Zhou S, Ding K, Li S, Li Y, Wu B, Huang Q, Chen W, Shen M, Meng E, Chen X, Hu H, Zhang J, Zhou J, Zou L, Ma L, Han S

Paper source

Read full text (PMC)Journal

Embed this Rigor badge

Adcurare Rigor Review — 3.25★ (Adequate), 0 critical, 3 high issues for “An LLM chatbot to facilitate primary-to-specialist care transitions: a randomized controlled trial.”

Links back to this Rigor Record.

Adcurare Editorial

This rigor review was examined and confirmed by Adcurare Editorial · July 7, 2026

Adcurare Rigor Review

Rigor Reviewed

3.25 / 5

Adequate — real issues to address

Interventional Study

Critical

High-priority

How this rating was calculated▸

Started at 5★ — deductions

IntegrityIntegrity concern ×3−1.5★
ReportingData & code availability partially met−0.25★

Model-reported confidence: highReviewer agreement: moderateEngine: 7.6.0Source: full text

Not verified in this report

No reported statistical tests were found to recompute.

Problems surfaced4 types

Data/code availability incomplete
Methods and results do not match
Implausibly large reported effect
Internal contradictions in the reported numbers

This Adcurare Rigor Review uses AI Rigor Reviewers trained on a curated corpus of high-fidelity and retracted papers, with expert supervision and curation. It can still make mistakes; verify each finding against the source before relying on it.

The manuscript describes a methodologically rigorous pragmatic RCT of an LLM chatbot for care transitions, with strong reporting on most rigor dimensions. The main weaknesses are in data/code availability (vague data access statement, proprietary chatbot code) and minor statistical reporting gaps (normality test method not named, some p-values as inequalities).

This evaluation synthesizes three independent reviewer assessments and a copyedit pass. Of the eight rigor dimensions, three are fully applicable (scientific premise, study design, ethical approvals, biological variables, key resources, statistical analysis, data code availability, reporting transparency). The statistics verification component could not independently recompute any reported test statistic due to the format of reported results; the statistical analysis dimension's status reflects reporting adequacy, not mathematical verification against raw data.

12 major claims checked against the paper's own evidence: all adequately supported.

3 integrity concerns flagged (0 high).

Reporting

Eight Dimensions7 pass · 1 warn

The paper thoroughly cites prior research demonstrating healthcare system fragmentation, the potential of LLMs, and the lack of real-world evidence. It clearly articulates the rationale for co-design over passive data collection. Limitations of prior work (e.g., lack of patient-facing LLM evidence, biases in local data) are explicitly addressed and motivate the study design.

Sub-criteria

prior work citedADEQUATE

premise rationaleADEQUATE

limitations addressedADEQUATE

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 95% conf

The scientific premise is well-established through extensive citation of prior work on healthcare system inefficiencies, LLM potential, and co-design methodology.

Evidence

direct quote[Introduction, paragraphs 1-3]

“Health systems face increasing strain from fragmented infrastructure, under-resourced primary care and inefficient triage mechanisms”

direct quote[Introduction, paragraph 4]

“This omission begs a central question: should LLMs reflect local practices or help reform them?”

direct quote[Introduction, paragraphs 2-4]

“Critically, evidence is lacking for LLM chatbots that directly interact with socioeconomically diverse patient populations while supporting both curative and caring aspects of medicine in high-volume clinical environments”

PASSReviewer 2· 95% conf

The paper clearly establishes its scientific foundation, citing prior work on healthcare system inefficiencies and LLM applications, and provides a logical rationale for the study.

Evidence

direct quote[Introduction, paragraphs 1-3]

“The growing burden of multimorbidity and aging populations has exposed vulnerabilities in healthcare delivery worldwide.”

direct quote[Introduction, paragraph 3]

direct quote[Introduction, paragraph 4]

“This omission begs a central question: should LLMs reflect local practices or help reform them?”

PASSReviewer 3· 92% conf

The scientific premise is well-established, citing prior work on healthcare system inefficiencies, LLM applications, and the gap in evidence for co-designed patient-facing chatbots.

Evidence

direct quote[Introduction, par. 1-3]

“Large language models (LLMs) possess transformative potential to re-engineer hospital workflows and address the systemic inefficiencies amplified by escalating demand. However, current applications remain largely confined to support healthcare professionals in controlled settings… Critically, evidence is lacking for LLM chatbots that directly interact with socioeconomically diverse patient populations.”

direct quote[Introduction, par. 5]

“This omission begs a central question: should LLMs reflect local practices or help reform them? The answer is critical for global health equity, as passively collected local dialogues may codify and even scale systemic inequities.”

paraphrase[Introduction, final paragraph; Results, 'Comparative performance of development strategies']

“The decision to deploy the co-designed PreA was empirically grounded by a previous simulated experiment comparing co-design with fine-tuning on local dialogues.”

The trial used individual-level computer-generated randomization without stratification, with allocation concealment. It is single-blinded (physicians masked to PreA-intervention groups). A power analysis with a target sample size of 2,010 participants was prespecified. Inclusion/exclusion criteria are clearly defined. Blinding is adequate for a pragmatic trial. For a human RCT, replicate_distinction is not applicable, and controls (standard care) and independent replication (not applicable for a pivotal trial) are appropriately handled. Outlier handling is implicitly covered by the use of appropriate statistical tests and pre-specified analysis.

Sub-criteria

controlsADEQUATE

power analysisADEQUATE

blinding levelsADEQUATE

outlier handlingINADEQUATE

randomization unitADEQUATE

inclusion exclusionADEQUATE

randomization methodADEQUATE

replicate distinctionN/A

independent replicationN/A

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 95% conf

The study design is rigorous and well-reported for a pragmatic RCT, meeting key criteria for randomization, allocation concealment, sample size justification, and inclusion/exclusion criteria.

Evidence

direct quote[Methods, Randomization and blinding]

“Participants were allocated to one of the three groups using an individual-level, computer-generated randomization sequence without stratification.”

direct quote[Methods, Randomization and blinding]

direct quote[Methods, Sample size]

“The target minimum sample size of 2,010 participants (670 per study arm) was prespecified based on a power analysis using preliminary data from the pilot study of 90 patients.”

PASSReviewer 2· 90% conf

The study design is rigorous for a pragmatic RCT, with adequate randomization, blinding, power analysis, and inclusion/exclusion criteria. Several sub-criteria are not applicable for a human RCT.

Evidence

direct quote[Methods, Randomization and blinding]

“Participants were allocated to one of the three groups using an individual-level, computer-generated randomization sequence without stratification. Allocation was concealed to prevent selection bias.”

direct quote[Methods, Sample size]

“The target minimum sample size of 2,010 participants (670 per study arm) was prespecified based on a power analysis using preliminary data from the pilot study of 90 patients.”

direct quote[Methods, Participants]

“Inclusion criteria were (1) aged between 20 years and 80 years; (2) visit the participating physicians at the study medical centers; (3) eligible for communicative interaction via mobile phone; (4) eligible to complete the post-consultation questionnaires; and (5) have signed informed consent.”

PASSReviewer 3· 85% conf

The study design is a pragmatic, multicenter RCT with adequate randomization, participant-level allocation concealment, single-blinding, and a priori sample size calculation. Applicable sub-criteria for a human interventional trial are met; some bench-science sub-criteria are correctly marked n/a.

Evidence

direct quote[Methods, 'Randomization and blinding']

“This trial was single-blinded: while the patients knew their group assignments (PreA-only, PreA-human, or No-PreA), the physicians were uninformed about the PreA-intervention groups (PreA-only or PreA-human). Furthermore, research staff involved in data analysis remained blinded to group assignments throughout the study.”

direct quote[Methods, 'Sample size']

“The target minimum sample size of 2,010 participants (670 per study arm) was prespecified based on a power analysis using preliminary data from the pilot study of 90 patients. This minimum target sample size ensured sufficient power (>80%) for the primary outcome at a significance level of 0.05.”

For a human RCT, the applicable sub-criteria are sex_reported, age_weight_health, and demographics. The paper reports detailed demographics in Table 1, including sex (55.1% female, 44.9% male), age (mean 47.6 ± 14.6 years), ethnicity, education, income, and work status. As both sexes are enrolled, sex_justified is not applicable. Species_strain_source and housing_conditions are not applicable.

Sub-criteria

demographicsADEQUATE

sex reportedADEQUATE

sex justifiedN/A

age weight healthADEQUATE

housing conditionsN/A

species strain sourceN/A

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 95% conf

Human demographic variables including sex, age, and socioeconomic factors are comprehensively reported and well-balanced across groups, which is the relevant set for a human RCT.

Evidence

direct quote[Results, Patient flow and baseline data; Table 1]

“Participants had a mean age of 47.6 ± 14.6 years and included 1,141 women (55.1%) and 928 men (44.9%).”

direct quote[Table 1; Methods, Participants]

“Age, years, mean ± s.d.: 47.6 ± 14.6”

direct quote[Table 1]

“Table 1 Distribution of baseline covariates across three trial groups”

PASSReviewer 2· 95% conf

Human demographics are reported in detail (age, sex, ethnicity, education, income, work status, etc.), and the study includes both sexes.

Evidence

direct quote[Results, Table 1]

“Participants had a mean age of 47.6 ± 14.6 years and included 1,141 women (55.1%) and 928 men (44.9%).”

direct quote[Results, Table 1]

“Educational attainment among them was distributed as follows: below primary school (313, 15.1%), high school (1,073, 51.9%) and college or higher (683, 33.0%).”

PASSReviewer 3· 95% conf

Relevant human biological variables (sex, age, health status, race/ethnicity) are comprehensive reported and well-balanced across groups. Applicable sub-criteria are all met.

Evidence

direct quote[Results, 'Patient flow and baseline data'; Table 1]

“Participants had a mean age of 47.6 ± 14.6 years and included 1,141 women (55.1%) and 928 men (44.9%). … Educational attainment among them was distributed as follows: below primary school (313, 15.1%), high school (1,073, 51.9%) and college or higher (683, 33.0%). … These baseline covariates were well balanced across the three trial arms, with no significant differences in distribution (Table 1).”

The Methods explicitly name the Chinese Academy of Medical Sciences and Peking Union Medical College, the local medical ethics committee of the First Affiliated Hospital of Guilin Medical University, and the IRBs of the Affiliated Hospital of Gansu Medical College. Informed consent was obtained from all participants. Compliance with the Declaration of Helsinki and ICH-GCP is stated. The trial is prospectively registered (ChiCTR2400094159).

Sub-criteria

iacuc statementN/A

informed consentADEQUATE

irb ethics statementADEQUATE

regulatory complianceADEQUATE

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 95% conf

Ethical approvals from named bodies, informed consent, and regulatory compliance are explicitly stated and adequate.

Evidence

direct quote[Methods, Ethics approval]

“The Chinese Academy of Medical Sciences and Peking Union Medical College and the local medical ethics committee of the First Affiliated Hospital of Guilin Medical University approved the study.”

direct quote[Methods, Ethics approval]

“We obtained informed consent from all participants in this study.”

direct quote[Methods, Ethics approval]

“The trial followed the Declaration of Helsinki and the International Conference of Harmonization Guidelines for Good Clinical Practice.”

PASSReviewer 2· 95% conf

Ethical approvals from named committees, informed consent, and regulatory compliance are clearly reported.

Evidence

direct quote[Methods, Ethics approval]

“The Chinese Academy of Medical Sciences and Peking Union Medical College and the local medical ethics committee of the First Affiliated Hospital of Guilin Medical University approved the study.”

direct quote[Methods, Ethics approval]

“We obtained informed consent from all participants in this study.”

direct quote[Methods, Ethics approval]

“The trial followed the Declaration of Helsinki and the International Conference of Harmonization Guidelines for Good Clinical Practice.”

PASSReviewer 3· 96% conf

Ethical approval is clearly documented: named ethics committees, informed consent, and regulatory compliance with Declaration of Helsinki and GCP are stated. The trial is registered.

Evidence

direct quote[Methods, 'Ethics approval']

“The Chinese Academy of Medical Sciences and Peking Union Medical College and the local medical ethics committee of the First Affiliated Hospital of Guilin Medical University approved the study. The institutional review boards of the Affiliated Hospital of Gansu Medical College approved the study protocol.”

direct quote[Methods, 'Ethics approval']

“We obtained informed consent from all participants in this study.”

direct quote[Methods, 'Ethics approval']

“The trial followed the Declaration of Helsinki and the International Conference of Harmonization Guidelines for Good Clinical Practice.”

For this clinical trial of a digital intervention, antibodies, cell lines, mycoplasma, organisms, and wet-lab reagents are not applicable. The investigational product (PreA) is identified: an LLM chatbot (OpenAI; GPT-4.0 mini). Software tools for analysis are identified (Python v.3.7 and R v.4.3.0). The code repository is provided.

Sub-criteria

mycoplasma testingN/A

reagents identifiedADEQUATE

organisms identifiedN/A

antibodies identifiedN/A

cell line authenticationN/A

software tools identifiedADEQUATE

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 95% conf

The investigational product (PreA chatbot) is identified, software tools are named with versions, and the trial protocol is described. Other resource sub-criteria are not applicable.

Evidence

direct quote[Abstract; Methods, Model development]

“we developed PreA (Pre-Assessment), an LLM chatbot (OpenAI; GPT-4.0 mini) for primary-to-specialist care transitions”

direct quote[Methods, Statistical analysis]

“Python v.3.7 and R v.4.3.0 were used to perform the statistical analyses and present the results.”

WARNReviewer 2· 85% conf

As a clinical trial of an LLM chatbot (software as an intervention), the software is the key resource and is reported transparently (OpenAI GPT-4.0 mini), but the version and model specifics are not fully detailed, and the code is not publicly available.

Evidence

direct quote[Introduction, paragraph 5]

“we developed PreA (Pre-Assessment), an LLM chatbot (OpenAI; GPT-4.0 mini) for primary-to-specialist care transitions”

direct quote[Methods, Statistical analysis]

“Python v.3.7 and R v.4.3.0 were used to perform the statistical analyses and present the results.”

direct quote[Data and Code Availability]

“The PreA chatbot is not publicly available as it is the subject of ongoing commercial licensing discussions and is protected intellectual property held by the Chinese Academy of Medical Sciences & Peking Union Medical College”

PASSReviewer 3· 82% conf

For a clinical trial of an LLM chatbot, the key resources are the chatbot itself (software tool) and the baseline LLM model (GPT-4.0 mini). The software tool is identified but not publicly available; the base model is named. The investigational product (chatbot) is described but not publicly accessible. In a human trial of a digital intervention, reagent-level criteria are n/a.

Evidence

direct quote[Methods, 'Co-designed architecture and clinical integration'; 'Code availability']

“PreA (Pre-Assessment), an LLM chatbot (OpenAI; GPT-4.0 mini) for primary-to-specialist care transitions.”

direct quote[Methods, 'Analysis of healthcare delivery']

“Python v.3.7 and R v.4.3.0 were used to perform the statistical analyses and present the results.”

direct quote[Methods, 'Code availability']

“Code for classification analysis and data visualization can be found at https://github.com/ShashaHan-collab/PreA-OutpatientRCT (ref. ).”

Tests are named: two-tailed t-tests, Mann-Whitney U, chi-squared, ANOVA, Wilcoxon signed-rank, etc. For a large human trial, assumptions are adequately handled by design (t-tests with unequal variances, nonparametric tests for skewed data). Exact p-values and 95% CIs are reported for primary and secondary outcomes. Effect sizes are given as relative reductions/increases with CIs. Data presentation includes box plots with individual points, and error bars are defined (SD, with 1.5×IQR). Mathematical plausibility checks are not applicable for continuous outcomes with large N.

Sub-criteria

tests namedADEQUATE

exact p valuesADEQUATE

effect sizes ciADEQUATE

data presentationADEQUATE

software identifiedADEQUATE

assumptions verifiedADEQUATE

mathematical plausibilityN/A

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 90% conf

Statistical tests are named, assumptions are handled by standard methods, exact p-values and confidence intervals are reported, and data presentation includes individual points and defined error bars.

Evidence

direct quote[Methods, Statistical analysis]

“two-sample Student's t-tests with unequal variances were used for approximately normal data, while a nonparametric Mann–Whitney U-test was applied to skewed distributions”

direct quote[Results, Outpatient workflow]

“PreA-only 3.14 ± 2.25 versus No-PreA 4.41 ± 2.77 min; P < 0.001; Fig. ), corresponding to a 28.7% (95% CI 22.7–34.8) relative reduction.”

direct quote[Figure 2 legend]

“The center of the box plot represents the median, with the boundaries representing the first and third quartiles. The whiskers represent the furthest data points from the edge of the box within 1.5 × IQR.”

PASSReviewer 2· 90% conf

Statistical tests are named, exact p-values are reported with effect sizes and confidence intervals, software is identified, and data presentation is adequate. Some assumptions are handled by design in this large pragmatic trial.

Evidence

direct quote[Results, Outpatient workflow]

“The PreA-only group had a significantly shorter consultation duration compared to the No-PreA group (PreA-only 3.14 ± 2.25 versus No-PreA 4.41 ± 2.77 min; P < 0.001; Fig. 2a), corresponding to a 28.7% (95% CI 22.7–34.8) relative reduction.”

direct quote[Methods, Statistical analysis]

“We assessed the normality of value distributions and used two-sample t-tests with unequal variances for intergroup comparisons. For significantly skewed dimensions, we employed non-parametric Mann-Whitney U-tests.”

PASSReviewer 3· 81% conf

The statistical analysis is generally robust: tests are named, exact p-values are reported (mostly), some effect sizes with CIs are given, and data presentation (box plots with individual points, error bars defined, per-group n stated) is adequate. Scalability to a large trial precludes exhaustive assumption verification at the bench level, but the methods used (t-tests with unequal variance, nonparametric tests where needed) demonstrate appropriate handling. Mathematical plausibility is not_applicable for these continuous outcomes with large Ns.

Evidence

direct quote[Results, 'Outpatient workflow' and 'Patient-centeredness and care coordination']

“PreA-only group had a significantly shorter consultation duration compared to the No-PreA group (PreA-only 3.14 ± 2.25 versus No-PreA 4.41 ± 2.77 min; P < 0.001; Fig. ), corresponding to a 28.7% (95% CI 22.7–34.8) relative reduction.”

direct quote[Methods, 'Analysis of healthcare delivery']

direct quote[Figure 2, Figure 3, Figure 4 captions; Methods]

“Box plots show the distribution consultation duration across the PreA-only ( n = 691 participants), PreA-human ( n = 689 participants), and No-PreA ( n = 689 participants) groups. The center of the box plot represents the median, with the boundaries representing the first and third quartiles. The whiskers represent the furthest data points from the edge of the box within 1.5 × IQR.”

Data availability: the paper states that an anonymous data repository link is provided (source data via the code repository), but raw conversation data are not publicly available for privacy reasons. Anonymized, nondialogue individual-level data can be requested via email to S.H. with a proposal, but no managed-access platform is named. This is reported_but_inadequate because the mechanism lacks a concrete platform; it is a bare 'email the author' request. Code availability: the code for comparative statistical analyses and data visualization is available at a GitHub repository, but the PreA chatbot itself is not publicly available (reported_but_inadequate). Overall, 1/2 applicable criteria (code sharing) is partially adequate but incomplete.

Sub-criteria

code sharingINADEQUATE

accession numbersN/A

repository depositINADEQUATE

data availability statementINADEQUATE

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 95% conf

A data availability statement with a managed access procedure is provided, and analyzable code is shared in a public repository. Raw dialogue data is not shared due to privacy, which is appropriate.

Evidence

direct quote[Data availability]

“Anonymized, nondialogue individual-level data underlying the results can be requested by qualified researchers for academic use. Requests should include a research proposal, statistical analysis plan and justification for data use, and can be submitted via email to S.H.”

direct quote[Code availability]

“Code for classification analysis and data visualization can be found at https://github.com/ShashaHan-collab/PreA-OutpatientRCT”

direct quote[Data availability]

“Review of the proposals may take up to 2 months, and approved requests will be granted access via a secure platform after execution of a data access agreement.”

WARNReviewer 2· 85% conf

A data availability statement is present, but it is vague for individual-level data ('available on request' with a non-platform-specific email), and the custom code is not publicly available.

Evidence

direct quote[Data and Code Availability]

“The PreA chatbot is not publicly available as it is the subject of ongoing commercial licensing discussions”

WARNReviewer 3· 85% conf

A data availability statement is present but vague: 'available on reasonable request' without a concrete managed-access mechanism or timeline for non-dialogue data. Raw dialogue data are not available. Code for analysis is publicly available on GitHub.

Evidence

direct quote[Data availability]

direct quote[Code availability]

“Code for classification analysis and data visualization can be found at https://github.com/ShashaHan-collab/PreA-OutpatientRCT (ref. ).”

Methods are extensive, describing the co-design, model development, RCT procedures, and statistical analyses. Trial registration number is provided. A CONSORT flow diagram is included. All pre-specified outcomes (primary, secondary, exploratory) are reported, including null or non-significant results (e.g., no significant difference in clinical note features, subgroup analyses). Limitations are explicitly discussed in the Discussion section. Conclusions are proportionate to the evidence, noting the context-dependence and need for validation. Funding and competing interests are disclosed.

Sub-criteria

funding coiADEQUATE

trial registrationADEQUATE

reporting guidelineADEQUATE

methods completenessADEQUATE

all outcomes reportedADEQUATE

limitations discussedADEQUATE

conclusions proportionalADEQUATE

Per reviewer (3) — the votes behind the verdict

PASSReviewer 1· 95% conf

Methods are detailed and replicable, the trial is registered, outcomes are comprehensively reported including null results, limitations are discussed, conclusions are proportional, and funding/COI are disclosed.

Evidence

direct quote[Abstract; Methods, Ethics approval]

“Chinese Clinical Trial Registry identifier: ChiCTR2400094159”

direct quote[Discussion, paragraph 7]

“Several limitations warrant consideration when interpreting our findings. The generalizability of our time-reduction findings may be context-dependent”

direct quote[Results, Physician clinical decision-making]

“Classification analysis yielded near-random discriminability (F1 score 0.57; P = 0.81; ΔF1 < 0.02).”

PASSReviewer 2· 90% conf

The manuscript is highly transparent: methods are detailed, trial registration is reported with identifier, a CONSORT flow diagram is present, outcomes are comprehensively reported, limitations are discussed, and funding/COI are stated.

Evidence

direct quote[Methods, Ethics approval]

“This trial is registered at the Chinese Clinical Trial Registry (identifier ChiCTR2400094159 (https://www.chictr.org.cn/showprojEN.html?proj=251179) ).”

direct quote[Results, Fig. 1]

“Fig. 1 CONSORT flow diagram for the randomized controlled trial.”

direct quote[Discussion, last paragraph]

PASSReviewer 3· 88% conf

Reporting is comprehensive: methods are detailed, the trial is registered, subgroup and negative results are presented, limitations are discussed, and funding/COI is stated. The conclusion is proportional to the evidence.

Evidence

direct quote[Abstract; Methods, 'Ethics approval']

“Chinese Clinical Trial Registry identifier: ChiCTR2400094159 (https://www.chictr.org.cn/showprojEN.html?proj=251179) .”

direct quote[Discussion, final paragraph]

“Several limitations warrant consideration when interpreting our findings. The generalizability of our time-reduction findings may be context-dependent, as our study was conducted in high-volume, resource-limited hospital settings. … The single-blinded, pragmatic trial design, while reflecting real-world conditions where patients would naturally know their preconsultation experience, introduces potential performance bias.”

absence[Methods]

No explicit mention of a completed reporting checklist submission in the paper text.

5 copyedit issues flagged: mostly consistency, punctuation, typo.

Checked 57 references: 50 verified — 7 not checked.

Reproducibility

2 data/code links checked; 2 live.

Registered (1 ID: Chinese Clinical Trial Registry). Reporting guideline cited: CONSORT.

Adcurare Rigor Action ItemsFinal pass before journal submission

Ready after minor pre-submission edits. The manuscript is in strong shape for submission, but should address the data availability statement (use a managed-access platform), specify the exact GPT model version, name the normality test used, and fix the F1 formula typo and minor formatting issues before submission.

1.
HIGHdata code
Replace the email-based data access statement with a link to a managed-access platform (e.g., YODA, Vivli, or a study-specific data use committee) that provides a structured application process and approval timeline.
Journal reviewers and editors increasingly expect data access to go through a formal platform rather than a simple email request, and the current statement is vague ('available on request').
2.
HIGHstatistics
In Methods, Statistical analysis, name the normality test used (e.g., Shapiro-Wilk or Kolmogorov-Smirnov) and state the threshold (e.g., P > 0.05) used to decide between parametric and non-parametric tests.
Two reviewers flagged this as a reporting gap; naming the test strengthens reproducibility and statistical rigor.
3.
HIGHreporting
In Methods, Model development, specify the exact OpenAI model version identifier (e.g., gpt-4o-mini-2024-07-18) rather than only the generic 'GPT-4.0 mini'.
The investigational product is the key resource; knowing the precise model version enables reproducibility and version-specific safety/performance assessment.
4.
MEDIUMstatistics
Replace all inequality p-values (e.g., P < 0.001) with exact values (e.g., P = 0.0003) where possible, or at least for the three primary comparisons.
Exact p-values are preferred in clinical trial reporting for transparency and meta-analysis.
5.
MEDIUMreporting
In Methods, add a sentence confirming that a completed CONSORT checklist was submitted as supplementary material (if true) or add the checklist fields to the Reporting Summary.
One reviewer noted the lack of explicit mention of a CONSORT checklist submission in the paper text, though a Nature Portfolio Reporting Summary is linked.
6.
MEDIUMreporting
In Methods, add a brief subsection on outlier handling and sensitivity analyses (e.g., winsorization, per-protocol vs ITT analysis, or a statement that no data points were excluded).
All three reviewers noted outlier_handling as inadequate or not reported; this is a standard section in clinical trial methods.
7.
MEDIUMreporting
In the Discussion, acknowledge the potential performance bias introduced by the single-blind design (patients unmasked) and discuss its possible impact on subjective outcomes like patient experience.
The single-blind design is already described, but the Discussion does not explicitly address the risk of bias from unblinded patients on patient-reported outcomes.
8.
LOWcopyedit
Fix the F1 formula in Methods, Statistical analysis: change 'F1 Score=TP/(2TP + FP + FN)' to 'F1 = 2TP/(2TP + FP + FN)' to match the standard harmonic mean definition.
The copyedit pass identified a typo in the formula that could cause confusion; the standard F1 score is 2*TP/(2TP+FP+FN).
9.
LOWcopyedit
Remove the space before the final period in: 'identifier: ChiCTR2400094159 (https://...).' and remove spaces inside parentheses around the GitHub URL.
Typographical inconsistencies flagged in the copyedit pass; minor but improves professional appearance.
10.
LOWdata code
If possible, deposit the anonymous non-dialogue dataset (e.g., aggregated data underlying Table 1 and main figures) in a repository like Zenodo with a DOI, and update the Data Availability section accordingly.
Although individual-level patient data cannot be shared, aggregate data supporting the figures could be deposited to strengthen reproducibility.
11.
LOWother
In the Discussion, briefly clarify that the total patient time for PreA-only (3.51 min PreA interaction + 3.14 min physician consultation = 6.65 min) is longer than the No-PreA consultation (4.41 min), to prevent misinterpretation of 'consultation duration'.
The integrity verification component noted a potential clarity issue where readers might conflate consultation duration with total patient time; a brief clarification would preempt confusion.

Full text (70,199 chars)Analyzed: Jul 7, 2026

Read alongside