How to Check Data and Code Availability in Medical Research with AI (2026)
Linda
Learn how to check data and code availability in medical research with AI. Audit real biomedical papers, repository access, and reproducibility gaps
A data availability statement may name a public repository, require a formal access request, or explain why patient-level information cannot be released. A code availability statement may point to software, notebooks, or an author who can consider requests. None of these statements, by itself, establishes that a study can be independently reproduced. This guide shows how to audit both statements using published biomedical examples and a source-grounded Noah AI workflow.
Key Takeaways
- Separate what authors report from what reviewers verify. A repository citation, an accessible repository page, and a successful reproduction are three different levels of evidence.
- Check data and code independently. Public scripts do not necessarily provide access to sensitive clinical datasets.
- Record exact access conditions. Distinguish public links, written requests, institutional approval, and synthetic demonstration data.
- Document the next check. An auditable table is more useful than an unsupported numerical reproducibility score.
- Keep the human reviewer responsible. AI can organize source claims and gaps, but repository inspection, code execution, and final judgments need independent verification.
What Are Data and Code Availability Statements?
A Data Availability Statement tells readers how to locate or request the data underlying a publication. Depending on the study, it may cite accession numbers in a scientific archive, a general-purpose repository, supplemental files, or a restricted-access application process. For biomedical research involving patient records, restrictions can be necessary even when a paper is open access.A Code Availability Statement describes access to software or analysis scripts used to produce reported results. It may provide a GitHub link, a versioned software archive, notebooks, or instructions for contacting the authors. Code access should be assessed alongside documentation, software versions, dependencies, input formats, and the availability of the relevant datasets.The critical distinction is between statement completeness, resource accessibility, and actual reproducibility. A statement can be precise while access remains controlled. Conversely, a repository may be publicly visible but still lack enough information to regenerate a result.
Why Data and Code Availability Matter for Medical Research
Transparent sharing arrangements help researchers scrutinize analyses, reuse methods, and understand which conclusions can be independently checked. In clinical and biomedical work, this must be balanced against consent, patient confidentiality, licensing, and institutional governance.For example, a single-cell RNA sequencing analysis may rely on public GEO accessions and archived scripts. A clinical precision-medicine study may publish code while restricting patient-level records. These arrangements pose different verification tasks; neither should be reduced to a simple label of “good” or “bad” research.Publisher guidance illustrates why detail matters. PLOS data-sharing guidance discusses the distinction between open access and ethical or legal restrictions. These policies provide useful review criteria, but a reviewer must check which requirements actually apply to the journal and paper under examination.
Challenges in Verifying Research Data and Code
- Vague access wording: “available on request” may omit eligibility, approval procedures, timing, and permitted uses.
- Links without provenance: a GitHub page may exist without an identifiable release or connection to the exact published analysis.
- Incomplete dependencies: scripts can require packages, parameters, computing resources, or private inputs that are not fully documented.
- Sensitive source data: de-identification does not automatically make patient-level datasets suitable for public release.
- Synthetic examples: demonstration files may validate a method's implementation but cannot substitute for reproducing empirical findings from the original dataset.
- Unclear verification status: finding a repository is not equivalent to checking every file, downloading all inputs, or executing the analysis.
How Noah AI Supports Data and Code Availability Assessment
In the documented workflow, Noah AI was prompted to examine four biomedical publications, extract their availability arrangements, distinguish reported access from verified access, and compile a Data Availability Assessment Table. The prompt explicitly instructed the agent not to invent access results, software execution, numerical scores, or reproduced findings.This is an AI-assisted evidence-organization and source-review task. It should not be described as automated code execution or guaranteed journal-policy compliance. The reviewer's job is to follow the cited resources, inspect what is actually provided, and preserve any unresolved conditions.

Five-Step Data and Code Verification Workflow
- Locate the original statements. Read the published article's data availability, code availability, methods, and supplementary sections. Copy the source locations and note any relevant updates.
- Trace datasets to their sources. Record repository names, accession numbers, DOIs, and whether original data are openly retrievable or controlled. Distinguish public metadata from accessible underlying files.
- Inspect the code access route. Check the article's Code Availability Statement, Methods section, supplementary materials, linked repositories, and referenced software archives before determining how the analysis code can be accessed.
If no code link is found in the main article, record "Code link not identified in the main text; supplementary materials and repositories require further checking" rather than claiming that the authors did not share their code.Distinguish code that is publicly accessible, available upon request, not independently verified, and confirmed unavailable. A repository link does not establish that the required scripts and dependencies are complete, that the code has been executed, or that the published results can be reproduced.
- Identify the reproducibility boundary. Decide whether the review establishes only a statement, a working link, inspected files, code execution, or replication of published outputs. Do not advance a resource to a higher verification level without evidence.
- Record a concrete next action. For example: inspect every GEO accession, request governed clinical data, run an analysis on synthetic inputs, or ask an author for code and benchmark access details.
Four Biomedical Case Studies: Data Availability Assessment
The following four papers illustrate distinct access models. The summary reflects the published statements and the Noah research table shown in Figure 2. Where independent execution or full dataset inspection has not occurred, that boundary is stated explicitly.
- Cao et al. (2024): PENCIL — Public Data and Publicly Linked Code
The PENCIL reusability report lists twelve Gene Expression Omnibus (GEO) series and links to both a GitHub repository and a Zenodo software record. These are specific, traceable identifiers rather than a generic promise to share materials.However, the Noah table expressly labels record-level GEO checks as NOT VERIFIED and code execution as NOT EXECUTED. The next checks are to verify the listed GEO records and related inputs, identify the appropriate software version, and reproduce a selected figure under recorded conditions. Public links are a starting point, not a completed replication.
- Bertsimas et al. (2024): R.O.A.D. — Restricted Clinical Data and Public Code
The R.O.A.D. to precision medicine states that anonymized individual-level patient data are not publicly available. Access may be considered after a written request and detailed research proposal, subject to participating institutions' procedures. The authors also link a public GitHub repository containing notebooks.The meaningful next checks are to document the data-application process and test whether the notebooks can run with permissible alternative inputs. The Noah screenshot records that code was NOT EXECUTED. Public code therefore should not be mistaken for unrestricted access to the clinical dataset or for successful reproduction of its results.
- Zhang et al. (2025): MRT Missing Data — Restricted Original Data, Synthetic Examples
In Missing data in microrandomized trials: Challenges and opportunities, the original datasets are available upon reasonable request rather than open public download. The paper also describes synthetic data and deposited code intended to illustrate the analysis procedure.This distinction matters. Executing the supplied script with synthetic data could help evaluate implementation and outputs in a demonstration setting. It would not reproduce the study's empirical results from the restricted original records. The Noah table marks code execution as NOT EXECUTED and recommends running the example while labeling its scope accurately.
- Zhuang et al. (2026): Clinical LLM — Data and Code by Request
Clinical large language model centered on electronic medical records states that underlying data cannot be shared publicly because of patient privacy. Requests are considered case by case under applicable institutional policies, while analytic code may be requested from the corresponding author.Here, the access pathway—not just the presence of a statement—is the object of review. A researcher should clarify eligibility, required approvals, the scope of any code that could be supplied, and the availability of the paper's 301MedQA benchmark materials. The Noah table lists code access as NOT VERIFIED; it does not establish that a request was made or denied.

Best Practices and Common Mistakes
- Use exact links and identifiers. Cite the paper, repository, accession, version, and date checked whenever possible.
- Do not merge different access levels. Public metadata, restricted underlying files, synthetic examples, and code upon request are distinct arrangements.
- Check repository documentation. An accessible README or notebook may still leave required data, dependencies, or commands uncertain.
- Protect controlled patient information. Do not infer that identifiable or governed data should be shared openly.
- Avoid unsupported judgments. “Not verified” means the review did not establish a claim; it does not mean the paper is fraudulent or the resource is inaccessible.
- Keep an audit trail. Record statements inspected, links followed, files observed, code executed if applicable, and tasks left for human review.
How to Review Data and Code Availability with Noah AI
The demonstrated Noah workflow begins in Agent → Deep Research. The research question asks the system to verify data and code statements across biomedical publications, assess repository access, identify reproducibility gaps, and document findings. Four specific DOI-linked studies are supplied so the output remains tied to identifiable publications.
- Open Noah AI Agent and select Deep Research.
- Provide the study titles, DOI links, and explicit verification questions for data, code, access restrictions, and reproducibility.
- Require the agent to differentiate claims in the paper from verified repository observations and actions not performed.
- Request a four-column Data Availability Assessment Table with one study per row and a practical next check.
- Review the citations and statements manually before using the output in a manuscript, evidence review, or reproducibility assessment.
The example screenshots document the research input and the generated summary table. They do not demonstrate that Noah downloaded patient data, executed GitHub repositories, or reproduced published statistical outputs.
Final Data and Code Availability Checklist
- Have the original article and supplementary statements been located?
- Are the data source, repository name, accession number, or DOI recorded accurately?
- Is it clear whether source files are public, restricted, synthetic, or available on request?
- Has the code access route been recorded separately from the data access route?
- Are the repository version, documentation, dependencies, and required inputs identifiable?
- Does the audit distinguish linked resources from inspected files and executed code?
- Are privacy restrictions and institutional approvals described without assuming noncompliance?
- Are unresolved questions and human verification steps documented?
Conclusion: Availability Is a Starting Point, Not Proof of Reproducibility
Data and code availability statements are essential for transparent biomedical publishing, but their meaning depends on the resources and conditions behind them. The four studies in this audit illustrate that public archives, restricted clinical datasets, synthetic demonstrations, and author-mediated access each create different opportunities and limitations for verification.Noah AI can help structure a source-grounded comparison and make missing checks visible. Researchers must still inspect primary sources, confirm access permissions, test code in an appropriate environment, and decide which findings can truly be reproduced. A clear account of what is reported, verified, and not yet checked is more informative than a superficial availability score.
Review Research Evidence with Noah AI
Organize availability statements, trace source references, and identify unanswered verification questions across biomedical publications.Try Noah AI for Free — Free Credits Available