Building GreenOS: what went wrong with my evidence corpus

GreenOS is my attempt to turn the cannabis research and source material around my projects into a useful, traceable evidence collection. I want to be able to ask a specific question, find the relevant studies and follow a finding back to where it came from. Eventually, I want that work to be useful to other publishers and research teams as well.

A corpus is simply an organised collection of source material. Building one has taught me how much can go wrong between having a paper and having a dependable record of what it says. Text extraction can miss part of a page. A background paragraph can be mistaken for a result. A missing correction flag can be read as an all-clear.

That is where most of the interesting work has been. I am building the way sources are identified, extracted, challenged and kept up to date. I have made progress, and I have also had to correct earlier claims about how much of the system was finished.

A research question gives the collection a purpose

In September I focused one pilot on chronic pain. The discovery work used bounded queries across PubMed, Europe PMC, ClinicalTrials.gov and Crossref, with a mix of benefits, harms and interactions. The candidate catalogue contained metadata: enough to identify and organise reports, not a set of reviewed clinical findings.

That helped clarify what I was actually trying to build. A study, a paper about that study and a finding extracted from that paper are different things. One study can have several reports. A review can describe many studies. Counting every matching document as another piece of independent evidence would make a tidy database and a misleading answer.

The useful record needs to preserve those relationships: the people or population studied, what was compared, which outcome was measured, when it was measured and the passage supporting the value. The broader rubric is still being connected to the research loop. Finding a relevant title is an earlier stage than producing that record.

The one-hour experiment that stopped after seven minutes

I asked the research runner, which I call Luna, to work for an hour. It stopped after seven minutes and six seconds because the review backlog had reached its limit. There was plenty more material it could generate. The system could not yet deal with that material responsibly at the same rate.

That was a useful failure. Adding more model calls would have given me a larger queue. I needed a way to separate simple, provable checks from the questions that required interpretation, and to stop ambiguous records without stopping every other piece of work.

The next version added deterministic checks for fields and source identity, followed by a separate model pass that judged whether the supplied passage supported the proposed observation. It caught the sort of errors I actually cared about: the wrong field, an unsupported statement, background discussion presented as a finding, and missing context.

A second model pass is still a model pass. It can help identify a mismatch; it does not become an independent scientific review because it has a different job title. I kept internal acceptance, scientific appraisal and commercial release as separate decisions.

Three bugs that changed the way I handle evidence

Text that did not belong to the claimed source

The audit found that an extraction packet could contain text absent from the article it claimed to represent. A source link alongside the text had not been enough. The fix rebuilt the permitted paragraph spans from the retained, hash-checked XML whenever the record was validated. Changed text or text outside the allowed passage was then held.

It also found figure, table and supplementary passages entering a route intended for the article body. Tightening that scope caused some previously passing observations to fail. That made the numbers less impressive and the record more accurate.

An absent field interpreted as good news

The correction checker had relied on provider flags that were not actually present in the response. Missing data was being treated as no correction. I replaced that assumption with the publication types and correction relationships the provider does return, and retained the response that supported the decision.

Even then, no indexed correction in that response means exactly that. It does not establish that no correction exists anywhere. Keeping the statement that narrow matters when the source can change after I first collect it.

A restore that still depended on the original folder

Another defect was outside the research itself. Restored evidence still depended on paths in the original working folder. The archive could be unpacked, yet it was not genuinely self-contained. I changed the path handling and tested the restored copy with the original directory unavailable.

That one has a fairly wide application. A backup that works only while the original environment survives has not answered the question I care about. I want to know whether I can recover the work when that environment is gone.

Rights belong to the record, not the folder

Rights have their own trail through the system. Crossref makes an especially useful distinction: its bibliographic metadata is generally reusable, while abstracts retain publisher or author copyright. A response containing both does not give the two fields identical terms.

PMC also states that not every article is available for text mining and reuse, and that licence terms vary by article. That is why I record the source, the relevant permission, the fields it covers and the intended use. An accessible page is a route to material, not the end of the rights work.

The first small Crossref pilot deliberately kept to bibliographic metadata, with no abstracts or full text. Its candidates stayed held for review. That was a workable starting point because I could inspect what each field was, rather than pretending every file already in the project was ready to become a customer product.

What a useful record contains

I also keep missing values specific. Not checked, not reported, not applicable and conflicting each tell a different story. A passage without a dose value does not prove that the full paper never reported the dose. It may only mean I have not reached that part of it.

Hashes help keep track of which source and which extraction were reviewed. A hash is a fingerprint of the file's contents: if the contents change, a fresh comparison can reveal that. It can identify a changed record. It cannot tell me that a study's conclusion is sound.

What is working, and what is still unfinished

By the end of the 21 September audit, the corrected runner had accepted a small set of source-report observations for internal research. None became commercially eligible through that acceptance step. Earlier, higher admission figures were superseded after the source checks became stricter.

The remaining work was substantial: connecting the detailed evidence rubric, improving extraction of the older library, recording paginated discovery properly, and proving sustained useful output rather than a short successful run. Those gaps are part of the build, not an administrative footnote after it.

qmd has helped me find my own project decisions and runbooks while working through this. It searches the notes around GreenOS. It is not the engine deciding which research finding is correct, and it does not rewrite the corpus.

What I find rewarding here is turning a messy collection into something whose workings are visible. A useful answer needs more than a plausible sentence. I want the source, the interpretation and the remaining uncertainty to travel together, so that someone else can understand where the answer came from.

I am building towards that. The seven-minute run, the wrongly accepted passages and the awkward restore were all progress in their own way: they showed me what the system still could not do. The next version has to answer those problems, not just collect more material.

How qmd helps me find the decisions behind my projects

The shared process behind my publishing pipelines