Beyond RAG: When To Use Semantics, Retrieval, Graph, And Agent Inference

By Sam Awotunde, Director, Biomedical Informatics and Innovation Strategy Lead, AI/ML, Quantitative and Digital Sciences, Pfizer; and Harini Gopalakrishnan, CEO and Founder, TheHGFactor and Chief AI Officer, Streamcare Group Health

In Part 1, we looked at why life sciences AI agents need more than retrieval-augmented generation (RAG) and the importance of context. For life sciences, context is not just a set of retrieved documents. It includes structured clinical facts, unstructured evidence, governed definitions, stable identifiers, controlled terminology, and rules about which relationships can be inferred and which must be validated. That is why the target prioritization or medical evidence generation are useful examples, as they require the system to bring together structured data, unstructured evidence, semantic meaning, and scientific judgment. But it also raises other practical questions in terms of technical implementation:

• Do all AI use cases need the same level of semantic complexity?
• Should everything be represented in a graph?
• Should all terminology be resolved before retrieval?

The answer to all that depends on the consequence of getting the relationship wrong and the level of regulatory rigor needed for the use case. Let’s dive in.

Classifying Your Needs

A practical way to think about this is to classify AI use cases into three tiers: assistive, operational, and regulated.

Tier 1: Assistive

Consider an oncology researcher pursuing a target and exploring, what emerging mechanisms are being investigated for resistance to KRAS G12C inhibitors? Emerging mechanisms can include novel technology, such as multiomics analytics, applying LLM based techniques like AlphaFold to study druggable pockets, or leveraging advanced capabilities such as organ-on-a-chip studies among others. In all such cases, the user is exploring a vast scientific landscape of data, identifying emerging hypotheses, and navigating literature. Semantic expansion, embeddings, retrieval ranking, and LLM-assisted concept matching may be useful to pursue all such mechanisms involved in identifying KRAS G12C inhibitors in external pursuits and linking them to internal assays for correlation studies. However, the overarching theme here is to quickly assemble and mine the varied information to create an exploratory hypothesis, not to meet regulatory standards of rigor. The system should provide basic citations, provenance, and transparency, but not every inferred relationship needs to be represented as a validated enterprise fact.

In this tier, AI can help the user navigate the information space. A graph cross walk is not necessary. In fact, modeling every relationship into a graph can create unnecessary technical debt, which can be avoided. Instead, it’s acceptable to allow the frontier models to do a LLM based retrieval to keep architecture simple. These use cases for this stack are typically to support early-stage hypothesis generation, like target identification for a new study, not necessarily to help prioritize identified targets.

The pattern, therefore, becomes:

Propose → Retrieve → Rank → Synthesize

Tier 2: Operational

Now consider a drug discovery team evaluating the use case seen in Part 1: Given the available genetic, functional, and preclinical evidence, should a PTEN-associated therapeutic hypothesis be prioritized for further investigation?

This workflow requires evidence from structured genomic, expression, dependency screening, and assay datasets, alongside scientific publications, internal experimental reports, and pathway biology. The system may use AI to interpret the question, identify relevant biological concepts, and search across these sources. However, as seen in Part 1, important semantic distinctions must be preserved before evidence can be reliably combined for accuracy. We can’t let an LLM infer it like in the Tier 1 scenario.

To recap, PTEN genomic deletion, a PTEN loss-of-function sequence variant, and PTEN protein loss by immunohistochemistry (IHC) are biologically related but not equivalent observations. A semantic layer is needed to resolve PTEN aliases, normalize the different biomarker states, and connect them to governed biological concepts such as PI3K–AKT signaling. This ensures that we do not automatically treat all forms of PTEN loss as interchangeable.

The pattern, therefore, becomes:

Propose → Semantic validation → Retrieve → Rank → Synthesize

The retrieval layer can then identify the relevant genomic and functional data, scientific documents, experimental results, tables, and figures. Context orchestration determines which evidence is necessary for the specific target-prioritization question, while the agent synthesizes the findings, including conflicting or inconclusive evidence.

The system may also consult organizational memory to identify whether the hypothesis was previously evaluated, which experiments failed, and why a target was deprioritized.

In this tier, graph RAG may be useful, particularly when relationships between targets, pathways, biomarkers, and disease mechanisms need to be traversed. However, it is not automatically mandatory. Governed terminology, semantic validation, and appropriate retrieval may be sufficient for many operational workflows.

The point is not to add graph complexity by default; it is to introduce semantic control where the scientific or business process requires it.

Tier 3: Regulated

Now consider a workflow supporting a clinical submission, validated endpoint interpretation, safety analysis, or a regulatory assessment involving biomarker-defined patient populations.

Here, governed semantic relationships become especially important for reproducibility. An LLM should not decide that two endpoint definitions are equivalent because their descriptions look similar.

Consider the PTEN example again. If clinical studies define patient populations using PTEN genomic deletion in one case and PTEN protein loss by immunohistochemistry in another, an LLM should not automatically treat those populations as equivalent.

Their comparability depends on the underlying biological definitions, assays, thresholds, and study criteria (refer to the Part 1 example on study definitions).

Similarly, the system should not invent a crosswalk between biomarker classifications, reinterpret a validated Medical Dictionary for Regulatory Activities (MedDRA) hierarchy, or conclude that two patient populations are comparable without governed rules or human validation.

The pattern, therefore, becomes:

Propose → Graph retrieval → Semantic validation → Retrieve → Rank → Synthesize

In this tier, the semantic layer establishes approved terminology and governed relationships. Retrieval identifies the supporting evidence, and the agent reasons within those boundaries. A knowledge graph can serve as a control mechanism where explicit relationships, traceability, and reproducibility are needed, but it is not the only possible implementation.

The principle is straightforward: Use AI where inference is useful; use governed semantics where inference is unacceptable.

The Three Tiers Should Apply Across The Stack

This classification is not only about graphs. The same governance question should apply across the catalog, ontology, warehouse, retrieval system, and agent. A catalog in enterprises can mean many things: technical, operational, business, and semantic. For this discussion, the focus is the semantic catalog, or semantic layer, because that is what governs relationships between business concepts, scientific entities, metrics, and rules.

The semantic catalog becomes particularly important for graph RAG because it provides governed meaning and relationships that retrieval can use instead of relying only on LLM inference (e.g., Study A àdefines patient population à PTEN loss of function).

Technical metadata tells you where something lives. Semantics tells you what it means. Governance tells you what the system is allowed to infer or use. For any AI use case, three governance questions are useful:

1. What can the model infer?
2. What must the enterprise define?
3. What evidence must be traceable?

An exploratory literature agent will answer these differently from a medical-response workflow or a regulatory analysis agent. This prevents two common architectural extremes:

• The first is under-governance, where an LLM is allowed to infer relationships that should have been deterministic.
• The second is over-engineering, where teams attempt to model every possible enterprise relationship in a knowledge graph before the first valuable AI use case has gone live.

Neither is necessary.

Bringing It All Together: Revisit The Architecture For The Drug Discovery Example

Most large pharmaceutical organizations already own many of the components required for this architecture. They have warehouses, lake houses, catalogs, terminology-management platforms, and ontologies. Some have graph databases, such as Neo4j or Amazon Neptune. They have vector stores, enterprise search systems, literature platforms, and increasingly multiple agent frameworks.

The problem is that these systems often evolve independently.

Revisiting the PTEN example:

• a genomic data set may encode “PTEN_DEL”
• an experimental report may describe “PTEN loss”
• another source may use the historical alias MMAC1
• a publication may refer to the “absence of PTEN staining”
• a pathology report may record “PTEN negative by IHC.”

The detailed architecture and flow is provided below in Figure 1.

Figure 1. Layered architecture for a PTEN evidence question. Context orchestration breaks a composite question about PTEN-altered tumors into sub-questions, routes each to the layers it needs, and assembles the results for the AI system. The semantic layer makes governed concepts consistently queryable, distinguishing “PTEN genomic alterations,” “PTEN protein loss,” and the broader “loss of PTEN function.” The knowledge graph retrieves through explicit relationships (for example, PTEN biomarker state → assay → patient population → study), and unstructured retrieval finds evidence in publications and internal documents. Ontology and semantic mapping underpin both: they treat PTEN, MMAC1 and NCBI Gene 5728 as the same gene while keeping deletion, loss-of-function variant, and protein loss by IHC distinct.

Knowledge Graph Layer (Ontology)

Ontologies establish the underlying biological meaning. Knowledge graphs connect PTEN to signaling pathways, assays, disease contexts, and experimental evidence to ensure the context is complete for further downstream retrieval.

Semantic Layer

A semantic layer makes these concepts and relationships consistently queryable across enterprise sources. Without semantic alignment, the AI system misses relevant evidence and incorrectly combines observations that are biologically related but not equivalent. Semantic mapping resolves source-specific terms to common concepts while preserving distinctions between alteration types.

Context Orchestration

Context layer performs the next task: determining which structured data sets, scientific documents, pathway relationships, and prior experimental memories are relevant to the target-prioritization decision being made.

The goal is cohesion across existing capabilities rather than replacing them with another monolithic system. The platform should not be another isolated AI application. It should be a reusable context architecture that makes enterprise knowledge increasingly usable by every agent built on top of it.

• A model can recognize that PTEN deletion and PTEN protein loss are biologically related. A governed semantic layer preserves the distinction between those observations
• A model can summarize preclinical findings. A retrieval system determines whether it found the right study, experimental results, tables, figures, and assay definitions based on the relevance features
• A model can reason across evidence. Context orchestration determines which structured results, scientific interpretations, and prior experimental findings should inform that reasoning.

The final AI-ready semantic fabric helps establish whether the evidence placed before the model is sufficiently relevant, traceable, and governed for the intended task.

Finally, before choosing the right architecture, whether it is graph, vector database, or agent framework, teams should answer a more basic set of questions to avoid the over engineering pitfall. Using the oncology medical and PTEN examples, the decision map might look like Table 1.

Table 1: Breaking down the tech decision stack for a target prioritization AI system

For organizations building clinical, medical, or R&D agents, we would start with four steps:

1. classify the use case as assistive, operational, or regulated
2. define what AI-ready means for the structured and unstructured data involved
3. identify which concepts and relationships require governed semantics
4. pilot one end-to-end journey where structured data, semantic sources, retrieval, and agent reasoning work together.

That is the architectural shift; not from RAG to something entirely different, but from retrieving documents to orchestrating governed enterprise context.

Takeaway: Structured data identifies quantitative signals. Unstructured evidence provides the experimental and biological interpretation. Governed semantics ensures that related but distinct PTEN observations are not conflated. Context orchestration determines which evidence is needed to evaluate the therapeutic hypothesis.

The model did not get smarter; the context did.

About The Authors

Sam Awotunde is a transformation solution architect at the intersection of AI, data, and biopharmaceutical R&D. At Pfizer, he builds the systems and teams that turn raw scientific data into decision-grade intelligence — leading cross-functional efforts in AI/machine learning, data products, and advanced analytics that scale across the enterprise. A strategic advisor to senior leadership, he translates emerging technologies into measurable enterprise impact, accelerating AI adoption across organizations.

Harini Gopalakrishnan is a life sciences technology strategist with deep expertise in AI, data platforms, and the pharmaceutical/biotech industry. She is the CEO and founder of The TheHaze.ai while also providing advisory services and AI strategy for life sciences via TheHGFactor LLC. She is also the chief AI officer at Streamcare Group Health and fractional industry general manager for Vespa.ai, a tech company powering AI search. She previously served as global CTO for life sciences at Snowflake and has held leadership roles at AWS, Sanofi, and Cognizant. She blends technical depth with industry insight, shaping go-to-market and AI solutions for pharma. Harini holds a master’s degree in bioinformatics and an undergraduate degree in computer science engineering.