SeltaSquare: Pharmacovigilance Model Now Factors in “Severity,” Cutting False-Positive Rate by 65% in Simulation
페이지 정보
본문
Addresses the information loss of conventional adverse event detection by distinguishing mild, moderate, and severe cases
Combines patient characteristic adjustment with AI-based severity prediction to refine drug safety analysis
Selects signal candidates using an integrated score and suggests priorities for expert review
False-positive rate down 65.2% vs. ROR and 58.3% vs. PRR; validation in real-world clinical settings remains a task for future work
SeltaSquare has presented a pharmacovigilance model that distinguishes between mild and severe adverse events, analyzes their association with drugs, and sets priorities for expert review.
The model combines the statistical results of the drug–adverse event association analysis with severity differences estimated by an AI model into a single score.
This score is used to rank the drug–adverse event pairs that warrant further investigation.
Donguk Kim, Head of the Data Center at SeltaSquare, presented “A Pharmacovigilance Signal Detection Framework Incorporating Adverse Event Severity” in the pharmacovigilance session of the 2026 KoNECT International Conference (KIC 2026), held on the 14th at the Lotte Hotel Seoul.
KIC 2026 is an international event jointly hosted by the Ministry of Health and Welfare, the Ministry of Food and Drug Safety, and the Korea National Enterprise for Clinical Trials (KoNECT). It opened on the 13th and runs through the 15th. Now in its 12th year, it covers key issues in clinical development, including artificial intelligence (AI), real-world data (RWD), and decentralized clinical trials (DCT).
SeltaSquare supports pharmaceutical and biotech companies with adverse event collection, assessment, and regulatory reporting, as well as RWD-based research. RWD refers to data accumulated in actual clinical practice, such as electronic medical records and insurance claims data.
The company also develops solutions that use AI and digital technologies to reduce repetitive tasks such as literature searches and data entry.
“Lumping mild and severe cases together causes information loss”
Pharmacovigilance (PV) is the practice of detecting, assessing, and preventing adverse events and other safety issues that occur during medication use. A signal is information that suggests a new potential causal relationship between a drug and an adverse event, or a new aspect of an already known relationship.
Conventional spontaneous report analysis examines whether a specific drug and adverse event are reported together more often than with other drugs. The representative measures are the Proportional Reporting Ratio (PRR) and the Reporting Odds Ratio (ROR).
PRR compares the proportion of reports for the drug of interest that involve a specific adverse event against the same proportion for other drugs. ROR calculates the ratio of reports with a specific adverse event to reports with other adverse events, then compares this ratio between the drug of interest and other drugs.
“Existing measures are useful for quickly narrowing down candidates for review in large datasets, but because the total number of drug users is unknown, it is difficult to calculate actual incidence rates,” Kim pointed out. “In particular, values fluctuate widely when the number of reports is small, and because severity is not distinguished, it is hard to know how mild or serious the symptoms are.”
He continued, “Putting information on mild and severe cases into a single category results in a significant loss of information. Developing a signal detection method that reflects adverse event severity was another motivation for starting this study.”
“Differences in occurrence and expected severity must be calculated together”
The analysis Kim presented is based on an “active comparator, new-user design,” which compares new users of drugs with similar therapeutic purposes. Patient cohorts are formed from those who have newly started the drug of interest and the comparator drug.
Adverse events are classified into four categories: no occurrence, mild, moderate, and severe. Unlike a binary analysis that only distinguishes whether an event occurred, this approach uses “ordinal” information that preserves the order of increasing severity.
Patients in the two drug groups may differ in characteristics such as underlying conditions and concomitant medications. To reduce confounding from these differences, the study applied inverse probability of treatment weighting (IPTW).
This method adjusts each patient’s analytical weight based on the probability of receiving a given drug given their characteristics, so that the two groups become similar. It does not remove the influence of unmeasured factors.
In the statistical analysis, the association between the drug and the adverse event is examined at each severity threshold. The AI-based ordinal model takes the same patient characteristics, assumes the patient uses the drug of interest and the comparator drug in turn, estimates the expected adverse event severity in each case, and calculates the difference.
“Existing methods only distinguish whether an adverse event occurred or not,” Kim explained. “The method I propose calculates odds ratios by severity and also derives the difference in expected severity between the two drugs.” He added, “The two results are combined to create a single score.”
The weight given to each of the two results and the threshold for selecting candidates are tuned using positive reference pairs with known associations and negative reference pairs that serve as a comparison baseline.
The same calculation is then applied to drug–adverse event pairs whose associations have not yet been confirmed, assigning each a score and a review rank.
4% false-positive rate in simulation; real-world validation remains a task
Not every signal candidate leads to an actual drug-related risk. If many pairs set as negative in the evaluation are wrongly classified as positive (false positives), the number of cases that experts must check also increases.
The researchers used simulation to evaluate whether the proposed model can reduce such misclassification while still identifying the pairs set as positive.
In an evaluation that repeated data splitting 50 times, the proposed model’s average false-positive rate was 4.0%. This was lower than 11.5% for the conventional ROR and 9.6% for PRR, representing relative reductions of 65.2% and 58.3%, respectively. The rate at which it identified pairs set as positive was also higher than that of the existing measures.
“After detecting signals, I thought about the order in which experts should review them,” Kim said.
A high score does not confirm a drug’s risk or causality. Experts must evaluate it by considering patient records and related studies. Further validation is also needed to confirm whether the same performance can be reproduced in real clinical settings and with external data, and whether it actually reduces the review burden.
“This study is a starting point,” Kim said. “Going forward, we will use more data and combine AI technologies to develop the methodology through diverse analyses. Our goal is to improve signal detection performance, including through joint research with researchers who hold relevant data.”
The Original Article: [약업신문]셀타스퀘어 약물감시 중증도까지 반영…시뮬레이션서 위양성률 65% 감소
