How many new drugs are currently in clinical trials? In which phase? For which diseases? The answers to these questions are critical for pharmaceutical companies, yet collecting them is still largely done manually by searching through dozens of different websites. A team of three students from Big Blue Data Academy decided to change that.

The problem: massive data volume, zero uniformity

Every pharmaceutical company presents its pipeline data differently online. There is no common standard, meaning that until now, no single tool could reliably handle and process all this data.

Intelligencia AI, a company that utilizes artificial intelligence to accelerate drug development and reduce clinical risk, tasked the team with solving exactly this problem—at a scale covering 20 of the world’s largest pharmaceutical companies, including Roche, Eli Lilly, Sanofi, BMS, GSK, and Novartis.

The solution: eight AI agents

The team designed and built a system of eight specialized AI agents, with each handling a different part of the workflow. The first agent identifies the correct websites, the second "scrapes" them, the third recognizes their structure, and so on, down to the final agent that synthesizes all the data into a single, clean file.

To ensure the system's reliability, the team implemented a series of smart technical solutions: the system performs three independent searches for each company and keeps the result that appears at least twice, effectively preventing AI hallucinations (incorrect answers). At the same time, it "mimics" human web browsing to bypass technical obstacles, and every piece of extracted information comes with a confidence score.

The final result: over 500 drug entries, organized into 10 uniform data fields, ready for immediate integration into Intelligencia AI's platform.

Dimitris Mousadakos, CEO of Big Blue Data Academy, characteristically notes: "The students built something that a team of analysts would need weeks to do manually."

A launching pad for future projects

Four directions for further development were identified, which include the automatic recognition of any website structure without manual configuration, monitoring data changes with automatic alerts for new drugs or phase transitions, extracting data from images and infographics using AI vision, as well as accessing geographically restricted websites through smart request routing.