José Antonio Oramas Mogrovejo associate professor

Research José Antonio Oramas Mogrovejo

Research team

Expertise

Research interest: - Visual Representation Learning. - Model Explanation and Interpretation. - Collective Representations and Relational Learning. - Disentangled Representation Learning.

Auto-generated simulator-based Datasets to advance machine Learning in Autonomous Shipping (ADLAS). 01/01/2025 - 31/12/2026

Abstract

ADLAS intends to develop datasets for AI model training in autonomous maritime ships. It is motivated by the need for safe and efficient automated vessel technology. Our research aims to expand AI exploration in autonomous vessels by creating diverse datasets for model testing in simulated environments. Simulators allow for rigorous evaluation of AI model performance and safety under various conditions, which are difficult to build in real life scenarios. The generated datasets will be foundational for future projects and safety standards in the maritime industry. Hence, our research aligns with evolving safety standards and aims to validate AI systems in the field of maritime operations. The outcome will be an open-source dataset which will advance international research collaboration in autonomous maritime ships. Leveraging collaborations and real-world training data, our research aims to enhance AI model adaptability and generalization capabilities. Overall, ADLAS aims to create comprehensive datasets for AI model training in autonomous maritime ships, utilizing AMA's full mission simulators and real-world teaching data to advance safety standards and collaboration in the field.

Researcher(s)

Research team(s)

Project type(s)

  • Research Project

Investigating the pathways and dynamics of ligand binding to dipeptidyl peptidases 4, 8, and 9 using molecular dynamics and deep learning approaches. 01/11/2024 - 31/10/2026

Abstract

The project proposal aims to investigate the dynamics associated with ligand binding on dipeptidyl peptidases (DPP) 4, 8, and 9. Despite biological interest in these systems, obtaining inhibition selectivity remains a challenge, given the similar active site architectures. However, very recently, compounds have been synthesized that are 10-100 orders of magnitude more selective for DPP9 than for DPP8 and DPP4. Although this is very promising, the issue remains that we do not understand the physicochemical and structural reasons for this selectivity. To address this lack of understanding, the proposal aims to investigate the dynamics of ligand binding to DPP4, 8, and 9, using a combination of molecular dynamics (MD)-based simulations and deep learning (DL) techniques. By generating large datasets of MD trajectories and using DL to analyse these simulations, key patterns that influence ligand binding will be investigated. The project will also focus on the functional role of the two channels that link the internal binding pocket with the solvent, with the aim of then identifying small molecules capable of binding in one of the channels. Several studies have used MD to study the dynamics of DPPs and to identify key residues involved in ligand binding. However, there have been no studies that have adapted DL techniques to investigate ligand binding dynamics in DPPs. This project is a collaboration between the Laboratory of Medicinal Chemistry (FBD; UA) and IDLab (UA/IMEC).

Researcher(s)

Research team(s)

Project type(s)

  • Research Project

Learning beyond memorizing: Topological data analysis of deep learning's generalization ability. 01/10/2024 - 31/05/2028

Abstract

Deep learning has achieved remarkable success in solving complex problems across diverse domains. Despite its widespread use, the fundamental concept of generalization to unseen data — which ensures that the model does not memorize (i.e., overfits) the training data but instead learns the underlying features that represent a broader range of examples — remains poorly understood. Generalization performance is commonly assessed post hoc via prediction accuracy on test data. Analyzing generalization without test data, however, unveils the learning process and whether the model is capturing the intended features. This commonly involves evaluating the model complexity, through an analysis of decision boundaries (which delineate different regions of the data space) and the model's learned parameters (which define the mapping of input data to predictions). Current efforts seek to establish generalization bounds or simple metrics correlating with the model's ability to generalize. This project instead aims to exploit topological data analysis, or more precisely persistent homology, to characterize the intrinsic structures within decision boundaries, trained parameters and activations, that contribute to superior generalization. Understanding this relationship holds significant potential for enhancing model design, interpretability and resource efficiency, and providing valuable insights into the behavior and limitations of deep learning, guiding future research directions.

Researcher(s)

Research team(s)

Project type(s)

  • Research Project

Flanders Artificial Intelligence Research program (FAIR) – second cycle. 01/01/2024 - 31/12/2028

Abstract

The Flanders AI Research Program is a strategic basic research program with a consortium of eleven partners: the five Flemish universities (KU Leuven, University of Ghent, University of Antwerp, University of Hasselt, Vrije Universiteit Brussel) and six research centers (imec, Flanders Make, VIB, VITO, Sirris and ILVO). The program brings together 300+ researchers on new AI methods that can be used in innovative applications in health, industry, planet&energy and society. This way, the program contributes to a successful adoption of AI in Flanders. The ambition is for Flanders to occupy a strong international position in the field of strategic basic research in AI, and this within a strong and sustainable Flemish ecosystem. Five focus research themes have been selected: responsible AI, human-centered AI, sustainable AI (energy-efficient and high-performance), productive and data-efficient AI (systems that require little data, which perform by combining data with domain knowledge and experience of experts) and resilient and high-performant AI (robust against changes in the environment). The description of the work packages and their research tasks defines the aspects within these themes that will be investigated in the program. The AI solutions are demonstrated in real-life use cases. These results not only demonstrate the effectiveness, but also inspire companies for adoption and researchers for further research. The Flanders AI Research Program is part of the Flanders AI Policy Plan. More info: www.flandersairesearch.be

Researcher(s)

Research team(s)

Project type(s)

  • Research Project

IMEC-Adaptive Human Operator Interaction with Autonomous Systems (AHOI). 01/12/2023 - 31/03/2027

Abstract

Autonomous systems are gaining wider traction in various applications including autonomous vessel navigation, autonomous driving and robotics. This is due to the availability of large amounts of data which is motivating a large body of research into the interpretation of this data and independent decision making on top of it. The hallmark of autonomous systems is that they are considered to be independent of human intervention in a general sense, and contain the potential to enhance the performance, reduce the costs and improve the safety in a diverse range of application domains. However, a major problem remains that these systems leverage the developments in AI which remain black box. Due to this, the nature of their decision making remains an open concern and their violation of the set policies or regulations in a certain application domain remains a risk. In order to address these concerns, the nature of interaction between humans and autonomous systems should be reimagined. Rather than considering one being the supervisor of the other, the goal should be to investigate the continuous involvement of each other as a teaming problem. Adaptive Human Operator Interaction with Autonomous Systems (AHOI) aims to address these concerns. Our novel research approach brings together a diverse set of researchers involved in various research domains such as Artificial Intelligence (AI), Explainable AI (XAI), human behavioral science, maritime personnel training and human machine interfaces (HMI) to solve the teaming problem between humans and autonomous systems in a use case of collision avoidance in the short sea shipping. We aim to solve this problem at multiple levels.

Researcher(s)

Research team(s)

Project website

Project type(s)

  • Research Project

Flanders Artificial Intelligence European Digital Innovation Hub (Flanders Al EDIH). 01/11/2022 - 31/10/2025

Abstract

Many Flemish companies are aware of the potential impact of AI, but most SMEs have not yet investigated how AI might affect their business. Given the technology potential, there is an urgency to accelerate the adoption of AI in Flanders. The Flanders AI EDIH accelerates the adoption of AI among (especially) SMEs and public sector organisations by an integrated service offering: (1) Test before invest: initial advice, individual coaching, AI technical feasibility study, legal workshop, Start AI, (2) Skills and training: AI inspiration session, thematic webinar & event, masterclass, AI Summer school, (3) Support to find investments: info session, financial literacy course, matchmaking & finfinder guidance (4) Innovation ecosystem and networking: talent & skills matchmaking, matchmaking on AI supply & demand, matchmaking on joint research, Flanders AI Forum. The Flanders AI EDIH consists of complementary partners who guarantee a cross-sectoral and accessible Flemish operation, with a local physical presence in every Flemish province. The Flanders AI EDIH strengthens existing Flemish initiatives to prevent a fragmentation of available AI innovation services, and actively aligns its service offer through close collaboration with the Flemish Industry Partnership, the Flanders AI Research Programme, the Flanders AI Academy (VAIA), the Flemish Supercomputer Centre (VSC) and Enterprise Europe Network (EEN) Flanders. The Flanders AI EDIH currently offers a number of smart spaces as testing and experimentation facilities, maintain direct links with existing innovation actors and initiatives (such as the Flemish sectorial focused DIHs) and is closely aligned with the Flemish AI policy plan. At European level, the Flanders AI EDIH has ongoing structural collaborations through the AI DIH Network, the Smart Connectivity DIH Network (SCoDIHNet), the EPoSS Smart Systems EDIH Task Force, the Vanguard AI Pilot.

Researcher(s)

Research team(s)

Project type(s)

  • Research Project

Accident-prone Vision-based Simulation for Autonomous Safety-critical Systems 01/11/2022 - 31/10/2025

Abstract

Autonomous navigation has been gaining much traction recently. As a result, we see autonomy developing in vehicles and finding its way in many transportation sectors (including smart shipping). Nevertheless, the current state-of-the-art (SOTA) technology is not mature enough to have a widespread application at a higher autonomy level (e.g. level 4 and above). The main reason is that these systems are trained on a lot of real-world data, which often lacks accident-prone scenarios. In order to solve this problem, I propose a solution based on data-driven neural simulations that provide realistic data based on real-world samples and generate unsafe scenarios (collisions, accidents, etc.). Moreover, my system also provides safety checks to validate unsafe scenarios and provide safe boundaries for the current autonomous systems.

Researcher(s)

Research team(s)

Project type(s)

  • Research Project

Antwerp Text Mining Centre (TEXTUA). 01/01/2022 - 31/12/2026

Abstract

Most knowledge is stored in unstructured data like text, which must be structured before it can be mined. The need and opportunities for this automatic text analysis have considerably increased recently with developments in Artificial Intelligence, not only in the humanities and social sciences, but also in the exact and medical sciences. The mission of the ATMC is to provide scalable solutions to researchers from any scientific discipline that wants to analyze and use large amounts of textual data. Text data should be seen here in a broad sense including automatically transcribed speech, written text in images, and images automatically described in text. ATMC bundles the unique existing expertise in digital text analysis at the University of Antwerp with special emphasis on explainable AI and will provide the capacity to support the growing number of interdisciplinary queries that reach us today.

Researcher(s)

Research team(s)

Project type(s)

  • Research Project

IMEC-Super Bio-Accelerated Mineral weathering: a new climate risk hedging reactor technology (BAM). 01/09/2021 - 31/08/2025

Abstract

Conventional climate change mitigation alone will not be able to stabilise atmospheric CO2 concentrations at a level compatible with the 2°C warming limit of the Paris Agreement. Safe and scalable negative emission technologies (NETs), which actively remove CO2 from the atmosphere and ensure long-term carbon (C) sequestration, will be needed. Fast progress in NET-development is needed, if NETs are to serve as a risk-hedging mechanism for unexpected geopolitical events and for the transgression of tipping points in the Earth system. Still, no NETs are even on the verge of achieving a substantial contribution to the climate crisis in a sustainable, energy-efficient and cost-effective manner. BAM! develops 'super bio-accelerated mineral weathering' (BAM) as a radical, innovative solution to the NET challenge. While enhanced silicate weathering (ESW) was put forward as a potential NET earlier, we argue that current research focus on either 1/ ex natura carbonation or 2/ slow in natura ecosystem-based ESW, hampers the potential of the technology to provide a substantial contribution to negative emissions within the next two decades. BAM! focuses on an unparalleled reactor effort to maximize biotic weathering stimulation at low resource inputs, and implementation of an automated, rapidlearning process that allows to fast-adopt and improve on critical weathering rate breakthroughs. The direct transformational impact of BAM! lies in its ambition to develop a NET that serves as a climate risk hedging tool on the short term (within 10-20 years). BAM! builds on the natural powers that have triggered dramatic changes in the Earth's weathering environment, embedding them into a novel, reactor-based technology. The ambitious end-result is the development of an indispensable environmental remediation solution, that transforms large industrial CO2 emitters into no-net CO2 emitters.

Researcher(s)

Research team(s)

Project type(s)

  • Research Project

IDLab - Internet and Data Lab 01/01/2021 - 31/12/2026

Abstract

The IOF consortium IDLab is composed of academic supervisors at the IDLab Research Group, a UAntwerp research group with members from the Faculty of Science and the Faculty of Applied Engineering. IDLab develops innovative digital solutions in the area of two main research lines: (1) Internet technologies, focusing on wireless networking and Internet of Things (IoT), and (2) Data science, focussing on distributed intelligence and Artificial Intelligence (AI). The mission of the IDLab consortium is to be the number one research and innovation partner in Flanders and leading partner worldwide, in the above research areas, especially applied in a city and its metropolitan surroundings (industry, ports & roads). To realize its mission, IDLab looks at integrated solutions from an application and technology perspective. From an application point of view, we explicitly provide solutions for all stakeholders in metropolitan areas aiming to cross-fertilize these applications. From a technological point of view, our research includes hardware prototyping, connectivity and AI, enabling us to provide a complete integrated solution to our industrial partners from sensor to software. Over the past years, IDLab has been connecting the city and its surroundings with sensors and actuators. It is time to (1) reliably and efficiently connect the data in an integrated way to (2) turn them into knowledgeable insights and intelligent actions. This perfectly matches with our two main research lines that we want to extensively valorise the upcoming years. The IDLab consortium has a unique position in the Flemish eco-system to realize this mission as it is strategically placed across different research and innovation stakeholders: (1) IDLab is a research group embedded in the Strategic Research Centre imec, a leading research institute in the domain of nano-electronics, and more recently through groups such as IDLab, in the domain of digital technology. (2) IDLab has a strategic link with IDLab Ghent, a research group at Ghent University. While each group has its own research activities, we define a common strategy and for the Flemish ecosystem, we are perceived as the leading partner in the research we are performing. (3) IDLab is the co-founder of The Beacon, an Antwerp-based eco-system on innovation where start-ups, scale ups, etc. that work on IoT and AI solutions for the city, logistics, mobility and industry 4.0 come together. (4) Within the valorisation at UAntwerp, IDLab contributes to the valorisation within the domain 'Metropolitanism, Smart City and Mobility'. To realize our valorisation targets, IDLab will define four valorisation programs: VP1: Emerging technologies for next-generation IoT; VP2: Human-like artificial Intelligence; VP3: Learning at the edge; VP4: Deterministic communication networks. Each of these valorisation programs is led by one of the (co-)promoters of the IDLab consortium, and every program is composed of two or three innovation lines. This way, the IDLab research will be translated into a clear program offer towards our (industrial) partners, allowing us to build a tailored offer. Each valorisation program will contribute to the different IOF objectives, but in a differentiated manner. Based on our current experience, some valorisation programs are focusing more on local partners, while others are mainly targeting international and EU funded research projects.

Researcher(s)

Research team(s)

Project type(s)

  • Research Project

Digitize the monitoring of construction projects by connecting and enhancing Building Information Models with real-time on-site progress and activity data, analyzed with AI technology (BoB). 01/01/2022 - 31/12/2023

Abstract

Building Information Modeling (BIM) provides insight into the design and planning of a building. However, the benefits of this digital display are not yet fully exploited at the construction site. Schedule updates today are often still done manually and are prone to errors, leading to errors, delays and cost overruns. BoB solves these problems by linking BIM to real-time progress and activity data on the construction site. BoB will use AI-driven technology to detect collective activities (e.g. pouring concrete, formwork, digging) and automatically link the current state of the building from image data to the BIM design. This gives site stakeholders much-needed visibility into actual progress, reduces cost overruns, increases efficiency, prevents errors and reduces construction waste. The connected data platform designed in BoB will be a stepping stone to a fully connected digitized construction site.

Researcher(s)

Research team(s)

Project type(s)

  • Research Project

Learning-based representations for the automation of hyperspectral microscopic imaging and predictive maintenance. 01/09/2020 - 30/11/2024

Abstract

In this project we will focus on designing a model for representation learning that will enable the detection of pollution in microscopic samples at the earliest time possible from hyperspectral images (HSI). Current methods for this task operate of on top of RGB images derived from HSIs. Taking this into account we will focus our efforts on designing a method capable of analyzing the full raw data cube that composes each HSI sample and identifying potential signals to enable the accurate detection of pollution in the sample. In addition, as industrial customers become increasingly aware of the growing maintenance costs and downtime caused by the unexpected machinery failures, predictive maintenance solutions for biopharma companies gain more interest to maintain a competitive advantage. To address this issue, we will investigate methods to analyze data traces coming from different sources, e.g. computer logs, operator reports, quality of the collected samples, etc., in order to identify temporal patterns that can serve as strong indicators of a potential anomaly that will occur in the near future on the monitored systems. Finally, for both of the tasks mentioned above, model explanation algorithms will be investigated and designed so that the predictions made by their respective models can be justified. Moreover, these explanation algorithms, will serve to debug the trained models and assess their validity and robustness towards artifacts, e.g. biases, data leakage, etc., introduced during the training stage.

Researcher(s)

Research team(s)

Project type(s)

  • Research Project

Multimodal Relational Interpretation for Deep Models. 01/05/2020 - 30/04/2024

Abstract

Interpretation and explanation of deep models is critical towards wide adoption of systems that rely on them. Model interpretation consists on getting an insight on the information learned by a model from a set of examples. Model explanation focuses on justifying the predictions made by a model on a given input. While there a is a continuously increasing amount of efforts addressing the task of model explanation, its interpretation counterpart has received significant less attention. In this project we aim taking a solid step forward on the interpretation and understanding of deep neural networks. More specifically, we will focus our efforts on four complementary directions. First, by reducing the computational costs of model interpretation algorithms and by improving the clarity of the visualizations they produce. Second, by developing interpretation algorithms that are capable of discovering complex structures encoded in the models to be interpreted. Third, by developing algorithms to produce multimodal interpretations based on different types of data such as images and text. Fourth, by proposing an evaluation protocol to objectively assess the performance of algorithms for model interpretation. As a result, we aim to propose a set of principles and foundations that can be followed to improve the understanding of any existing or future deep complex model.

Researcher(s)

Research team(s)

Project website

Project type(s)

  • Research Project

A-budget IMEC. 01/01/2020 - 31/12/2021

Abstract

This project is part of the IMEC Frame Agreement and is being given as structural investment for fundamental research based on yearly set KPIs from the group to IMEC. This A-budget is defined within the IMEC Way of Working and part of the frame agreement of the University of Antwerp and IMEC.

Researcher(s)

Research team(s)

Project type(s)

  • Research Project