Speakers
Description
EOSC-ARENA is a European project aimed at advancing Generative AI (GenAI) capabilities for open science within the European Open Science Cloud, bringing together work on infrastructure, model adaptation, responsible AI, knowledge retrieval, and community-facing services. Given the breadth of the project, this work focuses exclusively on our contribution to the technical layer for GenAI model serving and knowledge retrieval.
Our contribution aims to enable scalable, reproducible, and interoperable access to large language and multimodal models across heterogeneous cloud environments, while supporting service-level agreements, API compatibility, observability, and integration with existing AI4EOSC developments. The technical trajectory builds on OSCAR, developed by UPV, and the LLM4EOSC solution developed by IFCA, considering their role in flexible deployment modes for GenAI services, including serverless inference, managed federated instances, and self-hosted private model-serving environments. Particular attention is given to the integration of OSCAR with model-serving technologies such as KServe and vLLM, and to the role of the Infrastructure Manager in automating deployments across distributed cloud resources and in future EOSC EU Node scenarios.
This contribution is framed by complementary extensions that equip the infrastructure with agentic Service Level Agreement (SLA) management capabilities through frameworks such as Governify and SLA Wizard. From this agentic perspective, all platform components will be provided with advanced observability mechanisms, enabling the continuous collection and analysis of operational, performance, and metrics across the infrastructure. Based on this information, the system will support the definition of policies, objectives, thresholds, and evaluation criteria for the different resources and services, and potentially trigger appropriate management or enforcement actions. These capabilities will contribute to a more adaptive, proactive, and fine-grained infrastructure management.
Overall, the work aims to provide an integrated foundation for controlled GenAI serving, intelligent retrieval, and future agentic AI workflows in EOSC-ARENA.
This work has been supported by the EOSC-ARENA project, funded by the European Union under Horizon Europe Grant Agreement No. 101292597 (HORIZON-INFRA-2025-01).