icbnet

Modern simulation applications that carry out large scale iterative processes, such as Monte-Carlo simulations, or manipulate large data structures, tend to be extremely time- consuming due to the shortage of computational resources or the inherent nature of the simulation itself. Typical simulations can take up to several days to complete on conventional systems, while high-end supercomputers can be cost-prohibitive. Therefore, the need for effective parallelization of software execution and resource management is more imperative. The goal of this paper is to present a fully-distributed platform that enables software simulations to be executed within user-acceptable time periods, by predicting the resource requirements of each simulation. In this context, the platform analyzes files that contain historical data about past executions of the particular simulation. Processor and memory utilization, overall execution time, level of parallelization and distributed execution are some of the information collected and used by an efficient training algorithm, in order to determine the optimal amount of resources to be allocated in a particular simulation. The training algorithm applies several machine-learning techniques such as multi-linear regression in order to efficiently predict the resource vector that will satisfy the user requirements.