Scotch software: an enduring success

Almost 33 years old and still going strong. Scotch continues to be used in a wide range of fields, from medicine to aeronautics, space and the automotive industry. Created in 1992 by François Pellegrini, this open source software now benefits from an Inria consortium via the InriaSoft programme, with one objective: to ensure its long-term survival.

Scotch's objective hasn't changed much since it was created," says its author, who is part of the Tadaam project team at the Inria centre at the University of Bordeaux. However, it still needs to be adapted to its various uses. In addition to research aimed at improving it, it therefore requires maintenance, which is made possible by the consortium".

For the time being, two engineers, sometimes assisted by trainees, are responsible for meeting the needs of Scotch's industrial and scientific users, who are constantly on the lookout for technological robustness.

Scotch's aim is to help its users parallelize their calculations, using a logic of separation in which several processors work simultaneously on the same overall task, but without getting in each other's way. The results of calculations on data that would often not fit in the memory of a single computer are obtained more quickly. It's like dividing up a difficult job between several people," says François Pellegrini. Instead of letting a single computer handle all the calculations in a simulation, several computers can each handle just one part of it.

To achieve this, the solution is to “slice and dice” graphs - mathematical structures representing the relationships in a set of data - in order to divide the calculation requirements into balanced subsets between the different processors.

The challenge? To reduce as far as possible the amount of information that needs to be exchanged between each part," answers the researcher. For example, if everyone in an office constantly needs to exchange information with a colleague at the other end of the room, it won't work optimally! Scotch therefore breaks down the problem in the best possible way so that each sub-problem can be solved as independently as possible from the others, by limiting the connections between them. This process, known as ‘partitioning’, is essential for optimising simulations in High-Performance Computing (HPC).

The Scotch software was born at a time when owning a personal computer was not so widespread and Windows users did not yet have access to the right click! Yet its principle has not changed, even with today's computers, which are infinitely more powerful than those of the time.

In François Pellegrini's view, this is hardly surprising: "Machines do more... but users always want more too! The descriptions of the problems to be solved are finer, the mesh denser, but the problem remains the same. So it's always necessary to chop up the work to improve performance and reduce the workload on the processors. And Scotch, a cutting-edge innovation in 1992, has been able to adapt to new uses over time to become a durable industrial software brick.

Today, the software is mainly used for large-scale digital simulation. For example? In the healthcare sector, it can be used to help reproduce a human heart. In the aeronautics sector, it helps an industrial group to design a digital model of an aircraft before embarking on its construction. Even NASA uses it for some of its digital simulation projects.

Scotch is also used as an embedded technology brick in several larger software packages. This is the case with PETSc, for example, a huge scientific computing library. Or OpenFOAM, a vast software library combining numerous tools. Users may not be aware of it, but there's a lot of Scotch in these behemoths.

By his own admission, François Pellegrini doesn't have the time to keep up with his “baby”. The researcher confides, "Some of my colleagues have had to maintain their products for several months, even years. That's not my aim."

Even if usage rates are difficult to determine, due to downloads from a multitude of sources, one thing is certain: in 2021, Scotch was Inria's sixth most popular software, with almost 280,000 downloads. It's a success story that didn't happen overnight. Originally, my thesis supervisor had no idea that this type of software could have so much influence," recalls François Pellegrini. For my part, I wanted to switch it over to an open licence so that it could be distributed as widely as possible. The researcher wanted this principle of openness for economic reasons - ‘public money should only pay once’ - but also to maximise Scotch's scientific impact. So any scientist can freely download the software and manipulate it as they wish.

But why create a consortium dedicated to its maintenance and sustainability? "Expertise is still rare. So you need specialists to develop the software, improve it and resolve bugs. And thanks to this structure, all this can be done without me, but with people who are trained to do it. It's also a winning choice for Scotch's many users: thanks to this consortium, they can be sure that the software will be updated regularly and that there will be no interruption to its operation, even if its creator is no longer able to devote himself to it.

Today, Scotch is undergoing a major development, with the creation of ScotchPy: an overlay enabling Scotch to interface with the Python computer language, which is increasingly used today, especially for processing large quantities of data. Historically coded in the C programming language, Scotch can now also be manipulated by Python users. This language is perfect for prototyping," says François Pellegrini. "Building an interface that links Scotch to Python requires some “plumbing” work, because we have bricks in C that would be expensive to recode. To put it plainly, we're going to offer Python, but keep the C under the carpet for all the background work."

After more than three decades, Scotch has clearly not finished spreading across the planet. This software proves that open source can be particularly relevant for maximising the scientific and industrial impact of a technological innovation.