24
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      AiiDA 1.0, a scalable computational infrastructure for automated reproducible workflows and data provenance

      research-article

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          The ever-growing availability of computing power and the sustained development of advanced computational methods have contributed much to recent scientific progress. These developments present new challenges driven by the sheer amount of calculations and data to manage. Next-generation exascale supercomputers will harden these challenges, such that automated and scalable solutions become crucial. In recent years, we have been developing AiiDA (aiida.net), a robust open-source high-throughput infrastructure addressing the challenges arising from the needs of automated workflow management and data provenance recording. Here, we introduce developments and capabilities required to reach sustained performance, with AiiDA supporting throughputs of tens of thousands processes/hour, while automatically preserving and storing the full data provenance in a relational database making it queryable and traversable, thus enabling high-performance data analytics. AiiDA’s workflow language provides advanced automation, error handling features and a flexible plugin model to allow interfacing with external simulation software. The associated plugin registry enables seamless sharing of extensions, empowering a vibrant user community dedicated to making simulations more robust, user-friendly and reproducible.

          Related collections

          Most cited references38

          • Record: found
          • Abstract: not found
          • Article: not found

          Efficient iterative schemes forab initiototal-energy calculations using a plane-wave basis set

            Bookmark
            • Record: found
            • Abstract: not found
            • Article: not found

            The SIESTA method forab initioorder-Nmaterials simulation

              Bookmark
              • Record: found
              • Abstract: not found
              • Article: not found

              cp2k:atomistic simulations of condensed matter systems

                Bookmark

                Author and article information

                Contributors
                mail@sphuber.net
                giovanni.pizzi@epfl.ch
                Journal
                Sci Data
                Sci Data
                Scientific Data
                Nature Publishing Group UK (London )
                2052-4463
                8 September 2020
                8 September 2020
                2020
                : 7
                : 300
                Affiliations
                [1 ]GRID grid.5333.6, ISNI 0000000121839049, National Centre for Computational Design and Discovery of Novel Materials (MARVEL), École Polytechnique Fédérale de Lausanne, ; CH-1015 Lausanne, Switzerland
                [2 ]GRID grid.5333.6, ISNI 0000000121839049, Theory and Simulation of Materials (THEOS), Faculté des Sciences et Techniques de l’Ingénieur, École Polytechnique Fédérale de Lausanne, ; CH-1015 Lausanne, Switzerland
                [3 ]GRID grid.5333.6, ISNI 0000000121839049, Laboratory of Molecular Simulation (LSMO), Institut des Sciences et Ingénierie Chimiques, École Polytechnique Fédérale de Lausanne (EPFL), Rue de l’Industrie 17, ; Sion, CH-1951 Valais Switzerland
                [4 ]GRID grid.133342.4, ISNI 0000 0004 1936 9676, Microsoft Station Q, University of California, ; Santa Barbara, California 93106-6105 USA
                [5 ]GRID grid.7400.3, ISNI 0000 0004 1937 0650, Department of Chemistry, , University of Zürich, ; Zürich, Switzerland
                [6 ]GRID grid.6441.7, ISNI 0000 0001 2243 2806, Vilnius University Institute of Biotechnology, Saulėtekio al. 7, ; LT-10257 Vilnius, Lithuania
                [7 ]GRID grid.38142.3c, ISNI 000000041936754X, John A. Paulson School of Engineering and Applied Sciences, , Harvard University, ; Cambridge, Massachusetts 02138 United States
                [8 ]GRID grid.420831.c, ISNI 0000 0004 0529 6285, Robert Bosch LLC, Research and Technology Center North America, 255 Main St, ; Cambridge, Massachusetts 02142 USA
                Author information
                http://orcid.org/0000-0001-5845-8880
                http://orcid.org/0000-0002-6069-5241
                http://orcid.org/0000-0001-6902-1289
                http://orcid.org/0000-0002-1524-5903
                http://orcid.org/0000-0002-1387-5717
                http://orcid.org/0000-0002-7496-3075
                http://orcid.org/0000-0002-2547-155X
                http://orcid.org/0000-0003-4962-2495
                http://orcid.org/0000-0002-7471-3316
                http://orcid.org/0000-0002-7731-6236
                http://orcid.org/0000-0002-1433-1017
                http://orcid.org/0000-0002-9764-0199
                http://orcid.org/0000-0002-3583-4377
                Article
                638
                10.1038/s41597-020-00638-4
                7479590
                31896794
                1625b05e-c60d-4682-a8f5-fd585877c1bc
                © The Author(s) 2020

                Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

                History
                : 24 March 2020
                : 19 August 2020
                Funding
                Funded by: This work is supported by the MARVEL National Centre for Competency in Research funded by the Swiss National Science Foundation (grant agreement ID~51NF40-182892), the European Centre of Excellence MaX ``Materials design at the Exascale' (grant no.~824143) and by the Swiss Platform for Advanced Scientific Computing (PASC).
                Funded by: This work is supported by the MARVEL National Centre for Competency in Research funded by the Swiss National Science Foundation (grant agreement ID~51NF40-182892), the European Centre of Excellence MaX ``Materials design at the Exascale' (grant no.~824143) and by the Swiss Platform for Advanced Scientific Computing (PASC).
                Funded by: This work is supported by the MARVEL National Centre for Competency in Research funded by the Swiss National Science Foundation (grant agreement ID~51NF40-182892), the European Centre of Excellence MaX ``Materials design at the Exascale' (grant no.~824143) and by the Swiss Platform for Advanced Scientific Computing (PASC).
                Funded by: This work is supported by the MARVEL National Centre for Competency in Research funded by the Swiss National Science Foundation (grant agreement ID~51NF40-182892), the European Centre of Excellence MaX ``Materials design at the Exascale' (grant no.~824143) and by the Swiss Platform for Advanced Scientific Computing (PASC).
                Funded by: This work is supported by the MARVEL National Centre for Competency in Research funded by the Swiss National Science Foundation (grant agreement ID~51NF40-182892), the European Centre of Excellence MaX ``Materials design at the Exascale' (grant no.~824143) and by the Swiss Platform for Advanced Scientific Computing (PASC).
                Funded by: This work is supported by the MARVEL National Centre for Competency in Research funded by the Swiss National Science Foundation (grant agreement ID~51NF40-182892), the European Centre of Excellence MaX ``Materials design at the Exascale' (grant no.~824143) and by the Swiss Platform for Advanced Scientific Computing (PASC).
                Funded by: This work is supported by the MARVEL National Centre for Competency in Research funded by the Swiss National Science Foundation (grant agreement ID~51NF40-182892), the European Centre of Excellence MaX ``Materials design at the Exascale' (grant no.~824143) and by the Swiss Platform for Advanced Scientific Computing (PASC).
                Funded by: This work is supported by the MARVEL National Centre for Competency in Research funded by the Swiss National Science Foundation (grant agreement ID~51NF40-182892), the European Centre of Excellence MaX ``Materials design at the Exascale' (grant no.~824143) and by the Swiss Platform for Advanced Scientific Computing (PASC).
                Funded by: This work is supported by the MARVEL National Centre for Competency in Research funded by the Swiss National Science Foundation (grant agreement ID~51NF40-182892), the European Centre of Excellence MaX ``Materials design at the Exascale' (grant no.~824143) and by the Swiss Platform for Advanced Scientific Computing (PASC).
                Funded by: This work is supported by the MARVEL National Centre for Competency in Research funded by the Swiss National Science Foundation (grant agreement ID~51NF40-182892), the European Centre of Excellence MaX ``Materials design at the Exascale' (grant no.~824143) and by the Swiss Platform for Advanced Scientific Computing (PASC).
                Funded by: This work is supported by the MARVEL National Centre for Competency in Research funded by the Swiss National Science Foundation (grant agreement ID~51NF40-182892), the European Centre of Excellence MaX ``Materials design at the Exascale' (grant no.~824143) and by the Swiss Platform for Advanced Scientific Computing (PASC).
                Funded by: This work is supported by the MARVEL National Centre for Competency in Research funded by the Swiss National Science Foundation (grant agreement ID~51NF40-182892), the European Centre of Excellence MaX ``Materials design at the Exascale' (grant no.~824143) and by the Swiss Platform for Advanced Scientific Computing (PASC).
                Funded by: This work is supported by the MARVEL National Centre for Competency in Research funded by the Swiss National Science Foundation (grant agreement ID~51NF40-182892), the European Centre of Excellence MaX ``Materials design at the Exascale' (grant no.~824143) and by the Swiss Platform for Advanced Scientific Computing (PASC).
                Funded by: This work is supported by the MARVEL National Centre for Competency in Research funded by the Swiss National Science Foundation (grant agreement ID~51NF40-182892), the European Centre of Excellence MaX ``Materials design at the Exascale' (grant no.~824143) and by the Swiss Platform for Advanced Scientific Computing (PASC).
                Funded by: This work is supported by the MARVEL National Centre for Competency in Research funded by the Swiss National Science Foundation (grant agreement ID~51NF40-182892), the European Centre of Excellence MaX ``Materials design at the Exascale' (grant no.~824143) and by the Swiss Platform for Advanced Scientific Computing (PASC).
                Funded by: This work is supported by the MARVEL National Centre for Competency in Research funded by the Swiss National Science Foundation (grant agreement ID~51NF40-182892), the European Centre of Excellence MaX ``Materials design at the Exascale' (grant no.~824143) and by the Swiss Platform for Advanced Scientific Computing (PASC).
                Funded by: This work is supported by the MARVEL National Centre for Competency in Research funded by the Swiss National Science Foundation (grant agreement ID~51NF40-182892), the European Centre of Excellence MaX ``Materials design at the Exascale' (grant no.~824143) and by the Swiss Platform for Advanced Scientific Computing (PASC).
                Categories
                Article
                Custom metadata
                © The Author(s) 2020

                computational methods,research management
                computational methods, research management

                Comments

                Comment on this article