Skip to main content
  • Home
  • Development
  • Documentation
  • Donate
  • Operational login
  • Browse the archive

swh logo
SoftwareHeritage
Software
Heritage
Archive
Features
  • Search

  • Downloads

  • Save code now

  • Add forge now

  • Help

  • dfc4d0e
  • /
  • batchcombine.py
Raw File Download
Permalinks

To reference or cite the objects present in the Software Heritage archive, permalinks based on SoftWare Hash IDentifiers (SWHIDs) must be used.
Select below a type of object currently browsed in order to display its associated SWHID and permalink.

  • content
  • directory
content badge Iframe embedding
swh:1:cnt:eb7434982e8876423d86e499e2161174cb264b30
directory badge Iframe embedding
swh:1:dir:dfc4d0e52c7919b391edb34f84a3cf44572f4a87
Citations

This interface enables to generate software citations, provided that the root directory of browsed objects contains a citation.cff or codemeta.json file.
Select below a type of object currently browsed in order to generate citations for them.

  • content
  • directory
Generate software citation in BibTex format (requires biblatex-software package)
Generating citation ...
Generate software citation in BibTex format (requires biblatex-software package)
Generating citation ...
batchcombine.py
#!/usr/bin/env python
#
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#
#<LicenseText>
#
# CitcomS.py by Eh Tan, Eun-seo Choi, and Pururav Thoutireddy.
# Copyright (C) 2002-2005, California Institute of Technology.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
#
#</LicenseText>
#
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#

'''
Paste and combine Citcom data

Usage: batchcombine.py <machinefile | node-list> datadir datafile timestep nodex nodey nodez ncap nprocx nprocy nprocz [fields]
'''



def machinefile2nodes(machinefile, totalnodes):
    nodelist = machinefile2nodelist(machinefile, totalnodes)
    nodes = nodelist2nodes(nodelist)
    return nodes



def machinefile2nodelist(machinefile, totalnodes):
    '''Read the machinefile to get a list of machine names. If machinefile
    is not readable, treat it as a string containing the machine names.
    Return the list of machine names. The length of the list is totalnodes.
    '''
    try:
        nodelist = file(machinefile).readlines()
    except IOError:
        nodelist = machinefile.split()

    # check the length of nodelist
    n = len(nodelist)
    if not n == totalnodes:
        print 'WARNING: length of machinefile does not match number of processors, try to duplicate machinefile...'
        if (totalnodes > n) and ((totalnodes % n) == 0):
            # try to match number of processors by duplicating nodelist
            nodelist *= (totalnodes / n)
        else:
            raise ValueError, 'incorrect machinefile size'

    return nodelist



def nodelist2nodes(nodelist):
    # generate a string of machine names
    nodes = ' '.join([x.strip() for x in nodelist])

    return nodes



def batchpaste(datadir, datafile, fields, timestep, nodes):
    from socket import gethostname
    hostname = gethostname()

    import os
    cwd = os.getcwd()

    for rank, node in enumerate(nodes):
        if node == 'localhost' or node == hostname:
            # local paste
            import pasteCitcomData
            pasteCitcomData.run(datadir, datafile, fields, rank, timestep, cwd)

        else:
            # remote paste

            # replace 'rsh' with 'ssh' if necessary
            remote_shell = 'rsh'

            cmd = '%(remote_shell)s %(node)s pasteCitcomData.py %(datadir)s %(datafile)s %(fields)s %(rank)d %(timestep)d %(cwd)s' % vars()
            os.system(cmd)

    return



def batchcombine(nodes, datadir, datafile, timestep, nodex, nodey, nodez,
                 ncap, nprocx, nprocy, nprocz, fields):
    # paste
    batchpaste(datadir, datafile, fields, timestep, nodes)


    # combine
    import combine
    combine.combine(datafile, fields, timestep,
                    nodex, nodey, nodez,
                    ncap, nprocx, nprocy, nprocz)


    # delete pasted files
    import glob
    filenames = glob.glob('%(datafile)s.*.%(timestep)d.pasted' % vars())

    import os
    for filename in filenames:
        os.remove(filename)


    # create .general file
    import dxgeneral
    combined_files = []
    for cap in range(ncap):
        combined_files.append('%(datafile)s.cap%(cap)02d.%(timestep)d' % vars())

    dxgeneral.write(fields, combined_files)

    return



if __name__ == '__main__':

    import sys

    if not (12 <= len(sys.argv) <= 13):
        print __doc__
        sys.exit(1)

    machinefile = sys.argv[1]
    datadir = sys.argv[2]
    datafile = sys.argv[3]
    timestep = int(sys.argv[4])
    nodex = int(sys.argv[5])
    nodey = int(sys.argv[6])
    nodez = int(sys.argv[7])
    ncap = int(sys.argv[8])
    nprocx = int(sys.argv[9])
    nprocy = int(sys.argv[10])
    nprocz = int(sys.argv[11])
    if len(sys.argv) == 13:
        fields = sys.argv[12]
    else:
        fields = 'coord,velo,visc'

    totalnodes = nprocx * nprocy * nprocz * ncap
    nodelist = machinefile2nodelist(machinefile, totalnodes)

    batchcombine(nodelist, datadir, datafile, timestep, nodex, nodey, nodez,
                 ncap, nprocx, nprocy, nprocz, fields)



# version
# $Id$

# End of file

Software Heritage — Copyright (C) 2015–2025, The Software Heritage developers. License: GNU AGPLv3+.
The source code of Software Heritage itself is available on our development forge.
The source code files archived by Software Heritage are available under their own copyright and licenses.
Terms of use: Archive access, API— Contact— JavaScript license information— Web API

back to top