Python package settings : setup. py, a Fortran for custom processing wrapping

Python package settings : setup. py, a Fortran for custom processing wrapping … here is a solution to the problem.

Python package settings : setup. py, a Fortran for custom processing wrapping

I

have a python package that I want to distribute. I have the installation package to download tarball, unzip it and install it :

python setup.py install

Works great.

I also want to upload the package to PyPi and install it using pip.

However, the package contains a f2py wrapper of fortran that needs to be compiled at build time and the generated .so file moved to the final installation folder. I’m confused about how to use :

python3 setup.py sdist

The second is:

pip3 install pkg_name_here.tar.gz

The reason is when I run

python3 setup.py sdist

A custom command is running, part of which is an attempt to move the compiled *so file to the installation folder, which has not yet been created. The example of the code outline I used in this case here

from setuptools.command.install import install
from setuptools.command.develop import develop
from setuptools.command.egg_info import egg_info

'''
BEGIN CUSTOM INSTALL COMMANDS
These classes are used to hook into setup.py's install process. Depending on the context:
$ pip install my-package

Can yield `setup.py install`, `setup.py egg_info`, or `setup.py develop`
'''

def custom_command():
    import sys
    if sys.platform in ['darwin', 'linux']:
        os.system('./custom_command.sh')

class CustomInstallCommand(install):
    def run(self):
        install.run(self)
        custom_command()

class CustomDevelopCommand(develop):
    def run(self):
        develop.run(self)
        custom_command()

class CustomEggInfoCommand(egg_info):
    def run(self):
        egg_info.run(self)
        custom_command()

'''
END CUSTOM INSTALL COMMANDS 
'''

setup(
    ...
    cmdclass={
        'install': CustomInstallCommand,
        'develop': CustomDevelopCommand,
        'egg_info': CustomEggInfoCommand,
    },
    ...
)

In my case, custom_command() compiles and wraps fortran and copies the lib file to the installation folder.

I wonder if there is a way to just run these custom commands during installation with pip? That is, avoid running custom_command() during packaging and only during installation.

Update

I made some progress at Pierre de Buyl’s advice, but it still doesn’t work.

setup.py file currently looks like this:

 def setup_f90_ext(parent_package='',top_path=''):
    from numpy.distutils.misc_util import Configuration
    from os.path import join

config = Configuration('',parent_package,top_path)

tort_src = [join('PackageName/','tort.f90')]
    config.add_library('tort', sources=tort_src,
                          extra_f90_compile_args=['-fopenmp -lgomp -O3'],
                          extra_link_args=['-lgomp'])

sources = [join('PackageName','f90wrap_tort.f90')]

config.add_extension(name='',
                          sources=sources,
                          extra_f90_compile_args=['-fopenmp -lgomp -O3'],
                          libraries=['tort'],
                          extra_link_args=['-lgomp'],
                          include_dirs=['build/temp*/'])

return config

if __name__ == '__main__':

from numpy.distutils.core import setup
    import subprocess
    import os
    import sys

version_file = open(os.getcwd()+'/PackageName/'+ 'VERSION')
    __version__ = version_file.read().strip()

subprocess.call(cmd, shell=True)

config = {'name':'PackageName',
              'version':__version__,
              'project_description':'Package description',
              'description':'Description',
              'long_description': open('README.txt').read(),#read('README.txt'),
}
    config2 = dict(config,**setup_f90_ext(parent_package='PackageName',top_path='').todict())
    setup(**config2)   

where f90wrap_tort.f90 is

the fortran file for f90wrapped and tort.f90 is the original fortran.

If I run the command twice, this file works with python setup.py install

I get the following error the first time I run python setup.py install:

    gfortran:f90: ./PackageName/f90wrap_tort.f90
f951: Warning: Nonexistent include directory ‘build/temp*/’ [-Wmissing-include-dirs]
./PackageName/f90wrap_tort.f90:4:8:

use tort_mod, only: test_node
        1
Fatal Error: Can't open module file ‘tort_mod.mod’ for reading at (1): No such file or directory
compilation terminated.
f951: Warning: Nonexistent include directory ‘build/temp*/’ [-Wmissing-include-dirs]
./PackageName/f90wrap_tort.f90:4:8:

use tort_mod, only: test_node
        1
Fatal Error: Can't open module file ‘tort_mod.mod’ for reading at (1): No such file or directory

The reason I put the include_dirs=['build/temp*/'] parameter in the extension is because I noticed after running python setup.py install that the first time tort_mod was being built and stored there.

What I can’t figure out is how to get the right link so that it’s done in one step.

Who can see what I’m missing?

Solution

After some Google searching, I made the following suggestions:

  1. Use NumPy’s distutils
  2. Use the add_library keyword for normal Fortran files (see here )。 This builds Fortran files as libraries, but does not attempt to interact with them using f2py.
  3. Use f90wrap to pre-build f90 wrappers, include them in your package archive, and specify these files as source files in your extension.

I didn’t test the whole solution because it was pip time-consuming, but that’s what SciPy did with some of their modules, see here .

There is an item in NumPy’s documentation that is exceeded add_ library

Edit 1: After configuring the build with include_dirs=['build/temp.linux-x86_64-2.7']), I got this directory structure on my first build attempt.

build/lib.linux-x86_64-2.7
├── crystal_torture
│   ├── cluster.py
│   ├── dist.f90
│   ├── f90wrap_tort.f90
│   ├── graph.py
│   ├── __init__.py
│   ├── minimal_cluster.py
│   ├── node.py
│   ├── node.pyc
│   ├── pymatgen_doping.py
│   ├── pymatgen_interface.py
│   ├── tort.f90
│   ├── tort.py
│   └── tort.pyc
└── crystal_torture.so

Related Problems and Solutions