How can I load a Python module given its full path? Note that the file can be anywhere in the filesystem, as it is a configuration option.

python configuration python-import python-module

shareimprove this question

edited Feb 16 '14 at 15:36

ferkulat

1,86921630

asked Sep 15 '08 at 22:30

derfred

4,90231720

add a comment

23 Answers

active oldest votes

up vote

885

down vote

accepted

For Python 3.5+ use:

import importlib.util

spec = importlib.util.spec_from_file_location("module.name", "/path/to/file.py")

foo = importlib.util.module_from_spec(spec)

spec.loader.exec_module(foo)

foo.MyClass()

For Python 3.3 and 3.4 use:

from importlib.machinery import SourceFileLoader

foo = SourceFileLoader("module.name", "/path/to/file.py").load_module()

foo.MyClass()

(Although this has been deprecated in Python 3.4.)

Python 2 use:

import imp

foo = imp.load_source('module.name', '/path/to/file.py')

foo.MyClass()

There are equivalent convenience functions for compiled Python files and DLLs.

See also. http://bugs.python.org/issue21436.

shareimprove this answer

edited Dec 7 '15 at 14:31

answered Sep 15 '08 at 22:41

Sebastian Rittau

10.8k31821

30

If I knew the namespace - 'module.name' - I would already use __import__. – Sridhar Ratnakumar Aug 10 '09 at 21:54

43

@SridharRatnakumar the value of the first argument of imp.load_source only sets the .__name__ of the returned module. it doesn't effect loading. – Dan D. Dec 14 '11 at 4:51

14

@DanD. — the first argument of imp.load_source() determines the key of the new entry created in the sys.modules dictionary, so the first argument does indeed affect loading. – Brandon Rhodes Apr 21 '13 at 16:32

8

The imp module is deprecated since version 3.4: The imp package is pending deprecation in favor of importlib. – Chiel ten Brinke Dec 8 '13 at 11:20

85

One might think that python imports are getting more and more complicated with each new version. – AXO Mar 8 '16 at 13:32

show 29 more comments

up vote

302

down vote

The advantage of adding a path to sys.path (over using imp) is that it simplifies things when importing more than one module from a single package. For example:

import sys

# the mock-0.3.1 dir contains testcase.py, testutils.py & mock.py

sys.path.append('/foo/bar/mock-0.3.1')

from testcase import TestCase

from testutils import RunTests

from mock import Mock, sentinel, patch

shareimprove this answer

answered Sep 24 '08 at 19:36

Daryl Spitzer

47k58140159

5

How do we use sys.path.append to point to a single python file instead of a directory? – Phani Jan 13 '14 at 17:46

14

:-) Perhaps your question would be better suited as a StackOverflow question, not a comment on an answer. – Daryl Spitzer Mar 6 '15 at 0:12

2

To all people who were trying to include a file to their path... by definition "the shell path is a colon delimited list of directories". I'm relatively new to python, but the python path also follows the unix design principle from what I have seen. Please correct me if I am wrong. – Michael Baptist Apr 15 '15 at 5:37

2

The python path can contain zip archives, "eggs" (a complex kind of zip archives), etc. Modules can be imported out of them. So the path elements are indeed containers of files, but they are not necessarily directories. – alexis Apr 30 '15 at 21:21

6

Beware of the fact that Python caches import statements. In the rare case that you have two different folders sharing a single class name (classX), the approach of adding a path to sys.path, importing classX, removing the path and repeating for the reamaining paths won't work. Python will always load the class from the first path from its cache. In my case I aimed at creating a plugin system where all plugins implement a specific classX. I ended up using SourceFileLoader, note that its deprecation is controversial. – ComFreek Jul 3 '15 at 17:18

show 3 more comments

up vote

17

down vote

You can also do something like this and add the directory that the configuration file is sitting in to the Python load path, and then just do a normal import, assuming you know the name of the file in advance, in this case "config".

Messy, but it works.

configfile = '~/config.py'

import os

import sys

sys.path.append(os.path.dirname(os.path.expanduser(configfile)))

import config

shareimprove this answer

edited Jan 17 '14 at 22:38

Peter Mortensen

12.7k1982110

answered Sep 15 '08 at 22:44

ctcherry

23.2k44764

add a comment

up vote

16

down vote

You can use the

load_source(module_name, path_to_file)

method from imp module.

shareimprove this answer

edited Nov 11 '14 at 7:07

twasbrillig

5,44842450

answered Sep 15 '08 at 22:41

zuber

2,47321817

... and imp.load_dynamic(module_name, path_to_file) for DLLs – HEKTO Dec 16 '15 at 19:03

17

heads up that imp is deprecated now. – t1m0 Apr 6 '16 at 18:11

add a comment

up vote

12

down vote

It sounds like you don't want to specifically import the configuration file (which has a whole lot of side effects and additional complications involved), you just want to run it, and be able to access the resulting namespace. The standard library provides an API specifically for that in the form of runpy.run_path:

from runpy import run_path

settings = run_path("/path/to/file.py")

That interface is available in Python 2.7 and Python 3.2+

shareimprove this answer

answered May 20 '16 at 6:52

ncoghlan

24.8k65261

add a comment

up vote

10

down vote

def import_file(full_path_to_module):

try:

import os

module_dir, module_file = os.path.split(full_path_to_module)

module_name, module_ext = os.path.splitext(module_file)

save_cwd = os.getcwd()

os.chdir(module_dir)

module_obj = __import__(module_name)

module_obj.__file__ = full_path_to_module

globals()[module_name] = module_obj

os.chdir(save_cwd)

except:

raise ImportError

import_file('/home/somebody/somemodule.py')

shareimprove this answer

answered Sep 16 '08 at 1:43

Chris Calloway

1,0301712

32

Why write 14 lines of buggy code when this is already addressed by the standard library? You haven't done error checking on format or content of full_path_to_module or the os.whatever operations; and using a catch-all except: clause is rarely a good idea. – Chris Johnson Jun 7 '13 at 19:17

You should use more "try-finally"s in here. E.g. save_cwd = os.getcwd() try: … finally: os.chdir(save_cwd) – kay Sep 21 '14 at 1:33

9

@ChrisJohnson this is already addressed by the standard library yeah, but python has nasty habit of not being backward-compatible... as the checked answer says there're 2 different ways before and after 3.3. In that case I'd rather like to write my own universal function than check version on the fly. And yes, maybe this code isn't too well error-protected, but it shows an idea (which is os.chdir(), I haven't though about it), basing on which I can write a better code. Hence +1. – Sushi271 May 15 '15 at 10:27

add a comment

up vote

9

down vote

Do you mean load or import?

You can manipulate the sys.path list specify the path to your module, then import your module. For example, given a module at:

/foo/bar.py

You could do:

import sys

sys.path[0:0] = '/foo' # puts the /foo directory at the start of your path

import bar

shareimprove this answer

edited Jul 9 '10 at 5:30

Eric Schoonover

26.8k37139187

answered Sep 15 '08 at 22:46

Wheat

61237

@Wheat Why sys.path[0:0] instead of sys.path[0]? – user618677 Jan 9 '12 at 6:56

4

B/c sys.path[0] = xy overwrites the first path item while path[0:0] =xy is equivalent to path.insert(0, xy) – dom0 Nov 15 '12 at 14:16

1

hm the path.insert worked for me but the [0:0] trick did not. – jsh Sep 30 '13 at 3:18

8

sys.path[0:0] = ['/foo'] – Kevin Edwards Apr 1 '15 at 17:00

add a comment

up vote

8

down vote

I believe you can use imp.find_module() and imp.load_module() to load the specified module. You'll need to split the module name off of the path, i.e. if you wanted to load /home/mypath/mymodule.py you'd need to do:

imp.find_module('mymodule', '/home/mypath/')

...but that should get the job done.

shareimprove this answer

edited Jan 2 '15 at 11:08

Mathieu Rodic

4,36512938

answered Sep 15 '08 at 22:37

Matt

1313

add a comment

up vote

7

down vote

Here is some code that works in all Python versions, from 2.7-3.5 and probably even others.

config_file = "/tmp/config.py"

with open(config_file) as f:

code = compile(f.read(), config_file, 'exec')

exec(code, globals(), locals())

I tested it. It may be ugly but so far is the only one that works in all versions.

shareimprove this answer

answered Jun 3 '16 at 10:04

sorin

68.6k107346541

This answer worked for me where load_source did not because it imports the script and provides the script access to the modules and globals at the time of importing. – Klik Nov 22 '17 at 19:13

add a comment

up vote

5

down vote

I have come up with a slightly modified version of @SebastianRittau's wonderful answer (for Python > 3.4 I think), which will allow you to load a file with any extension as a module using spec_from_loader instead of spec_from_file_location:

from importlib.util import spec_from_loader, module_from_spec

from importlib.machinery import SourceFileLoader

spec = spec_from_loader("module.name", SourceFileLoader("module.name", "/path/to/file.py"))

mod = module_from_spec(spec)

spec.loader.exec_module(mod)

The advantage of encoding the path in an explicit SourceFileLoader is that the machinery will not try to figure out the type of the file from the extension. This means that you can load something like a .txt file using this method, but you could not do it with spec_from_file_location without specifying the loader because .txt is not in importlib.machinery.SOURCE_SUFFIXES.

shareimprove this answer

edited Nov 9 '17 at 9:27

Delgan

8,32733768

answered Apr 25 '17 at 5:45

Mad Physicist

25.5k135377

add a comment

up vote

3

down vote

This should work

path = os.path.join('./path/to/folder/with/py/files', '*.py')

for infile in glob.glob(path):

basename = os.path.basename(infile)

basename_without_extension = basename[:-3]

# http://docs.python.org/library/imp.html?highlight=imp#module-imp

imp.load_source(basename_without_extension, infile)

shareimprove this answer

edited Jan 4 '12 at 4:28

joran

128k17302356

answered Jan 4 '12 at 2:17

Hengjie

3,3752332

4

A more general way to cut the extension out is: name, ext = os.path.splitext(os.path.basename(infile)). Your method works because the previous restriction to .py extension. Also, you should probably import the module to some variable/dictionary entry. – ReneSac Dec 6 '12 at 13:16

add a comment

up vote

3

down vote

This area of Python 3.4 seems to be extremely tortuous to understand! However with a bit of hacking using the code from Chris Calloway as a start I managed to get something working. Here's the basic function.

def import_module_from_file(full_path_to_module):

"""

Import a module given the full path/filename of the .py file

Python 3.4

"""

module = None

try:

# Get module name and path from full path

module_dir, module_file = os.path.split(full_path_to_module)

module_name, module_ext = os.path.splitext(module_file)

# Get module "spec" from filename

spec = importlib.util.spec_from_file_location(module_name,full_path_to_module)

module = spec.loader.load_module()

except Exception as ec:

# Simple error printing

# Insert "sophisticated" stuff here

print(ec)

finally:

return module

This appears to use non-deprecated modules from Python 3.4. I don't pretend to understand why, but it seems to work from within a program. I found Chris' solution worked on the command line but not from inside a program.

shareimprove this answer

answered Apr 12 '15 at 12:22

Redlegjed

311

add a comment

up vote

3

down vote

I'm not saying that it is better, but for the sake of completeness, I wanted to suggest the exec function, available in both python 2 and 3. exec allows you to execute arbitrary code in either the global scope, or in an internal scope, provided as a dictionary.

For example, if you have a module stored in "/path/to/module" with the function foo(), you could run it by doing the following:

module = dict()

with open("/path/to/module") as f:

exec(f.read(), module)

module['foo']()

This makes it a bit more explicit that you're loading code dynamically, and grants you some additional power, such as the ability to provide custom builtins.

And if having access through attributes, instead of keys is important to you, you can design a custom dict class for the globals, that provides such access, e.g.:

class MyModuleClass(dict):

def __getattr__(self, name):

return self.__getitem__(name)

shareimprove this answer

answered Jun 2 '15 at 19:57

yoniLavi

1,2681421

1

execfile(), also – crowder Jun 20 '15 at 4:13

add a comment

up vote

3

down vote

To import a module from a given filename, you can temporarily extend the path, and restore the system path in the finally block reference:

filename = "directory/module.py"

directory, module_name = os.path.split(filename)

module_name = os.path.splitext(module_name)[0]

path = list(sys.path)

sys.path.insert(0, directory)

try:

module = __import__(module_name)

finally:

sys.path[:] = path # restore

shareimprove this answer

answered Oct 2 '15 at 11:14

Peter Zhu

689921

add a comment

up vote

2

down vote

Import package modules at runtime (Python recipe)

http://code.activestate.com/recipes/223972/

###################

## #

## classloader.py #

## #

###################

import sys, types

def _get_mod(modulePath):

try:

aMod = sys.modules[modulePath]

if not isinstance(aMod, types.ModuleType):

raise KeyError

except KeyError:

# The last [''] is very important!

aMod = __import__(modulePath, globals(), locals(), [''])

sys.modules[modulePath] = aMod

return aMod

def _get_func(fullFuncName):

"""Retrieve a function object from a full dotted-package name."""

# Parse out the path, module, and function

lastDot = fullFuncName.rfind(u".")

funcName = fullFuncName[lastDot + 1:]

modPath = fullFuncName[:lastDot]

aMod = _get_mod(modPath)

aFunc = getattr(aMod, funcName)

# Assert that the function is a *callable* attribute.

assert callable(aFunc), u"%s is not callable." % fullFuncName

# Return a reference to the function itself,

# not the results of the function.

return aFunc

def _get_class(fullClassName, parentClass=None):

"""Load a module and retrieve a class (NOT an instance).

If the parentClass is supplied, className must be of parentClass

or a subclass of parentClass (or None is returned).

"""

aClass = _get_func(fullClassName)

# Assert that the class is a subclass of parentClass.

if parentClass is not None:

if not issubclass(aClass, parentClass):

raise TypeError(u"%s is not a subclass of %s" %

(fullClassName, parentClass))

# Return a reference to the class itself, not an instantiated object.

return aClass

######################

## Usage ##

######################

class StorageManager: pass

class StorageManagerMySQL(StorageManager): pass

def storage_object(aFullClassName, allOptions={}):

aStoreClass = _get_class(aFullClassName, StorageManager)

return aStoreClass(allOptions)

shareimprove this answer

edited Dec 22 '13 at 2:17

Eric Leschinski

77.8k35306255

answered Sep 15 '08 at 22:43

user10370

371

add a comment

up vote

2

down vote

You can use the pkgutil module (specifically the walk_packages method) to get a list of the packages in the current directory. From there it's trivial to use the importlib machinery to import the modules you want:

import pkgutil

import importlib

packages = pkgutil.walk_packages(path='.')

for importer, name, is_package in packages:

mod = importlib.import_module(name)

# do whatever you want with module now, it's been imported!

shareimprove this answer

edited Jan 2 '15 at 11:04

Mathieu Rodic

4,36512938

answered Sep 13 '14 at 19:57

bob_twinkles

1869

add a comment

up vote

2

down vote

In Linux, adding a symbolic link in the directory your python script is located works.

ie:

ln -s /absolute/path/to/module/module.py /absolute/path/to/script/module.py

python will create /absolute/path/to/script/module.pyc and will update it if you change the contents of /absolute/path/to/module/module.py

then include the following in mypythonscript.py

from module import *

shareimprove this answer

edited Apr 11 at 17:21

Micah Smith

2,3241421

answered Nov 18 '14 at 13:06

user2760152

234

1

This is the hack I used, and it has caused me some problems. One of the more painful ones was that IDEA has an issue where it doesn't pickup altered code from within the link, but yet attempts to save what it thinks is there. A race condition where the last to save is what sticks... I lost a decent amount of work because of this. – Gripp Jun 16 '15 at 23:23

@Gripp not sure if I am understanding your issue, but I frequently (almost exclusively) edit my scripts on a remote server from my desktop via SFTP with a client like CyberDuck, and in that case as well it is a bad idea to try and edit the symlinked file, instead its much safer to edit the original file. You can catch some of these issues by using git and checking your git status to verify that your changes to the script are actually making it back to the source document and not getting lost in the ether. – user5359531 Aug 1 '17 at 19:39

add a comment

up vote

1

down vote

I made a package that uses imp for you. I call it import_file and this is how it's used:

>>>from import_file import import_file

>>>mylib = import_file('c:\\mylib.py')

>>>another = import_file('relative_subdir/another.py')

You can get it at:

http://pypi.python.org/pypi/import_file

or at

http://code.google.com/p/import-file/

shareimprove this answer

answered Jun 8 '11 at 19:41

ubershmekel

4,95524263

1

os.chdir ? (minimal characters to approve comment). – ychaouche Oct 14 '12 at 10:46

add a comment

up vote

1

down vote

quite simple way: suppose you want import file with relative path ../../MyLibs/pyfunc.py

libPath = '../../MyLibs'

import sys

if not libPath in sys.path: sys.path.append(libPath)

import pyfunc as pf

But if you make it without a guard you can finally get a very long path

shareimprove this answer

answered Jan 26 at 4:52

Andrei Keino

512

add a comment

up vote

0

down vote

Adding this to the list of answers as I couldn't find anything that worked. This will allow imports of compiled (pyd) python modules in 3.4:

import sys

import importlib.machinery

def load_module(name, filename):

# If the Loader finds the module name in this list it will use

# module_name.__file__ instead so we need to delete it here

if name in sys.modules:

del sys.modules[name]

loader = importlib.machinery.ExtensionFileLoader(name, filename)

module = loader.load_module()

locals()[name] = module

globals()[name] = module

load_module('something', r'C:\Path\To\something.pyd')

something.do_something()

shareimprove this answer

answered Jan 10 at 15:58

David

310114

add a comment

up vote

0

down vote

If your top-level module is not a file but is packaged as a directory with __init__.py, then the accepted solution almost works, but not quite. In Python 3.5+ the following code is needed (note the added line that begins with 'sys.modules'):

MODULE_PATH = "/path/to/your/module/__init__.py"

MODULE_NAME = "mymodule"

spec = importlib.util.spec_from_file_location("mymodule", MODULE_PATH)

module = importlib.util.module_from_spec(spec)

sys.modules[spec.name] = module

spec.loader.exec_module(module)

Without this line, when exec_module is executed, it tries to bind relative imports in your top level __init__.py to the top level module name -- in this case "mymodule". But "mymodule" isn't loaded yet so you'll get the error "SystemError: Parent module 'mymodule' not loaded, cannot perform relative import". So you need to bind the name before you load it. The reason for this is the fundamental invariant of the relative import system: "The invariant holding is that if you have sys.modules['spam'] and sys.modules['spam.foo'] (as you would after the above import), the latter must appear as the foo attribute of the former" as discussed here.

shareimprove this answer

answered May 17 at 15:23

Sam Grondahl

93611023

add a comment

up vote

0

down vote

A simple solution using importlib instead of the imp package (tested for Python 2.7, although it should work for Python 3 too):

import importlib

dirname, basename = os.path.split(pyfilepath) # pyfilepath: /my/path/mymodule.py

sys.path.append(dirname) # only directories should be added to PYTHONPATH

module_name = os.path.splitext(basename)[0] # /my/path/mymodule.py --> mymodule

module = importlib.import_module(module_name) # name space of defined module (otherwise we would literally look for "module_name")

Now you can directly use the namespace of the imported module, like this:

a = module.myvar

b = module.myfunc(a)

shareimprove this answer

answered May 24 at 12:07

Ataxias

8116