Front Matter

ext.html | ext.html | contents.html | Extending and Embedding the Python Interpreter | contents.html
---
# Front Matter
BEOPEN.COM TERMS AND CONDITIONS FOR PYTHON 2.0
BEOPEN PYTHON OPEN SOURCE LICENSE AGREEMENT VERSION 1
1. This LICENSE AGREEMENT is between BeOpen.com (``BeOpen''), having an
office at 160 Saratoga Avenue, Santa Clara, CA 95051, and the
Individual or Organization (``Licensee'') accessing and otherwise
using this software in source or binary form and its associated
documentation (``the Software'').
2. Subject to the terms and conditions of this BeOpen Python License
Agreement, BeOpen hereby grants Licensee a non-exclusive,
royalty-free, world-wide license to reproduce, analyze, test, perform
and/or display publicly, prepare derivative works, distribute, and
otherwise use the Software alone or in any derivative version,
provided, however, that the BeOpen Python License is retained in the
Software, alone or in any derivative version prepared by Licensee.
3. BeOpen is making the Software available to Licensee on an ``AS IS''
basis. BEOPEN MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, BEOPEN MAKES NO AND
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF THE SOFTWARE WILL NOT
INFRINGE ANY THIRD PARTY RIGHTS.
4. BEOPEN SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE
SOFTWARE FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS
AS A RESULT OF USING, MODIFYING OR DISTRIBUTING THE SOFTWARE, OR ANY
DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
5. This License Agreement will automatically terminate upon a material
breach of its terms and conditions.
6. This License Agreement shall be governed by and interpreted in all
respects by the law of the State of California, excluding conflict of
law provisions. Nothing in this License Agreement shall be deemed to
create any relationship of agency, partnership, or joint venture
between BeOpen and Licensee. This License Agreement does not grant
permission to use BeOpen trademarks or trade names in a trademark
sense to endorse or promote products or services of Licensee, or any
third party. As an exception, the ``BeOpen Python'' logos available
at http://www.pythonlabs.com/logos.html may be used according to the
permissions granted on that web page.
7. By copying, installing or otherwise using the software, Licensee
agrees to be bound by the terms and conditions of this License
Agreement.
CNRI OPEN SOURCE LICENSE AGREEMENT
Python 1.6 is made available subject to the terms and conditions in
CNRI's License Agreement. This Agreement together with Python 1.6 may
be located on the Internet using the following unique, persistent
identifier (known as a handle): 1895.22/1012. This Agreement may also
be obtained from a proxy server on the Internet using the following
URL: http://hdl.handle.net/1895.22/1012.
CWI PERMISSIONS STATEMENT AND DISCLAIMER
Copyright © 1991 - 1995, Stichting Mathematisch Centrum
Amsterdam, The Netherlands. All rights reserved.
Permission to use, copy, modify, and distribute this software and its
documentation for any purpose and without fee is hereby granted,
provided that the above copyright notice appear in all copies and that
both that copyright notice and this permission notice appear in
supporting documentation, and that the name of Stichting Mathematisch
Centrum or CWI not be used in advertising or publicity pertaining to
distribution of the software without specific, written prior
permission.
STICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO
THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
FITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE
FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
Acknowledgements
The following people have contributed sections to this document: Jim
Fulton, Konrad Hinsen, Chris Phoenix, and Neil Schemenauer.
### Abstract:
Python is an interpreted, object-oriented programming language. This
document describes how to write modules in C or C++ to extend the
Python interpreter with new modules. Those modules can define new
functions but also new object types and their methods. The document
also describes how to embed the Python interpreter in another
application, for use as an extension language. Finally, it shows how
to compile and link extension modules so that they can be loaded
dynamically (at run time) into the interpreter, if the underlying
operating system supports this feature.
This document assumes basic knowledge about Python. For an informal
introduction to the language, see the
Python Tutorial (../tut/tut.html). The
Python Reference Manual (../ref/ref.html) gives a more
formal definition of the language. The
Python Library Reference (../lib/lib.html) documents the
existing object types, functions and modules (both built-in and
written in Python) that give the language its wide application range.
For a detailed description of the whole Python/C API, see the separate
Python/C API Reference Manual (../api/api.html).


../index.html | front.html | Extending and Embedding the Python Interpreter | contents.html
---
# Extending and Embedding the Python Interpreter
Guido van Rossum
Fred L. Drake, Jr., editor
BeOpen PythonLabs
E-mail: python-docs@python.org
October 16, 2000
Release 2.0


1. Extending Python with C or C++

contents.html | ext.html | simpleExample.html | Extending and Embedding the Python Interpreter | contents.html
---
# 1. Extending Python with C or C++
It is quite easy to add new built-in modules to Python, if you know
how to program in C. Such extension modules can do two things
that can't be done directly in Python: they can implement new built-in
object types, and they can call C library functions and system calls.
To support extensions, the Python API (Application Programmers
Interface) defines a set of functions, macros and variables that
provide access to most aspects of the Python run-time system. The
Python API is incorporated in a C source file by including the header
`"Python.h"`.
The compilation of an extension module depends on its intended use as
well as on your system setup; details are given in later chapters.


4.2 Linking Requirements

embeddingInCplusplus.html | embedding.html | reporting-bugs.html | Extending and Embedding the Python Interpreter | contents.html
---
# 4.2 Linking Requirements
While the configure script shipped with the Python sources
will correctly build Python to export the symbols needed by
dynamically linked extensions, this is not automatically inherited by
applications which embed the Python library statically, at least on
Unix. This is an issue when the application is linked to the static
runtime library (libpython.a) and needs to load dynamic
extensions (implemented as .so files).
The problem is that some entry points are defined by the Python
runtime solely for extension modules to use. If the embedding
application does not use any of these entry points, some linkers will
not include those entries in the symbol table of the finished
executable. Some additional options are needed to inform the linker
not to remove these symbols.
Determining the right options to use for any given platform can be
quite difficult, but fortunately the Python configuration already has
those values. To retrieve them from an installed Python interpreter,
start an interactive interpreter and have a short session like this:
```text

>>> import distutils.sysconfig
>>> distutils.sysconfig.LINKFORSHARED
'-Xlinker -export-dynamic'
```
The contents of the string presented will be the options that should
be used. If the string is empty, there's no need to add any
additional options. The LINKFORSHARED definition
corresponds to the variable of the same name in Python's top-level
Makefile.


backToExample.html | intro.html | compilation.html | Extending and Embedding the Python Interpreter | contents.html
---
# 1.4 The Module's Method Table and Initialization Function
I promised to show how spam_system() is called from Python
programs. First, we need to list its name and address in a ``method
table'':
```text

static PyMethodDef SpamMethods[] = {
...
{"system", spam_system, METH_VARARGS},
...
{NULL, NULL} /* Sentinel */
};
```
Note the third entry ("METH_VARARGS"). This is a flag telling
the interpreter the calling convention to be used for the C
function. It should normally always be "METH_VARARGS" or
"METH_VARARGS | METH_KEYWORDS"; a value of `0` means that an
obsolete variant of PyArg_ParseTuple() is used.
When using only "METH_VARARGS", the function should expect
the Python-level parameters to be passed in as a tuple acceptable for
parsing via PyArg_ParseTuple(); more information on this
function is provided below.
The METH_KEYWORDS bit may be set in the third field if
keyword arguments should be passed to the function. In this case, the
C function should accept a third "PyObject *" parameter which
will be a dictionary of keywords. Use
PyArg_ParseTupleAndKeywords() to parse the arguments to
such a function.
The method table must be passed to the interpreter in the module's
initialization function. The initialization function must be named
initname(), where name is the name of the
module, and should be the only non-static item defined in
the module file:
```text

void
initspam()
{
(void) Py_InitModule("spam", SpamMethods);
}
```
Note that for C++, this method must be declared `extern "C"`.
When the Python program imports module spam for the first
time, initspam() is called. (See below for comments about
embedding Python.) It calls
Py_InitModule(), which creates a ``module object'' (which
is inserted in the dictionary `sys.modules` under the key
`"spam"`), and inserts built-in function objects into the newly
created module based upon the table (an array of PyMethodDef
structures) that was passed as its second argument.
Py_InitModule() returns a pointer to the module object
that it creates (which is unused here). It aborts with a fatal error
if the module could not be initialized satisfactorily, so the caller
doesn't need to check for errors.
When embedding Python, the initspam() function is not
called automatically unless there's an entry in the
_PyImport_Inittab table. The easiest way to handle this is to
statically initialize your statically-linked modules by directly
calling initspam() after the call to
Py_Initialize() or PyMac_Initialize():
```text

int main(int argc, char **argv)
{
/* Pass argv[0] to the Python interpreter */
Py_SetProgramName(argv[0]);

/* Initialize the Python interpreter. Required. */
Py_Initialize();

/* Add a static module */
initspam();
```
An example may be found in the file Demo/embed/demo.c in the
Python source distribution.
Note: Removing entries from `sys.modules` or importing
compiled modules into multiple interpreters within a process (or
following a fork() without an intervening
exec()) can create problems for some extension modules.
Extension module authors should exercise caution when initializing
internal data structures.
Note also that the reload() function can be used with
extension modules, and will call the module initialization function
(initspam() in the example), but will not load the module
again if it was loaded from a dynamically loadable object file
(.so on Unix, .dll on Windows).
A more substantial example module is included in the Python source
distribution as Modules/xxmodule.c. This file may be used as a
template or simply read as an example. The modulator.py
script included in the source distribution or Windows install provides
a simple graphical user interface for declaring the functions and
objects which a module should implement, and can generate a template
which can be filled in. The script lives in the
Tools/modulator/ directory; see the README file there
for more information.


2.3 Example

module-defn-options.html | building-on-unix.html | distributing.html | Extending and Embedding the Python Interpreter | contents.html
---
# 2.3 Example
Here is a more complicated example from Modules/Setup.in:
```text

GMP=/ufs/guido/src/gmp
mpz mpzmodule.c -I$(GMP) $(GMP)/libgmp.a
```
which could also be written as:
```text

mpz mpzmodule.c -I$(GMP) -L$(GMP) -lgmp
```


2.2 Module Definition Options

custom-interps.html | building-on-unix.html | module-defn-example.html | Extending and Embedding the Python Interpreter | contents.html
---
# 2.2 Module Definition Options
Several compiler options are supported:
Other compiler options can be included (snuck in) by putting them
in variables.
Source files can include files with .c, .C, .cc,
.cpp, .cxx, and .c++ extensions.
Other input files include files with .a, .o, .sl,
and .so extensions.


1.10.4 NULL Pointers

thinIce.html | refcounts.html | cplusplus.html | Extending and Embedding the Python Interpreter | contents.html
---
## 1.10.4 NULL Pointers
In general, functions that take object references as arguments do not
expect you to pass them NULL pointers, and will dump core (or
cause later core dumps) if you do so. Functions that return object
references generally return NULL only to indicate that an
exception occurred. The reason for not testing for NULL
arguments is that functions often pass the objects they receive on to
other function -- if each function were to test for NULL,
there would be a lot of redundant tests and the code would run more
slowly.
It is better to test for NULL only at the ``source'', i.e. when a
pointer that may be NULL is received, e.g. from
malloc() or from a function that may raise an exception.
The macros Py_INCREF() and Py_DECREF()
do not check for NULL pointers -- however, their variants
Py_XINCREF() and Py_XDECREF() do.
The macros for checking for a particular object type
(`Py type _Check()`) don't check for NULL pointers --
again, there is much code that calls several of these in a row to test
an object against various different expected types, and this would
generate redundant tests. There are no variants with NULL
checking.
The C function calling mechanism guarantees that the argument list
passed to C functions (`args` in the examples) is never
NULL -- in fact it guarantees that it is always a tuple.1.4 (#foot676)
It is a severe error to ever let a NULL pointer ``escape'' to
the Python user.


1.10.2 Ownership Rules

refcountsInPython.html | refcounts.html | thinIce.html | Extending and Embedding the Python Interpreter | contents.html
---
## 1.10.2 Ownership Rules
Whenever an object reference is passed into or out of a function, it
is part of the function's interface specification whether ownership is
transferred with the reference or not.
Most functions that return a reference to an object pass on ownership
with the reference. In particular, all functions whose function it is
to create a new object, e.g. PyInt_FromLong() and
Py_BuildValue(), pass ownership to the receiver. Even if in
fact, in some cases, you don't receive a reference to a brand new
object, you still receive ownership of the reference. For instance,
PyInt_FromLong() maintains a cache of popular values and can
return a reference to a cached item.
Many functions that extract objects from other objects also transfer
ownership with the reference, for instance
PyObject_GetAttrString(). The picture is less clear, here,
however, since a few common routines are exceptions:
PyTuple_GetItem(), PyList_GetItem(),
PyDict_GetItem(), and PyDict_GetItemString()
all return references that you borrow from the tuple, list or
dictionary.
The function PyImport_AddModule() also returns a borrowed
reference, even though it may actually create the object it returns:
this is possible because an owned reference to the object is stored in
`sys.modules`.
When you pass an object reference into another function, in general,
the function borrows the reference from you -- if it needs to store
it, it will use Py_INCREF() to become an independent
owner. There are exactly two important exceptions to this rule:
PyTuple_SetItem() and PyList_SetItem(). These
functions take over ownership of the item passed to them -- even if
they fail! (Note that PyDict_SetItem() and friends don't
take over ownership -- they are ``normal.'')
When a C function is called from Python, it borrows references to its
arguments from the caller. The caller owns a reference to the object,
so the borrowed reference's lifetime is guaranteed until the function
returns. Only when such a borrowed reference must be stored or passed
on, it must be turned into an owned reference by calling
Py_INCREF().
The object reference returned from a C function that is called from
Python must be an owned reference -- ownership is tranferred from the
function to its caller.


callingPython.html | intro.html | parseTupleAndKeywords.html | Extending and Embedding the Python Interpreter | contents.html
---
# 1.7 Format Strings for PyArg_ParseTuple()
The PyArg_ParseTuple() function is declared as follows:
```text

int PyArg_ParseTuple(PyObject *arg, char *format, ...);
```
The arg argument must be a tuple object containing an argument
list passed from Python to a C function. The format argument
must be a format string, whose syntax is explained below. The
remaining arguments must be addresses of variables whose type is
determined by the format string. For the conversion to succeed, the
arg object must match the format and the format must be
exhausted.
Note that while PyArg_ParseTuple() checks that the Python
arguments have the required types, it cannot check the validity of the
addresses of C variables passed to the call: if you make mistakes
there, your code will probably crash or at least overwrite random bits
in memory. So be careful!
A format string consists of zero or more ``format units''. A format
unit describes one Python object; it is usually a single character or
a parenthesized sequence of format units. With a few exceptions, a
format unit that is not a parenthesized sequence normally corresponds
to a single address argument to PyArg_ParseTuple(). In the
following description, the quoted form is the format unit; the entry
in (round) parentheses is the Python object type that matches the
format unit; and the entry in [square] brackets is the type of the C
variable(s) whose address should be passed. (Use the "&"operator to pass a variable's address.)
Note that any Python object references which are provided to the
caller are borrowed references; do not decrement their
reference count!
"s" (string or Unicode object) [char *]: Convert a Python string or Unicode object to a C pointer to a
character string. You must not provide storage for the string
itself; a pointer to an existing string is stored into the character
pointer variable whose address you pass. The C string is
null-terminated. The Python string must not contain embedded null
bytes; if it does, a TypeError exception is raised.
Unicode objects are converted to C strings using the default
encoding. If this conversion fails, an UnicodeError is
raised.
"s#" (string, Unicode or any read buffer compatible object)
[char *, int]: This variant on "s" stores into two C variables, the first one a
pointer to a character string, the second one its length. In this
case the Python string may contain embedded null bytes. Unicode
objects pass back a pointer to the default encoded string version of the
object if such a conversion is possible. All other read buffer
compatible objects pass back a reference to the raw internal data
representation.
"z" (string or `None`) [char *]: Like "s", but the Python object may also be `None`, in which
case the C pointer is set to NULL.
"z#" (string or `None` or any read buffer compatible object)
[char *, int]: This is to "s#" as "z" is to "s".
"u" (Unicode object) [Py_UNICODE *]: Convert a Python Unicode object to a C pointer to a null-terminated
buffer of 16-bit Unicode (UTF-16) data. As with "s", there is no need
to provide storage for the Unicode data buffer; a pointer to the
existing Unicode data is stored into the Py_UNICODE pointer variable whose
address you pass.
"u#" (Unicode object) [Py_UNICODE *, int]: This variant on "u" stores into two C variables, the first one
a pointer to a Unicode data buffer, the second one its length.
"es" (string, Unicode object or character buffer compatible
object) [const char *encoding, char **buffer]: This variant on "s" is used for encoding Unicode and objects
convertible to Unicode into a character buffer. It only works for
encoded data without embedded NULL bytes.
The variant reads one C variable and stores into two C variables, the
first one a pointer to an encoding name string (encoding), the
second a pointer to a pointer to a character buffer (**buffer,
the buffer used for storing the encoded data) and the third one a
pointer to an integer (*buffer_length, the buffer length).
The encoding name must map to a registered codec. If set to NULL,
the default encoding is used.
PyArg_ParseTuple() will allocate a buffer of the needed
size using PyMem_NEW(), copy the encoded data into this
buffer and adjust *buffer to reference the newly allocated
storage. The caller is responsible for calling
PyMem_Free() to free the allocated buffer after usage.
"es#" (string, Unicode object or character buffer compatible
object) [const char *encoding, char **buffer, int *buffer_length]: This variant on "s#" is used for encoding Unicode and objects
convertible to Unicode into a character buffer. It reads one C
variable and stores into two C variables, the first one a pointer to
an encoding name string (encoding), the second a pointer to a
pointer to a character buffer (**buffer, the buffer used for
storing the encoded data) and the third one a pointer to an integer
(*buffer_length, the buffer length).
The encoding name must map to a registered codec. If set to NULL,
the default encoding is used.
There are two modes of operation:
If *buffer points a NULL pointer,
PyArg_ParseTuple() will allocate a buffer of the needed
size using PyMem_NEW(), copy the encoded data into this
buffer and adjust *buffer to reference the newly allocated
storage. The caller is responsible for calling
PyMem_Free() to free the allocated buffer after usage.
If *buffer points to a non-NULL pointer (an already allocated
buffer), PyArg_ParseTuple() will use this location as
buffer and interpret *buffer_length as buffer size. It will then
copy the encoded data into the buffer and 0-terminate it. Buffer
overflow is signalled with an exception.
In both cases, *buffer_length is set to the length of the
encoded data without the trailing 0-byte.
"b" (integer) [char]: Convert a Python integer to a tiny int, stored in a C char.
"h" (integer) [short int]: Convert a Python integer to a C short int.
"i" (integer) [int]: Convert a Python integer to a plain C int.
"l" (integer) [long int]: Convert a Python integer to a C long int.
"c" (string of length 1) [char]: Convert a Python character, represented as a string of length 1, to a
C char.
"f" (float) [float]: Convert a Python floating point number to a C float.
"d" (float) [double]: Convert a Python floating point number to a C double.
"D" (complex) [Py_complex]: Convert a Python complex number to a C Py_complex structure.
"O" (object) [PyObject *]: Store a Python object (without any conversion) in a C object pointer.
The C program thus receives the actual object that was passed. The
object's reference count is not increased. The pointer stored is not
NULL.
"O!" (object) [typeobject, PyObject *]: Store a Python object in a C object pointer. This is similar to
"O", but takes two C arguments: the first is the address of a
Python type object, the second is the address of the C variable (of
type PyObject *) into which the object pointer is stored.
If the Python object does not have the required type,
TypeError is raised.
"O&" (object) [converter, anything]: Convert a Python object to a C variable through a converter
function. This takes two arguments: the first is a function, the
second is the address of a C variable (of arbitrary type), converted
to void *. The converter function in turn is called as
follows:
status`=`converter`(`object, address`);`
where object is the Python object to be converted and
address is the void * argument that was passed to
PyArg_ConvertTuple(). The returned status should be
`1` for a successful conversion and `0` if the conversion
has failed. When the conversion fails, the converter function
should raise an exception.
"S" (string) [PyStringObject *]: Like "O" but requires that the Python object is a string object.
Raises TypeError if the object is not a string object.
The C variable may also be declared as PyObject *.
"U" (Unicode string) [PyUnicodeObject *]: Like "O" but requires that the Python object is a Unicode object.
Raises TypeError if the object is not a Unicode object.
The C variable may also be declared as PyObject *.
"t#" (read-only character buffer) [char *, int]: Like "s#", but accepts any object which implements the read-only
buffer interface. The char * variable is set to point to the
first byte of the buffer, and the int is set to the length of
the buffer. Only single-segment buffer objects are accepted;
TypeError is raised for all others.
"w" (read-write character buffer) [char *]: Similar to "s", but accepts any object which implements the
read-write buffer interface. The caller must determine the length of
the buffer by other means, or use "w#" instead. Only
single-segment buffer objects are accepted; TypeError is
raised for all others.
"w#" (read-write character buffer) [char *, int]: Like "s#", but accepts any object which implements the
read-write buffer interface. The char * variable is set to
point to the first byte of the buffer, and the int is set to
the length of the buffer. Only single-segment buffer objects are
accepted; TypeError is raised for all others.
"(items)" (tuple) [matching-items]: The object must be a Python sequence whose length is the number of
format units in items. The C arguments must correspond to the
individual format units in items. Format units for sequences
may be nested.
Note: Prior to Python version 1.5.2, this format specifier
only accepted a tuple containing the individual parameters, not an
arbitrary sequence. Code which previously caused
TypeError to be raised here may now proceed without an
exception. This is not expected to be a problem for existing code.
It is possible to pass Python long integers where integers are
requested; however no proper range checking is done -- the most
significant bits are silently truncated when the receiving field is
too small to receive the value (actually, the semantics are inherited
from downcasts in C -- your mileage may vary).
A few other characters have a meaning in a format string. These may
not occur inside nested parentheses. They are:
"|": Indicates that the remaining arguments in the Python argument list are
optional. The C variables corresponding to optional arguments should
be initialized to their default value -- when an optional argument is
not specified, PyArg_ParseTuple() does not touch the contents
of the corresponding C variable(s).
":": The list of format units ends here; the string after the colon is used
as the function name in error messages (the ``associated value'' of
the exception that PyArg_ParseTuple() raises).
";": The list of format units ends here; the string after the colon is used
as the error message instead of the default error message.
Clearly, ":" and ";" mutually exclude each other.
Some example calls:
```text

int ok;
int i, j;
long k, l;
char *s;
int size;

ok = PyArg_ParseTuple(args, ""); /* No arguments */
/* Python call: f() */
```
```text

ok = PyArg_ParseTuple(args, "s", &s); /* A string */
/* Possible Python call: f('whoops!') */
```
```text

ok = PyArg_ParseTuple(args, "lls", &k, &l, &s); /* Two longs and a string */
/* Possible Python call: f(1, 2, 'three') */
```
```text

ok = PyArg_ParseTuple(args, "(ii)s#", &i, &j, &s, &size);
/* A pair of ints and a string, whose size is also returned */
/* Possible Python call: f((1, 2), 'three') */
```
```text

{
char *file;
char *mode = "r";
int bufsize = 0;
ok = PyArg_ParseTuple(args, "s|si", &file, &mode, &bufsize);
/* A string, and optionally another string and an integer */
/* Possible Python calls:
f('spam')
f('spam', 'w')
f('spam', 'wb', 100000) */
}
```
```text

{
int left, top, right, bottom, h, v;
ok = PyArg_ParseTuple(args, "((ii)(ii))(ii)",
&left, &top, &right, &bottom, &h, &v);
/* A rectangle and a point */
/* Possible Python call:
f(((0, 0), (400, 300)), (10, 10)) */
}
```
```text

{
Py_complex c;
ok = PyArg_ParseTuple(args, "D:myfunction", &c);
/* a complex, also providing a function name for errors */
/* Possible Python call: myfunction(1+2j) */
}
```


parseTuple.html | intro.html | buildValue.html | Extending and Embedding the Python Interpreter | contents.html
---
# 1.8 Keyword Parsing with PyArg_ParseTupleAndKeywords()
The PyArg_ParseTupleAndKeywords() function is declared as
follows:
```text

int PyArg_ParseTupleAndKeywords(PyObject *arg, PyObject *kwdict,
char *format, char **kwlist, ...);
```
The arg and format parameters are identical to those of the
PyArg_ParseTuple() function. The kwdict parameter
is the dictionary of keywords received as the third parameter from the
Python runtime. The kwlist parameter is a NULL-terminated
list of strings which identify the parameters; the names are matched
with the type information from format from left to right.
Note: Nested tuples cannot be parsed when using keyword
arguments! Keyword parameters passed in which are not present in the
kwlist will cause TypeError to be raised.
Here is an example module which uses keywords, based on an example by
Geoff Philbrick (philbrick@hks.com):
```text

#include <stdio.h>
#include "Python.h"

static PyObject *
keywdarg_parrot(self, args, keywds)
PyObject *self;
PyObject *args;
PyObject *keywds;
{
int voltage;
char *state = "a stiff";
char *action = "voom";
char *type = "Norwegian Blue";

static char *kwlist[] = {"voltage", "state", "action", "type", NULL};

if (!PyArg_ParseTupleAndKeywords(args, keywds, "i|sss", kwlist,
&voltage, &state, &action, &type))
return NULL;

printf("-- This parrot wouldn't %s if you put %i Volts through it.\n",
action, voltage);
printf("-- Lovely plumage, the %s -- It's %s!\n", type, state);

Py_INCREF(Py_None);

return Py_None;
}

static PyMethodDef keywdarg_methods[] = {
/* The cast of the function is necessary since PyCFunction values
* only take two PyObject* parameters, and keywdarg_parrot() takes
* three.
*/
{"parrot", (PyCFunction)keywdarg_parrot, METH_VARARGS|METH_KEYWORDS},
{NULL, NULL} /* sentinel */
};

void
initkeywdarg()
{
/* Create the module and add the functions */
Py_InitModule("keywdarg", keywdarg_methods);
}
```


1.10 Reference Counts

buildValue.html | intro.html | refcountsInPython.html | Extending and Embedding the Python Interpreter | contents.html
---
# 1.10 Reference Counts
In languages like C or C++, the programmer is responsible for
dynamic allocation and deallocation of memory on the heap. In C,
this is done using the functions malloc() and
free(). In C++, the operators new and
delete are used with essentially the same meaning; they are
actually implemented using malloc() and
free(), so we'll restrict the following discussion to the
latter.
Every block of memory allocated with malloc() should
eventually be returned to the pool of available memory by exactly one
call to free(). It is important to call
free() at the right time. If a block's address is
forgotten but free() is not called for it, the memory it
occupies cannot be reused until the program terminates. This is
called a memory leak. On the other hand, if a program calls
free() for a block and then continues to use the block, it
creates a conflict with re-use of the block through another
malloc() call. This is called using freed memory.
It has the same bad consequences as referencing uninitialized data --
core dumps, wrong results, mysterious crashes.
Common causes of memory leaks are unusual paths through the code. For
instance, a function may allocate a block of memory, do some
calculation, and then free the block again. Now a change in the
requirements for the function may add a test to the calculation that
detects an error condition and can return prematurely from the
function. It's easy to forget to free the allocated memory block when
taking this premature exit, especially when it is added later to the
code. Such leaks, once introduced, often go undetected for a long
time: the error exit is taken only in a small fraction of all calls,
and most modern machines have plenty of virtual memory, so the leak
only becomes apparent in a long-running process that uses the leaking
function frequently. Therefore, it's important to prevent leaks from
happening by having a coding convention or strategy that minimizes
this kind of errors.
Since Python makes heavy use of malloc() and
free(), it needs a strategy to avoid memory leaks as well
as the use of freed memory. The chosen method is called
reference counting. The principle is simple: every object
contains a counter, which is incremented when a reference to the
object is stored somewhere, and which is decremented when a reference
to it is deleted. When the counter reaches zero, the last reference
to the object has been deleted and the object is freed.
An alternative strategy is called automatic garbage collection.
(Sometimes, reference counting is also referred to as a garbage
collection strategy, hence my use of ``automatic'' to distinguish the
two.) The big advantage of automatic garbage collection is that the
user doesn't need to call free() explicitly. (Another claimed
advantage is an improvement in speed or memory usage -- this is no
hard fact however.) The disadvantage is that for C, there is no
truly portable automatic garbage collector, while reference counting
can be implemented portably (as long as the functions malloc()
and free() are available -- which the C Standard guarantees).
Maybe some day a sufficiently portable automatic garbage collector
will be available for C. Until then, we'll have to live with
reference counts.


1.10.1 Reference Counting in Python

refcounts.html | refcounts.html | ownershipRules.html | Extending and Embedding the Python Interpreter | contents.html
---
## 1.10.1 Reference Counting in Python
There are two macros, `Py_INCREF(x)` and `Py_DECREF(x)`,
which handle the incrementing and decrementing of the reference count.
Py_DECREF() also frees the object when the count reaches zero.
For flexibility, it doesn't call free() directly -- rather, it
makes a call through a function pointer in the object's type
object. For this purpose (and others), every object also contains a
pointer to its type object.
The big question now remains: when to use `Py_INCREF(x)` and
`Py_DECREF(x)`? Let's first introduce some terms. Nobody
``owns'' an object; however, you can own a reference to an
object. An object's reference count is now defined as the number of
owned references to it. The owner of a reference is responsible for
calling Py_DECREF() when the reference is no longer
needed. Ownership of a reference can be transferred. There are three
ways to dispose of an owned reference: pass it on, store it, or call
Py_DECREF(). Forgetting to dispose of an owned reference
creates a memory leak.
It is also possible to borrow1.2 (#foot614) a reference to an object. The borrower
of a reference should not call Py_DECREF(). The borrower must
not hold on to the object longer than the owner from which it was
borrowed. Using a borrowed reference after the owner has disposed of
it risks using freed memory and should be avoided
completely.1.3 (#foot960)
The advantage of borrowing over owning a reference is that you don't
need to take care of disposing of the reference on all possible paths
through the code -- in other words, with a borrowed reference you
don't run the risk of leaking when a premature exit is taken. The
disadvantage of borrowing over leaking is that there are some subtle
situations where in seemingly correct code a borrowed reference can be
used after the owner from which it was borrowed has in fact disposed
of it.
A borrowed reference can be changed into an owned reference by calling
Py_INCREF(). This does not affect the status of the owner from
which the reference was borrowed -- it creates a new owned reference,
and gives full owner responsibilities (i.e., the new owner must
dispose of the reference properly, as well as the previous owner).


A. Reporting Bugs

link-reqs.html | ext.html | about.html | Extending and Embedding the Python Interpreter | contents.html
---
# A. Reporting Bugs
Python is a mature programming language which has established a
reputation for stability. In order to maintain this reputation, the
developers would like to know of any deficiencies you find in Python
or its documentation.
All bug reports should be submitted via the Python Bug Tracker on
SourceForge (http://sourceforge.net/bugs/?group_id=5470). The
bug tracker offers a Web form which allows pertinent information to be
entered and submitted to the developers.
Before submitting a report, please log into SourceForge if you are a
member; this will make it possible for the developers to contact you
for additional information if needed. If you are not a SourceForge
member but would not mind the developers contacting you, you may
include your email address in your bug description. In this case,
please realize that the information is publically available and cannot
be protected.
The first step in filing a report is to determine whether the problem
has already been reported. The advantage in doing so, aside from
saving the developers time, is that you learn what has been done to
fix it; it may be that the problem has already been fixed for the next
release, or additional information is needed (in which case you are
welcome to provide it if you can!). To do this, search the bug
database using the search box near the bottom of the page.
If the problem you're reporting is not already in the bug tracker, go
back to the Python Bug Tracker
(http://sourceforge.net/bugs/?group_id=5470). Select the
``Submit a Bug'' link at the top of the page to open the bug reporting
form.
The submission form has a number of fields. The only fields that are
required are the ``Summary'' and ``Details'' fields. For the summary,
enter a very short description of the problem; less than ten
words is good. In the Details field, describe the problem in detail,
including what you expected to happen and what did happen. Be sure to
include the version of Python you used, whether any extension modules
were involved, and what hardware and software platform you were using
(including version information as appropriate).
The only other field that you may want to set is the ``Category''
field, which allows you to place the bug report into a broad category
(such as ``Documentation'' or ``Library'').
Each bug report will be assigned to a developer who will determine
what needs to be done to correct the problem. If you have a
SourceForge account and logged in to report the problem, you will
receive an update each time action is taken on the bug.
See Also:


1.1 A Simple Example

intro.html | intro.html | errors.html | Extending and Embedding the Python Interpreter | contents.html
---
# 1.1 A Simple Example
Let's create an extension module called "spam" (the favorite food
of Monty Python fans...) and let's say we want to create a Python
interface to the C library function system().1.1 (#foot936)This function takes a null-terminated character string as argument and
returns an integer. We want this function to be callable from Python
as follows:
```text

>>> import spam
>>> status = spam.system("ls -l")
```
Begin by creating a file spammodule.c. (Historically, if a
module is called "spam", the C file containing its implementation
is called spammodule.c; if the module name is very long, like
"spammify", the module name can be just spammify.c.)
The first line of our file can be:
```text

#include <Python.h>
```
which pulls in the Python API (you can add a comment describing the
purpose of the module and a copyright notice if you like).
All user-visible symbols defined by `"Python.h"` have a prefix of
"Py" or "PY", except those defined in standard header files.
For convenience, and since they are used extensively by the Python
interpreter, `"Python.h"` includes a few standard header files:
`<stdio.h>`, `<string.h>`, `<errno.h>`, and
`<stdlib.h>`. If the latter header file does not exist on your
system, it declares the functions malloc(),
free() and realloc() directly.
The next thing we add to our module file is the C function that will
be called when the Python expression "spam.system(string)"is evaluated (we'll see shortly how it ends up being called):
```text

static PyObject *
spam_system(self, args)
PyObject *self;
PyObject *args;
{
char *command;
int sts;

if (!PyArg_ParseTuple(args, "s", &command))
return NULL;
sts = system(command);
return Py_BuildValue("i", sts);
}
```
There is a straightforward translation from the argument list in
Python (e.g. the single expression `"ls -l"`) to the arguments
passed to the C function. The C function always has two arguments,
conventionally named self and args.
The self argument is only used when the C function implements a
built-in method, not a function. In the example, self will
always be a NULL pointer, since we are defining a function, not a
method. (This is done so that the interpreter doesn't have to
understand two different types of C functions.)
The args argument will be a pointer to a Python tuple object
containing the arguments. Each item of the tuple corresponds to an
argument in the call's argument list. The arguments are Python
objects -- in order to do anything with them in our C function we have
to convert them to C values. The function PyArg_ParseTuple()
in the Python API checks the argument types and converts them to C
values. It uses a template string to determine the required types of
the arguments as well as the types of the C variables into which to
store the converted values. More about this later.
PyArg_ParseTuple() returns true (nonzero) if all arguments have
the right type and its components have been stored in the variables
whose addresses are passed. It returns false (zero) if an invalid
argument list was passed. In the latter case it also raises an
appropriate exception so the calling function can return
NULL immediately (as we saw in the example).


1.10.3 Thin Ice

ownershipRules.html | refcounts.html | nullPointers.html | Extending and Embedding the Python Interpreter | contents.html
---
## 1.10.3 Thin Ice
There are a few situations where seemingly harmless use of a borrowed
reference can lead to problems. These all have to do with implicit
invocations of the interpreter, which can cause the owner of a
reference to dispose of it.
The first and most important case to know about is using
Py_DECREF() on an unrelated object while borrowing a
reference to a list item. For instance:
```text

bug(PyObject *list) {
PyObject *item = PyList_GetItem(list, 0);

PyList_SetItem(list, 1, PyInt_FromLong(0L));
PyObject_Print(item, stdout, 0); /* BUG! */
}
```
This function first borrows a reference to `list[0]`, then
replaces `list[1]` with the value `0`, and finally prints
the borrowed reference. Looks harmless, right? But it's not!
Let's follow the control flow into PyList_SetItem(). The list
owns references to all its items, so when item 1 is replaced, it has
to dispose of the original item 1. Now let's suppose the original
item 1 was an instance of a user-defined class, and let's further
suppose that the class defined a __del__() method. If this
class instance has a reference count of 1, disposing of it will call
its __del__() method.
Since it is written in Python, the __del__() method can execute
arbitrary Python code. Could it perhaps do something to invalidate
the reference to `item` in bug()? You bet! Assuming
that the list passed into bug() is accessible to the
__del__() method, it could execute a statement to the effect of
"del list[0]", and assuming this was the last reference to that
object, it would free the memory associated with it, thereby
invalidating `item`.
The solution, once you know the source of the problem, is easy:
temporarily increment the reference count. The correct version of the
function reads:
```text

no_bug(PyObject *list) {
PyObject *item = PyList_GetItem(list, 0);

Py_INCREF(item);
PyList_SetItem(list, 1, PyInt_FromLong(0L));
PyObject_Print(item, stdout, 0);
Py_DECREF(item);
}
```
This is a true story. An older version of Python contained variants
of this bug and someone spent a considerable amount of time in a C
debugger to figure out why his __del__() methods would fail...
The second case of problems with a borrowed reference is a variant
involving threads. Normally, multiple threads in the Python
interpreter can't get in each other's way, because there is a global
lock protecting Python's entire object space. However, it is possible
to temporarily release this lock using the macro
`Py_BEGIN_ALLOW_THREADS`, and to re-acquire it using
`Py_END_ALLOW_THREADS`. This is common around blocking I/O
calls, to let other threads use the CPU while waiting for the I/O to
complete. Obviously, the following function has the same problem as
the previous one:
```text

bug(PyObject *list) {
PyObject *item = PyList_GetItem(list, 0);
Py_BEGIN_ALLOW_THREADS
...some blocking I/O call...
Py_END_ALLOW_THREADS
PyObject_Print(item, stdout, 0); /* BUG! */
}
```


cplusplus.html | intro.html | building-on-unix.html | Extending and Embedding the Python Interpreter | contents.html
---
# 1.12 Providing a C API for an Extension Module
Many extension modules just provide new functions and types to be
used from Python, but sometimes the code in an extension module can
be useful for other extension modules. For example, an extension
module could implement a type ``collection'' which works like lists
without order. Just like the standard Python list type has a C API
which permits extension modules to create and manipulate lists, this
new collection type should have a set of C functions for direct
manipulation from other extension modules.
At first sight this seems easy: just write the functions (without
declaring them static, of course), provide an appropriate
header file, and document the C API. And in fact this would work if
all extension modules were always linked statically with the Python
interpreter. When modules are used as shared libraries, however, the
symbols defined in one module may not be visible to another module.
The details of visibility depend on the operating system; some systems
use one global namespace for the Python interpreter and all extension
modules (e.g. Windows), whereas others require an explicit list of
imported symbols at module link time (e.g. AIX), or offer a choice of
different strategies (most Unices). And even if symbols are globally
visible, the module whose functions one wishes to call might not have
been loaded yet!
Portability therefore requires not to make any assumptions about
symbol visibility. This means that all symbols in extension modules
should be declared static, except for the module's
initialization function, in order to avoid name clashes with other
extension modules (as discussed in section 1.4 (methodTable.html#methodTable)). And it
means that symbols that should be accessible from other
extension modules must be exported in a different way.
Python provides a special mechanism to pass C-level information (i.e.
pointers) from one extension module to another one: CObjects.
A CObject is a Python data type which stores a pointer (void
*). CObjects can only be created and accessed via their C API, but
they can be passed around like any other Python object. In particular,
they can be assigned to a name in an extension module's namespace.
Other extension modules can then import this module, retrieve the
value of this name, and then retrieve the pointer from the CObject.
There are many ways in which CObjects can be used to export the C API
of an extension module. Each name could get its own CObject, or all C
API pointers could be stored in an array whose address is published in
a CObject. And the various tasks of storing and retrieving the pointers
can be distributed in different ways between the module providing the
code and the client modules.
The following example demonstrates an approach that puts most of the
burden on the writer of the exporting module, which is appropriate
for commonly used library modules. It stores all C API pointers
(just one in the example!) in an array of void pointers which
becomes the value of a CObject. The header file corresponding to
the module provides a macro that takes care of importing the module
and retrieving its C API pointers; client modules only have to call
this macro before accessing the C API.
The exporting module is a modification of the spam module from
section 1.1 (simpleExample.html#simpleExample). The function spam.system()
does not call the C library function system() directly,
but a function PySpam_System(), which would of course do
something more complicated in reality (such as adding ``spam'' to
every command). This function PySpam_System() is also
exported to other extension modules.
The function PySpam_System() is a plain C function,
declared static like everything else:
```text

static int
PySpam_System(command)
char *command;
{
return system(command);
}
```
The function spam_system() is modified in a trivial way:
```text

static PyObject *
spam_system(self, args)
PyObject *self;
PyObject *args;
{
char *command;
int sts;

if (!PyArg_ParseTuple(args, "s", &command))
return NULL;
sts = PySpam_System(command);
return Py_BuildValue("i", sts);
}
```
In the beginning of the module, right after the line
```text

#include "Python.h"
```
two more lines must be added:
```text

#define SPAM_MODULE
#include "spammodule.h"
```
The `#define` is used to tell the header file that it is being
included in the exporting module, not a client module. Finally,
the module's initialization function must take care of initializing
the C API pointer array:
```text

void
initspam()
{
PyObject *m, *d;
static void *PySpam_API[PySpam_API_pointers];
PyObject *c_api_object;
m = Py_InitModule("spam", SpamMethods);

/* Initialize the C API pointer array */
PySpam_API[PySpam_System_NUM] = (void *)PySpam_System;

/* Create a CObject containing the API pointer array's address */
c_api_object = PyCObject_FromVoidPtr((void *)PySpam_API, NULL);

/* Create a name for this object in the module's namespace */
d = PyModule_GetDict(m);
PyDict_SetItemString(d, "_C_API", c_api_object);
}
```
Note that `PySpam_API` is declared `static`; otherwise
the pointer array would disappear when `initspam` terminates!
The bulk of the work is in the header file spammodule.h,
which looks like this:
```text

#ifndef Py_SPAMMODULE_H
#define Py_SPAMMODULE_H
#ifdef __cplusplus
extern "C" {
#endif

/* Header file for spammodule */

/* C API functions */
#define PySpam_System_NUM 0
#define PySpam_System_RETURN int
#define PySpam_System_PROTO (char *command)

/* Total number of C API pointers */
#define PySpam_API_pointers 1

#ifdef SPAM_MODULE
/* This section is used when compiling spammodule.c */

static PySpam_System_RETURN PySpam_System PySpam_System_PROTO;

#else
/* This section is used in modules that use spammodule's API */

static void **PySpam_API;

#define PySpam_System \
(*(PySpam_System_RETURN (*)PySpam_System_PROTO) PySpam_API[PySpam_System_NUM])

#define import_spam() \
{ \
PyObject *module = PyImport_ImportModule("spam"); \
if (module != NULL) { \
PyObject *module_dict = PyModule_GetDict(module); \
PyObject *c_api_object = PyDict_GetItemString(module_dict, "_C_API"); \
if (PyCObject_Check(c_api_object)) { \
PySpam_API = (void **)PyCObject_AsVoidPtr(c_api_object); \
} \
} \
}

#endif

#ifdef __cplusplus
}
#endif

#endif /* !defined(Py_SPAMMODULE_H */
```
All that a client module must do in order to have access to the
function PySpam_System() is to call the function (or
rather macro) import_spam() in its initialization
function:
```text

void
initclient()
{
PyObject *m;

Py_InitModule("client", ClientMethods);
import_spam();
}
```
The main disadvantage of this approach is that the file
spammodule.h is rather complicated. However, the
basic structure is the same for each function that is
exported, so it has to be learned only once.
Finally it should be mentioned that CObjects offer additional
functionality, which is especially useful for memory allocation and
deallocation of the pointer stored in a CObject. The details
are described in the Python/C API
Reference Manual (../api/api.html) in the section ``CObjects'' and in the
implementation of CObjects (files Include/cobject.h and
Objects/cobject.c in the Python source code distribution).


3.1 A Cookbook Approach

building-on-windows.html | building-on-windows.html | dynamic-linking.html | Extending and Embedding the Python Interpreter | contents.html
---
# 3.1 A Cookbook Approach
This section provides a recipe for building a Python extension on
Windows.
Grab the binary installer from http://www.python.org/ and
install Python. The binary installer has all of the required header
files except for config.h.
Get the source distribution and extract it into a convenient location.
Copy the config.h from the PC/ directory into the
include/ directory created by the installer.
Create a Setup file for your extension module, as described in
chapter 2 (building-on-unix.html#building-on-unix).
Get David Ascher's compile.py script from
http://starship.python.net/crew/da/compile/. Run the script to
create Microsoft Visual C++ project files.
Open the DSW file in Visual C++ and select Build.
If your module creates a new type, you may have trouble with this line:
```text

PyObject_HEAD_INIT(&PyType_Type)
```
Change it to:
```text

PyObject_HEAD_INIT(NULL)
```
and add the following to the module initialization function:
```text

MyObject_Type.ob_type = &PyType_Type;
```
Refer to section 3 of the Python FAQ
(http://www.python.org/doc/FAQ.html) for details on why you must
do this.


3.3 Using DLLs in Practice

dynamic-linking.html | building-on-windows.html | embedding.html | Extending and Embedding the Python Interpreter | contents.html
---
# 3.3 Using DLLs in Practice
Windows Python is built in Microsoft Visual C++; using other
compilers may or may not work (though Borland seems to). The rest of
this section is MSVC++ specific.
When creating DLLs in Windows, you must pass python15.lib to
the linker. To build two DLLs, spam and ni (which uses C functions
found in spam), you could use these commands:
```text

cl /LD /I/python/include spam.c ../libs/python15.lib
cl /LD /I/python/include ni.c spam.lib ../libs/python15.lib
```
The first command created three files: spam.obj,
spam.dll and spam.lib. Spam.dll does not contain
any Python functions (such as PyArg_ParseTuple()), but it
does know how to find the Python code thanks to python15.lib.
The second command created ni.dll (and .obj and
.lib), which knows how to find the necessary functions from
spam, and also from the Python executable.
Not every identifier is exported to the lookup table. If you want any
other modules (including Python) to be able to see your identifiers,
you have to say "_declspec(dllexport)", as in "void
_declspec(dllexport) initspam(void)" or "PyObject
_declspec(dllexport) *NiGetSpamData(void)".
Developer Studio will throw in a lot of import libraries that you do
not really need, adding about 100K to your executable. To get rid of
them, use the Project Settings dialog, Link tab, to specify
ignore default libraries. Add the correct
msvcrtxx.lib to the list of libraries.


Python 2.0 Documentation - October 16, 2000

Python Documentation | modindex.html
---
# Python Documentation
Release 2.0
October 16, 2000
---
See About the Python Documentation (about.html)
for information on suggesting changes.


About this document ...

manual-install.html | inst.html | Installing Python Modules | contents.html
---
# About this document ...
Installing Python Modules
This document was generated using the LaTeX2HTML (http://saftsack.fs.uni-bayreuth.de/~latex2ht/) translator.
LaTeX2HTML (http://saftsack.fs.uni-bayreuth.de/~latex2ht/) is Copyright ©
1993, 1994, 1995, 1996, 1997, Nikos
Drakos (http://cbl.leeds.ac.uk/nikos/personal.html), Computer Based Learning Unit, University of
Leeds, and Copyright © 1997, 1998, Ross
Moore (http://www.maths.mq.edu.au/~ross/), Mathematics Department, Macquarie University,
Sydney.
The application of LaTeX2HTML (http://saftsack.fs.uni-bayreuth.de/~latex2ht/) to the Python
documentation has been heavily tailored by Fred L. Drake,
Jr. Original navigation icons were contributed by Christopher
Petrilli.
---
## Comments and Questions
General comments and questions regarding this document should
be sent by email to python-docs@python.org (mailto:python-docs@python.org). If you find specific errors in
this document, please report the bug at the Python Bug
Tracker (http://sourceforge.net/bugs/?group_id=5470) at SourceForge (http://sourceforge.net/).
Questions regarding how to use the information in this
document should be sent to the Python news group, comp.lang.python (news:comp.lang.python), or the Python mailing list (http://www.python.org/mailman/listinfo/python-list) (which is gated to the newsgroup and
carries the same content).
For any of these channels, please be sure not to send HTML email.
Thanks.
---
manual-install.html | inst.html | Installing Python Modules | contents.html
---


alt-install-prefix.html | alt-install.html | alt-install-windows.html | Installing Python Modules | contents.html
---
## 4.2 Alternate installation: Unix (the prefix scheme)
The ``prefix scheme'' is useful when you wish to use one Python
installation to perform the build/install (i.e., to run the setup
script), but install modules into the third-party module directory of a
different Python installation (or something that looks like a different
Python installation). If this sounds a trifle unusual, it is--that's
why the ``home scheme'' comes first. However, there are at least two
known cases where the prefix scheme will be useful.
First, consider that many Linux distributions put Python in /usr,
rather than the more traditional /usr/local. This is entirely
appropriate, since in those cases Python is part of ``the system''
rather than a local add-on. However, if you are installing Python
modules from source, you probably want them to go in
/usr/local/lib/python1.X rather than
/usr/lib/python1.X. This can be done with
```text

/usr/bin/python setup.py install --prefix=/usr/local
```
Another possibility is a network filesystem where the name used to write
to a remote directory is different from the name used to read it: for
example, the Python interpreter accessed as /usr/local/bin/python
might search for modules in /usr/local/lib/python1.X,
but those modules would have to be installed to, say,
/mnt/@server/export/lib/python1.X. This
could be done with
```text

/usr/local/bin/python setup.py install --prefix=/mnt/@server/export
```
In either case, the --prefix option defines the
installation base, and the --exec-prefix option defines
the platform-specific installation base, which is used for
platform-specific files. (Currently, this just means non-pure module
distributions, but could be expanded to C libraries, binary executables,
etc.) If --exec-prefix is not supplied, it defaults to
--prefix. Files are installed as follows:
There is no requirement that --prefix or
--exec-prefix actually point to an alternate Python
installation; if the directories listed above do not already exist, they
are created at installation time.
Incidentally, the real reason the prefix scheme is important is simply
that a standard Unix installation uses the prefix scheme, but with
--prefix and --exec-prefix supplied by
Python itself (as `sys.prefix` and `sys.exec_prefix`). Thus,
you might think you'll never use the prefix scheme, but every time you
run `python setup.py install` without any other options, you're
using it.
Note that installing extensions to an alternate Python installation has
no effect on how those extensions are built: in particular, the Python
header files (Python.h and friends) installed with the Python
interpreter used to run the setup script will be used in compiling
extensions. It is your responsibility to ensure that the interpreter
used to run extensions installed in this way is compatibile with the
interpreter used to build them. The best way to do this is to ensure
that the two interpreters are the same version of Python (possibly
different builds, or possibly copies of the same build). (Of course, if
your --prefix and --exec-prefix don't even
point to an alternate Python installation, this is immaterial.)


4.4 Alternate installation: Mac OS

alt-install-windows.html | alt-install.html | custom-install.html | Installing Python Modules | contents.html
---
## 4.4 Alternate installation: Mac OS
Like Windows, Mac OS has no notion of home directories (or even of
users), and a fairly simple standard Python installation. Thus, only a
--prefix option is needed. It defines the installation
base, and files are installed under it as follows:
See section 2.1 (platform-variations.html#platform-variations) for information on supplying
command-line arguments to the setup script with MacPython.


alt-install.html | alt-install.html | alt-install-home.html | Installing Python Modules | contents.html
---
## 4.1 Alternate installation: Unix (the home scheme)
Under Unix, there are two ways to perform an alternate installation.
The ``prefix scheme'' is similar to how alternate installation works
under Windows and Mac OS, but is not necessarily the most useful way to
maintain a personal Python library. Hence, we document the more
convenient and commonly useful ``home scheme'' first.
The idea behind the ``home scheme'' is that you build and maintain a
personal stash of Python modules, probably under your home directory.
Installing a new module distribution is as simple as
```text

python setup.py install --home=<dir>
```
where you can supply any directory you like for the --home
option. Lazy typists can just type a tilde (); the
`install` command will expand this to your home directory:
```text

python setup.py install --home=~
```
The --home option defines the installation base
directory. Files are installed to the following directories under the
installation base as follows:


4.3 Alternate installation: Windows

alt-install-home.html | alt-install.html | alt-install-macos.html | Installing Python Modules | contents.html
---
## 4.3 Alternate installation: Windows
Since Windows has no conception of a user's home directory, and since
the standard Python installation under Windows is simpler than that
under Unix, there's no point in having separate --prefix
and --home options. Just use the --prefix
option to specify a base directory, e.g.
```text

python setup.py install --prefix="\Temp\Python"
```
to install modules to the \Temp directory on the current
drive.
The installation base is defined by the --prefix option;
the --exec-prefix option is not supported under Windows.
Files are installed as follows:


4 Alternate Installation

non-ms-compilers.html | inst.html | alt-install-prefix.html | Installing Python Modules | contents.html
---
# 4 Alternate Installation
Often, it is necessary or desirable to install modules to a location
other than the standard location for third-party Python modules. For
example, on a Unix system you might not have permission to write to the
standard third-party module directory. Or you might wish to try out a
module before making it a standard part of your local Python
installation; this is especially true when upgrading a distribution
already present: you want to make sure your existing base of scripts
still works with the new version before actually upgrading.
The Distutils `install` command is designed to make installing
module distributions to an alternate location simple and painless. The
basic idea is that you supply a base directory for the installation, and
the `install` command picks a set of directories (called an
installation scheme) under this base directory in which to
install files. The details differ across platforms, so read whichever
of the following section applies to you.


3.4 Alternate installation: Mac OS

alt-windows.html | alt-install.html | custom-install.html | Installing Python Modules | contents.html
---
## 3.4 Alternate installation: Mac OS
Like Windows, Mac OS has no notion of home directories (or even of
users), and a fairly simple standard Python installation. Thus, only a
--prefix option is needed. It defines the installation
base, and files are installed under it as follows:
** how do MacPython users run the interpreter with command-line args? **
** Corran Webster says: ``Modules are found in either :Lib or
:Mac:Lib, while extensions usually go in
:Mac:PlugIns''--does this mean that non-pure distributions should
be divided between :Mac:PlugIns and :Mac:Lib? If so, that
changes the granularity at which we care about modules: instead of
``modules from pure distributions'' and ``modules from non-pure
distributions'', it becomes ``modules from pure distributions'',
``Python modules from non-pure distributions'', and ``extensions from
non-pure distributions''. Is this necessary?!? **


alt-unix-prefix.html | alt-install.html | alt-windows.html | Installing Python Modules | contents.html
---
## 3.2 Alternate installation: Unix (the prefix scheme)
The ``prefix scheme'' is useful when you wish to use one Python
installation to perform the build/install (i.e., to run the setup
script), but install modules into the third-party module directory of a
different Python installation (or something that looks like a different
Python installation). If this sounds a trifle unusual, it is--that's
why the ``home scheme'' comes first. However, there are at least two
known cases where the prefix scheme will be useful.
First, consider that many Linux distributions put Python in /usr,
rather than the more traditional /usr/local. This is entirely
appropriate, since in those cases Python is part of ``the system''
rather than a local add-on. However, if you are installing Python
modules from source, you probably want them to go in
/usr/local/lib/python1.X rather than
/usr/lib/python1.X. This can be done with
```text

/usr/bin/python setup.py install --prefix=/usr/local
```
Another possibility is a network filesystem where the name used to write
to a remote directory is different from the name used to read it: for
example, the Python interpreter accessed as /usr/local/bin/python
might search for modules in /usr/local/lib/python1.X,
but those modules would have to be installed to, say,
/mnt/@server/export/lib/python1.X. This
could be done with
```text

/usr/local/bin/python setup.py install --prefix=/mnt/@server/export
```
In either case, the --prefix option defines the
installation base, and the --exec-prefix option defines
the platform-specific installation base, which is used for
platform-specific files. (Currently, this just means non-pure module
distributions, but could be expanded to C libraries, binary executables,
etc.) If --exec-prefix is not supplied, it defaults to
--prefix. Files are installed as follows:
There is no requirement that --prefix or
--exec-prefix actually point to an alternate Python
installation; if the directories listed above do not already exist, they
are created at installation time.
Incidentally, the real reason the prefix scheme is important is simply
that a standard Unix installation uses the prefix scheme, but with
--prefix and --exec-prefix supplied by
Python itself (as `sys.prefix` and `sys.exec_prefix`). Thus,
you might think you'll never use the prefix scheme, but every time you
run `python setup.py install` without any other options, you're
using it.
Note that installing extensions to an alternate Python installation has
no effect on how those extensions are built: in particular, the Python
header files (Python.h and friends) installed with the Python
interpreter used to run the setup script will be used in compiling
extensions. It is your responsibility to ensure that the interpreter
used to run extensions installed in this way is compatibile with the
interpreter used to build them. The best way to do this is to ensure
that the two interpreters are the same version of Python (possibly
different builds, or possibly copies of the same build). (Of course, if
your --prefix and --exec-prefix don't even
point to an alternate Python installation, this is immaterial.)


alt-install.html | alt-install.html | alt-unix-home.html | Installing Python Modules | contents.html
---
## 3.1 Alternate installation: Unix (the home scheme)
Under Unix, there are two ways to perform an alternate installation.
The ``prefix scheme'' is similar to how alternate installation works
under Windows and Mac OS, but is not necessarily the most useful way to
maintain a personal Python library. Hence, we document the more
convenient and commonly useful ``home scheme'' first.
The idea behind the ``home scheme'' is that you build and maintain a
personal stash of Python modules, probably under your home directory.
Installing a new module distribution is as simple as
```text

python setup.py install --home=<dir>
```
where you can supply any directory you like for the --home
option. Lazy typists can just type a tilde (); the
`install` command will expand this to your home directory:
```text

python setup.py install --home=~
```
The --home option defines the installation base
directory. Files are installed to the following directories under the
installation base as follows:


3.3 Alternate installation: Windows

alt-unix-home.html | alt-install.html | alt-macos.html | Installing Python Modules | contents.html
---
## 3.3 Alternate installation: Windows
Since Windows has no conception of a user's home directory, and since
the standard Python installation under Windows is simpler than that
under Unix, there's no point in having separate --prefix
and --home options. Just use the --prefix
option to specify a base directory, e.g.
```text

python setup.py install --prefix="\Temp\Python"
```
to install modules to the \Temp directory on the current
drive.
The installation base is defined by the --prefix option;
the --exec-prefix option is not supported under Windows.
Files are installed as follows:


3 Building Extensions: Tips and Tricks

how-install-works.html | inst.html | tweak-flags.html | Installing Python Modules | contents.html
---
# 3 Building Extensions: Tips and Tricks
(This is the section to read for people doing any sort of interesting
build. Things to talk about:
- the Setup file (any platform now, but Unix-biased)
- CFLAGS and LDFLAGS (must implement them first!)
- using non-MS compilers on Windows (how to convert
Python's library, ...)


6 Distutils Configuration Files

custom-install.html | inst.html | pre-distutils.html | Installing Python Modules | contents.html
---
# 6 Distutils Configuration Files


Contents

inst.html | inst.html | intro.html | Installing Python Modules
---
## Contents
Table of Contents
- Contents (contents.html)
  1 Introduction (intro.html)
  - 1.1 Best case: trivial installation (trivial-install.html)
  1.2 The new standard: Distutils (new-standard.html)
  1.3 The old way: no standards (old-way.html)
  2 Standard Build and Install (standard-install.html)
  - 2.1 Platform variations (platform-variations.html)
  2.2 Splitting the job up (splitting-up.html)
  2.3 How building works (how-build-works.html)
  2.4 How installation works (how-install-works.html)
  3 Building Extensions: Tips and Tricks (building-ext.html)
  - 3.1 Tweaking compiler/linker flags (tweak-flags.html)
  3.2 Using non-Microsoft compilers on Windows (non-ms-compilers.html)
  4 Alternate Installation (alt-install.html)
  - 4.1 Alternate installation: Unix (the home scheme) (alt-install-prefix.html)
  4.2 Alternate installation: Unix (the prefix scheme) (alt-install-home.html)
  4.3 Alternate installation: Windows (alt-install-windows.html)
  4.4 Alternate installation: Mac OS (alt-install-macos.html)
  5 Custom Installation (custom-install.html)
  6 Distutils Configuration Files (config-files.html)
  7 Pre-Distutils Conventions (pre-distutils.html)
  - 7.1 The Makefile.pre.in file (makefile-pre-in.html)
  7.2 Installing modules manually (manual-install.html)
  About this document ... (about.html)
End of Table of Contents


5 Custom Installation

alt-install-macos.html | inst.html | config-files.html | Installing Python Modules | contents.html
---
# 5 Custom Installation
Sometimes, the alternate installation schemes described in
section 4 (alt-install.html#alt-install) just don't do what you want. You might
want to tweak just one or two directories while keeping everything under
the same base directory, or you might want to completely redefine the
installation scheme. In either case, you're creating a custom
installation scheme.
You probably noticed the column of ``override options'' in the tables
describing the alternate installation schemes above. Those options are
how you define a custom installation scheme. These override options can
be relative, absolute, or explicitly defined in terms of one of the
installation base directories. (There are two installation base
directories, and they are normally the same--they only differ when you
use the Unix ``prefix scheme'' and supply different
--prefix and --exec-prefix options.)
For example, say you're installing a module distribution to your home
directory under Unix--but you want scripts to go in
 /scripts rather than  /bin.
As you might expect, you can override this directory with the
--install-scripts option; in this case, it makes most
sense to supply a relative path, which will be interpreted relative to
the installation base directory (your home directory, in this case):
```text

python setup.py install --home=~ --install-scripts=scripts
```
Another Unix example: suppose your Python installation was built and
installed with a prefix of /usr/local/python, so under a standard
installation scripts will wind up in /usr/local/python/bin. If
you want them in /usr/local/bin instead, you would supply this
absolute directory for the --install-scripts option:
```text

python setup.py install --install-scripts=/usr/local/bin
```
(This performs an installation using the ``prefix scheme,'' where the
prefix is whatever your Python interpreter was installed with--
/usr/local/python in this case.)
If you maintain Python on Windows, you might want third-party modules to
live in a subdirectory of prefix, rather than right in
prefix itself. This is almost as easy as customizing the
script installation directory--you just have to remember that there are
two types of modules to worry about, pure modules and non-pure modules
(i.e., modules from a non-pure distribution). For example:
```text

python setup.py install --install-purelib=Site --install-platlib=Site
```
The specified installation directories are relative to prefix.
Of course, you also have to ensure that these directories are in
Python's module search path, e.g. by putting a .pth file in
prefix (** should have a section describing .pth files and
cross-ref it here **).
If you want to define an entire installation scheme, you just have to
supply all of the installation directory options. The recommended way
to do this is to supply relative paths; for example, if you want to
maintain all Python module-related files under python in your
home directory, and you want a separate directory for each platform that
you use your home directory from, you might define the following
installation scheme:
```text

python setup.py install --home=~ \
--install-purelib=python/lib \
--install-platlib=python/lib.$PLAT \
--install-scripts=python/scripts
--install-data=python/data
```
or, equivalently,
```text

python setup.py install --home=~/python \
--install-purelib=lib \
--install-platlib='lib.$PLAT' \
--install-scripts=scripts
--install-data=data
```
`$PLAT` is not (necessarily) an environment variable--it will be
expanded by the Distutils as it parses your command line options (just
as it does when parsing your configuration file(s)).
Obviously, specifying the entire installation scheme every time you
install a new module distribution would be very tedious. Thus, you can
put these options into your Distutils config file (see
section 6 (config-files.html#config-files)):
```text

[install]
install-base=$HOME
install-purelib=python/lib
install-platlib=python/lib.$PLAT
install-scripts=python/scripts
install-data=python/data
```
or, equivalently,
```text

[install]
install-base=$HOME/python
install-purelib=lib
install-platlib=lib.$PLAT
install-scripts=scripts
install-data=data
```
Note that these two are not equivalent if you supply a different
installation base directory when you run the setup script. For example,
```text

python setup.py --install-base=/tmp
```
would install pure modules to /tmp/python/lib in the first
case, and to /tmp/lib in the second case. (For the second
case, you probably want to supply an installation base of
/tmp/python.)
You probably noticed the use of `$HOME` and `$PLAT` in the
sample configuration file input. These are Distutils configuration
variables, which bear a strong resemblance to environment variables. In
fact, you can use environment variables in config files--on platforms
that have such a notion--but the Distutils additionally define a few
extra variables that may not be in your environment, such as
`$PLAT`. (And of course, you can only use the configuration
variables supplied by the Distutils on systems that don't have
environment variables, such as Mac OS (** true? **).) See
section 6 (config-files.html#config-files) for details.
** need some Windows and Mac OS examples--when would custom
installation schemes be needed on those platforms? **


2.3 How building works

splitting-up.html | standard-install.html | how-install-works.html | Installing Python Modules | contents.html
---
## 2.3 How building works
As implied above, the `build` command is responsible for putting
the files to install into a build directory. By default, this is
build under the distribution root; if you're excessively
concerned with speed, or want to keep the source tree pristine, you can
change the build directory with the --build-base option.
For example:
```text

python setup.py build --build-base=/tmp/pybuild/foo-1.0
```
(Or you could do this permanently with a directive in your system or
personal Distutils configuration file; see
section 6 (config-files.html#config-files).) Normally, this isn't necessary.
The default layout for the build tree is as follows:
```text

--- build/ --- lib/
or
--- build/ --- lib.<plat>/
temp.<plat>/
```
where `<plat>` expands to a brief description of the current
OS/hardware platform. The first form, with just a lib directory,
is used for ``pure module distributions''--that is, module
distributions that include only pure Python modules. If a module
distribution contains any extensions (modules written in C/C++, or Java
for JPython), then the second form, with two `<plat>` directories,
is used. In that case, the temp.plat directory holds
temporary files generated by the compile/link process that don't
actually get installed. In either case, the lib (or
lib.plat) directory contains all Python modules (pure
Python and extensions) that will be installed.
In the future, more directories will be added to handle Python scripts,
documentation, binary executables, and whatever else is needed to handle
the job of installing Python modules and applications.


2.4 How installation works

how-build-works.html | standard-install.html | building-ext.html | Installing Python Modules | contents.html
---
## 2.4 How installation works
After the `build` command runs (whether you run it explicitly,
or the `install` command does it for you), the work of the
`install` command is relatively simple: all it has to do is copy
everything under build/lib (or build/lib.plat)
to your chosen installation directory.
If you don't choose an installation directory--i.e., if you just run
`setup.py install`--then the `install` command installs to
the standard location for third-party Python modules. This location
varies by platform and by how you built/installed Python itself. On
Unix and Mac OS, it also depends on whether the module distribution
being installed is pure Python or contains extensions (``non-pure''):
Notes:
(1): Most Linux distributions include Python as a standard part of
the system, so prefix and exec-prefix are usually
both /usr on Linux. If you build Python yourself on Linux (or
any Unix-like system), the default prefix and
exec-prefix are /usr/local.
(2): The default installation directory on Windows was
C:\Program Files\Python under
Python 1.6a1, 1.5.2, and earlier.
prefix and exec-prefix stand for the directories
that Python is installed to, and where it finds its libraries at
run-time. They are always the same under Windows and Mac OS, and very
often the same under Unix. You can find out what your Python
installation uses for prefix and exec-prefix by
running Python in interactive mode and typing a few simple commands.
Under Unix, just type `python` at the shell prompt; under Windows,
run ``Python 2.0 (interpreter)'' ** right? **; under Mac OS, ** ??? **.
Once the interpreter is started, you type Python code at the
"»> " prompt. For example, on my Linux system, I type the three
Python statements shown below, and get the output as shown, to find
out my prefix and exec-prefix:
```text

Python 1.5.2 (#1, Apr 18 1999, 16:03:16) [GCC pgcc-2.91.60 19981201 (egcs-1.1.1 on linux2
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> import sys
>>> sys.prefix
'/usr'
>>> sys.exec_prefix
'/usr'
```
If you don't want to install to the standard location, or if you don't
have permission to write there, then you need to read about alternate
installations in the next section.


Installing Python Modules

../index.html | contents.html | Installing Python Modules | contents.html
---
# Installing Python Modules
Greg Ward
E-mail: gward@python.net
### Abstract:
This document describes the Python Distribution Utilities
(``Distutils'') from the end-user's point-of-view, describing how to
extend the capabilities of a standard Python installation by building
and installing third-party Python modules and extensions.


Installing Python Modules


1 Introduction

contents.html | inst.html | trivial-install.html | Installing Python Modules | contents.html
---
# 1 Introduction
Although Python's extensive standard library covers many programming
needs, there often comes a time when you need to add some new
functionality to your Python installation in the form of third-party
modules. This might be necessary to support your own programming, or to
support an application that you want to use and that happens to be
written in Python.
In the past, there has been little support for adding third-party
modules to an existing Python installation. With the introduction of
the Python Distribution Utilities (Distutils for short) in Python 2.0,
this is starting to change. Not everything will change overnight,
though, so while this document concentrates on installing module
distributions that use the Distutils, we will also spend some time
dealing with the old ways.
This document is aimed primarily at the people who need to install
third-party Python modules: end-users and system administrators who just
need to get some Python application running, and existing Python
programmers who want to add some new goodies to their toolbox. You
don't need to know Python to read this document; there will be some
brief forays into using Python's interactive mode to explore your
installation, but that's it. If you're looking for information on how
to distribute your own Python modules so that others may use them, see
the Distributing Python Modules (../dist/dist.html) manual.


7.1 The Makefile.pre.in file

pre-distutils.html | pre-distutils.html | manual-install.html | Installing Python Modules | contents.html
---
## 7.1 The Makefile.pre.in file


7.2 Installing modules manually

makefile-pre-in.html | pre-distutils.html | about.html | Installing Python Modules | contents.html
---
## 7.2 Installing modules manually


1.2 The new standard: Distutils

trivial-install.html | intro.html | old-way.html | Installing Python Modules | contents.html
---
## 1.2 The new standard: Distutils
If you download a module source distribution, you can tell pretty
quickly if it was packaged and distributed in the standard way, i.e.
using the Distutils. First, the distribution's name and version number
will be featured prominently in the name of the downloaded archive, e.g.
foo-1.0.tar.gz or widget-0.9.7.zip. Next, the archive
will unpack into a similarly-named directory: foo-1.0 or
widget-0.9.7. Additionally, the distribution will contain a
setup script setup.py, and a README.txt (or possibly
README), which should explain that building and installing the
module distribution is a simple matter of running
```text

python setup.py install
```
If all these things are true, then you already know how to build and
install the modules you've just downloaded: run the command above.
Unless you need to install things in a non-standard way or customize the
build process, you don't really need this manual. Or rather, the above
command is everything you need to get out of this manual.


2.4 How installation works

node9.html | normal-install.html | alt-install.html | Installing Python Modules | contents.html
---
## 2.4 How installation works
After the `build` command runs (whether you run it explicitly,
or the `install` command does it for you), the work of the
`install` command is relatively simple: all it has to do is copy
everything under build/lib (or build/lib.plat)
to your chosen installation directory.
If you don't choose an installation directory--i.e., if you just run
`setup.py install`--then the `install` command installs to
the standard location for third-party Python modules. This location
varies by platform and by how you built/installed Python itself. On
Unix and Mac OS, it also depends on whether the module distribution
being installed is pure Python or contains extensions (``non-pure''):
Notes:
(1): Most Linux distributions include Python as a standard part of
the system, so prefix and exec-prefix are usually
both /usr on Linux. If you build Python yourself on Linux (or
any Unix-like system), the default prefix and
exec-prefix are /usr/local.
(2): The default installation directory on Windows was
C:\Program Files\Python under
Python 1.6a1, 1.5.2, and earlier.
prefix and exec-prefix stand for the directories
that Python is installed to, and where it finds its libraries at
run-time. They are always the same under Windows and Mac OS, and very
often the same under Unix. You can find out what your Python
installation uses for prefix and exec-prefix by
running Python in interactive mode and typing a few simple commands.
Under Unix, just type `python` at the shell prompt; under Windows,
run ``Python 2.0 (interpreter)'' ** right? **; under Mac OS, ** ??? **.
Once the interpreter is started, you type Python code at the
"»> " prompt. For example, on my Linux system, I type the three
Python statements shown below, and get the output as shown, to find
out my prefix and exec-prefix:
```text

Python 1.6 (#22, Sep 2 2000, 23:54:55) [GCC egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)] on linux2
Copyright (c) 1995-2000 Corporation for National Research Initiatives.
All Rights Reserved.
Copyright (c) 1991-1995 Stichting Mathematisch Centrum, Amsterdam.
All Rights Reserved.
>>> import sys
>>> sys.prefix
'/usr'
>>> sys.exec_prefix
'/usr'
```
If you don't want to install to the standard location, or if you don't
have permission to write there, then you need to read about alternate
installations in the next section.


2.1 Platform variations

normal-install.html | normal-install.html | node8.html | Installing Python Modules | contents.html
---
## 2.1 Platform variations
You should always run the setup command from the distribution root
directory, i.e. the top-level subdirectory that the module source
distribution unpacks into. For example, if you've just downloaded a
module source distribution foo-1.0.tar.gz onto a Unix system, the
normal thing to do is:
```text

gunzip -c foo-1.0.tar.gz | tar xf - # unpacks into directory foo-1.0
cd foo-1.0
python setup.py install
```
On Windows, you'd probably unpack the archive before opening the command
prompt. If you downloaded the archive file to
C:\Temp, then it probably unpacked (depending on
your software) into
C:\Temp\foo-1.0; from the command
prompt window, you would then run
```text

cd c:\temp\foo-1.0
python setup.py install
```
On Mac OS, ... ** again, how do you run Python scripts on Mac OS? **
** arg, my lovely ``bslash'' macro doesn't work in non-tt fonts! help
me LATEX, you're my only hope... **


2.2 Splitting the job up

node7.html | normal-install.html | node9.html | Installing Python Modules | contents.html
---
## 2.2 Splitting the job up
Running `setup.py install` builds and installs all modules in one
fell swoop. If you prefer to work incrementally--especially useful if
you want to customize the build process, or if things are going
wrong--you can use the setup script to do one thing at a time. This is
particularly helpful when the build and install will be done by
different users--e.g., you might want to build a module distribution
and hand it off to a system administrator for installation (or do it
yourself, with super-user privileges).
For example, you can build everything in one step, and then install
everything in a second step, by invoking the setup script twice:
```text

python setup.py build
python setup.py install
```
(If you do this, you will notice that running the `install`
command first runs the `build` command, which quickly notices
that it has nothing to do, since everything in the build
directory is up-to-date.)
** concrete reason for splitting things up? **


2.3 How building works

node8.html | normal-install.html | node10.html | Installing Python Modules | contents.html
---
## 2.3 How building works
As implied above, the `build` command is responsible for putting
the files to install into a build directory. By default, this is
build under the distribution root; if you're excessively
concerned with speed, or want to keep the source tree pristine, you can
change the build directory with the --build-base option.
For example:
```text

python setup.py build --build-base=/tmp/pybuild/foo-1.0
```
(Or you could do this permanently with a directive in your system or
personal Distutils configuration file; see
section 5 (config-files.html#config-files).) Normally, this isn't necessary.
The default layout for the build tree is as follows:
```text

--- build/ --- lib/
or
--- build/ --- lib.<plat>/
temp.<plat>/
```
where `<plat>` expands to a brief description of the current
OS/hardware platform. The first form, with just a lib directory,
is used for ``pure module distributions''--that is, module
distributions that include only pure Python modules. If a module
distribution contains any extensions (modules written in C/C++, or Java
for JPython), then the second form, with two `<plat>` directories,
is used. In that case, the temp.plat directory holds
temporary files generated by the compile/link process that don't
actually get installed. In either case, the lib (or
lib.plat) directory contains all Python modules (pure
Python and extensions) that will be installed.
In the future, more directories will be added to handle Python scripts,
documentation, binary executables, and whatever else is needed to handle
the job of installing Python modules and applications.


tweak-flags.html | building-ext.html | alt-install.html | Installing Python Modules | contents.html
---
## 3.2 Using non-Microsoft compilers on Windows
** One place to look: http://www.cyberus.ca/~g_will/pyExtenDL.shtml **


2 Standard Build and Install

old-way.html | inst.html | node7.html | Installing Python Modules | contents.html
---
# 2 Standard Build and Install
As described in section 1.2 (new-standard.html#new-standard), building and installing
a module distribution using the Distutils is usually one simple command:
```text

python setup.py install
```
On Unix, you'd run this command from a shell prompt; on Windows, you
have to open a command prompt window and do it there; on Mac OS ...
** what the heck do you do on Mac OS? **.


1.3 The old way: no standards

new-standard.html | intro.html | standard-install.html | Installing Python Modules | contents.html
---
## 1.3 The old way: no standards
Before the Distutils, there was no infrastructure to support installing
third-party modules in a consistent, standardized way. Thus, it's not
really possible to write a general manual for installing Python modules
that don't use the Distutils; the only truly general statement that can
be made is, ``Read the module's own installation instructions.''
However, if such instructions exist at all, they are often woefully
inadequate and targeted at experienced Python developers. Such users
are already familiar with how the Python library is laid out on their
platform, and know where to copy various files in order for Python to
find them. This document makes no such assumptions, and explains how
the Python library is laid out on three major platforms (Unix, Windows,
and Mac OS), so that you can understand what happens when the Distutils
do their job and know how to install modules manually when the
module author fails to provide a setup script.
Additionally, while there has not previously been a standard
installation mechanism, Python has had some standard machinery for
building extensions on Unix since Python ** version? **. This machinery
(the Makefile.pre.in file) is superseded by the Distutils, but it
will no doubt live on in older module distributions for a while. This
Makefile.pre.in mechanism is documented in the ``Extending &
Embedding Python'' manual, but that manual is aimed at module
developers--hence, we include documentation for builders/installers
here.
All of the pre-Distutils material is tucked away in
section 7 (pre-distutils.html#pre-distutils).


2.1 Platform variations

standard-install.html | standard-install.html | splitting-up.html | Installing Python Modules | contents.html
---
## 2.1 Platform variations
You should always run the setup command from the distribution root
directory, i.e. the top-level subdirectory that the module source
distribution unpacks into. For example, if you've just downloaded a
module source distribution foo-1.0.tar.gz onto a Unix system, the
normal thing to do is:
```text

gunzip -c foo-1.0.tar.gz | tar xf - # unpacks into directory foo-1.0
cd foo-1.0
python setup.py install
```
On Windows, you'd probably download foo-1.0.zip. If you
downloaded the archive file to C:\Temp, then it
would unpack into C:\Temp\foo-1.0;
you can use either a GUI archive manipulator (such as WinZip) or a
command-line tool (such as unzip or pkunzip) to
unpack the archive. Then, open a command prompt window (``DOS box''),
and run:
```text

cd c:\Temp\foo-1.0
python setup.py install
```
On Mac OS, you have to go through a bit more effort to supply
command-line arguments to the setup script:
- hit option-double-click on the script's icon (or option-drop it
onto the Python interpreter's icon)
- press the ``Set unix-style command line'' button
- set the ``Keep stdio window open on termination'' if you're
interested in seeing the output of the setup script (which is usually
voluminous and often useful)
- when the command-line dialog pops up, enter ``install'' (you
can, of course, enter any Distutils command-line as described in this
document or in the ``Distributing Python Modules'' document: just
leave of the initial `python setup.py` and you'll be fine)
** this should change: every Distutils setup script will need
command-line arguments for every run (and should probably keep stdout
around), so all this should happen automatically for setup scripts **


7 Pre-Distutils Conventions

config-files.html | inst.html | makefile-pre-in.html | Installing Python Modules | contents.html
---
# 7 Pre-Distutils Conventions


2.2 Splitting the job up

platform-variations.html | standard-install.html | how-build-works.html | Installing Python Modules | contents.html
---
## 2.2 Splitting the job up
Running `setup.py install` builds and installs all modules in one
run. If you prefer to work incrementally--especially useful if you
want to customize the build process, or if things are going wrong--you
can use the setup script to do one thing at a time. This is
particularly helpful when the build and install will be done by
different users--e.g., you might want to build a module distribution
and hand it off to a system administrator for installation (or do it
yourself, with super-user privileges).
For example, you can build everything in one step, and then install
everything in a second step, by invoking the setup script twice:
```text

python setup.py build
python setup.py install
```
(If you do this, you will notice that running the `install`
command first runs the `build` command, which--in this
case--quickly notices that it has nothing to do, since everything in
the build directory is up-to-date.)
You may not need this ability to break things down often if all you do
is install modules downloaded off the 'net, but it's very handy for more
advanced tasks. If you get into distributing your own Python modules
and extensions, you'll run lots of individual Distutils commands on
their own.


2 Standard Build and Install

old-way.html | inst.html | platform-variations.html | Installing Python Modules | contents.html
---
# 2 Standard Build and Install
As described in section 1.2 (new-standard.html#new-standard), building and installing
a module distribution using the Distutils is usually one simple command:
```text

python setup.py install
```
On Unix, you'd run this command from a shell prompt; on Windows, you
have to open a command prompt window (``DOS box'') and do it there; on
Mac OS, things are a tad more complicated (see below).