8.2 Discussion Forums

structured.html | futures.html | about.html | Documenting Python | contents.html
Previous:
8.1 Structured Documentation (structured.html)
Up:
8 Future Directions (futures.html)
Next:
About this document ... (about.html)
---
## 8.2 Discussion Forums
Discussion of the future of the Python documentation and related
topics takes place in the Documentation Special Interest Group, or
``Doc-SIG.'' Information on the group, including mailing list
archives and subscription information, is available at
http://www.python.org/sigs/doc-sig/. The SIG is open to all
interested parties.
Comments and bug reports on the standard documents should be sent
to python-docs@python.org. This may include comments
about formatting, content, grammatical and spelling errors, or
this document. You can also send comments on this document
directly to the author at fdrake@acm.org.


Documenting Python

../index.html | contents.html | Documenting Python | contents.html
Up:
Python Documentation Index (../index.html)
Next:
---
# Documenting Python
Fred L. Drake, Jr.
PythonLabs
Email: fdrake@acm.org
Release 2.3
July 29, 2003
### Abstract:
The Python language has a substantial body of
documentation, much of it contributed by various authors. The markup
used for the Python documentation is based on LATEX and requires a
significant set of macros written specifically for documenting Python.
This document describes the macros introduced to support Python
documentation and how they should be used to support a wide range of
output formats.
This document describes the document classes and special markup used
in the Python documentation. Authors may use this guide, in
conjunction with the template files provided with the
distribution, to create or maintain whole documents or sections.


8 Future Directions

cygwin.html | doc.html | structured.html | Documenting Python | contents.html
Previous:
7.3 Working on Cygwin (cygwin.html)
Up:
Documenting Python (doc.html)
Next:
8.1 Structured Documentation (structured.html)
---
# 8 Future Directions
The history of the Python documentation is full of changes, most of
which have been fairly small and evolutionary. There has been a
great deal of discussion about making large changes in the markup
languages and tools used to process the documentation. This section
deals with the nature of the changes and what appears to be the most
likely path of future development.


6.12 Grammar Production Displays

indexing.html | special-constructs.html | gui-markup.html | Documenting Python | contents.html
Previous:
6.11 Index-generating Markup (indexing.html)
Up:
6 Special Markup Constructs (special-constructs.html)
Next:
6.13 Graphical Interface Components (gui-markup.html)
---
## 6.12 Grammar Production Displays
Special markup is available for displaying the productions of a
formal grammar. The markup is simple and does not attempt to
model all aspects of BNF (or any derived forms), but provides
enough to allow context-free grammars to be displayed in a way
that causes uses of a symbol to be rendered as hyperlinks to the
definition of the symbol. There is one environment and a pair of
macros:
Note that the entire grammar does not need to be defined in a
single \productionlist environment; any number of
groupings may be used to describe the grammar. Every use of the
\token must correspond to a \production.
The following is an example taken from the
Python Reference Manual (../ref/identifiers.html):
```text

\begin{productionlist}
\production{identifier}
{(\token{letter}|"_") (\token{letter} | \token{digit} | "_")*}
\production{letter}
{\token{lowercase} | \token{uppercase}}
\production{lowercase}
{"a"..."z"}
\production{uppercase}
{"A"..."Z"}
\production{digit}
{"0"..."9"}
\end{productionlist}
```


6.13 Graphical Interface Components

grammar-displays.html | special-constructs.html | tools.html | Documenting Python | contents.html
Previous:
6.12 Grammar Production Displays (grammar-displays.html)
Up:
6 Special Markup Constructs (special-constructs.html)
Next:
7 Processing Tools (tools.html)
---
## 6.13 Graphical Interface Components
The components of graphical interfaces will be assigned markup, but
most of the specifics have not been determined.


Documenting Python


6.11 Index-generating Markup

references.html | special-constructs.html | grammar-displays.html | Documenting Python | contents.html
Previous:
6.10 Reference List Markup (references.html)
Up:
6 Special Markup Constructs (special-constructs.html)
Next:
6.12 Grammar Production Displays (grammar-displays.html)
---
## 6.11 Index-generating Markup
Effective index generation for technical documents can be very
difficult, especially for someone familiar with the topic but not
the creation of indexes. Much of the difficulty arises in the
area of terminology: including the terms an expert would use for a
concept is not sufficient. Coming up with the terms that a novice
would look up is fairly difficult for an author who, typically, is
an expert in the area she is writing on.
The truly difficult aspects of index generation are not areas with
which the documentation tools can help. However, ease
of producing the index once content decisions are made is within
the scope of the tools. Markup is provided which the processing
software is able to use to generate a variety of kinds of index
entry with minimal effort. Additionally, many of the environments
described in section 6.3 (info-units.html#info-units), ``Information Units,'' will
generate appropriate entries into the general and module indexes.
The following macro can be used to control the generation of index
data, and should be used in the document preamble:
There are a number of macros that are useful for adding index
entries for particular concepts, many of which are specific to
programming languages or even Python.
Additional macros are provided which are useful for conveniently
creating general index entries which should appear at many places
in the index by rotating a list of words. These are simple macros
that simply use \index to build some number of index
entries. Index entries build using these macros contain both
primary and secondary text.


6.3 Information Units

meta-info.html | special-constructs.html | showing-examples.html | Documenting Python | contents.html
Previous:
6.2 Meta-information Markup (meta-info.html)
Up:
6 Special Markup Constructs (special-constructs.html)
Next:
6.4 Showing Code Examples (showing-examples.html)
---
## 6.3 Information Units
XXX Explain terminology, or come up with something more ``lay.''
There are a number of environments used to describe specific
features provided by modules. Each environment requires
parameters needed to provide basic information about what is being
described, and the environment content should be the description.
Most of these environments make entries in the general index (if
one is being produced for the document); if no index entry is
desired, non-indexing variants are available for many of these
environments. The environments have names of the form
`feature desc`, and the non-indexing variants are named
`feature descni`. The available variants are explicitly
included in the list below.
For each of these environments, the first parameter, name,
provides the name by which the feature is accessed.
Environments which describe features of objects within a module,
such as object methods or data attributes, allow an optional
type name parameter. When the feature is an attribute of
class instances, type name only needs to be given if the
class was not the most recently described class in the module; the
name value from the most recent \classdesc is implied.
For features of built-in or extension types, the type name
value should always be provided. Another special case includes
methods and members of general ``protocols,'' such as the
formatter and writer protocols described for the
formatter module: these may be documented without any
specific implementation classes, and will always require the
type name parameter to be provided.


6.5 Inline Markup

showing-examples.html | special-constructs.html | misc-text-markup.html | Documenting Python | contents.html
Previous:
6.4 Showing Code Examples (showing-examples.html)
Up:
6 Special Markup Constructs (special-constructs.html)
Next:
6.6 Miscellaneous Text Markup (misc-text-markup.html)
---
## 6.5 Inline Markup
The macros described in this section are used to mark just about
anything interesting in the document text. They may be used in
headings (though anything involving hyperlinks should be avoided
there) as well as in the body text.


1 Introduction

contents.html | doc.html | directories.html | Documenting Python | contents.html
Previous:
Up:
Documenting Python (doc.html)
Next:
2 Directory Structure (directories.html)
---
# 1 Introduction
Python's documentation has long been considered to be good for a
free programming language. There are a number of reasons for this,
the most important being the early commitment of Python's creator,
Guido van Rossum, to providing documentation on the language and its
libraries, and the continuing involvement of the user community in
providing assistance for creating and maintaining documentation.
The involvement of the community takes many forms, from authoring to
bug reports to just plain complaining when the documentation could
be more complete or easier to use. All of these forms of input from
the community have proved useful during the time I've been involved
in maintaining the documentation.
This document is aimed at authors and potential authors of
documentation for Python. More specifically, it is for people
contributing to the standard documentation and developing additional
documents using the same tools as the standard documents. This
guide will be less useful for authors using the Python documentation
tools for topics other than Python, and less useful still for
authors not using the tools at all.
The material in this guide is intended to assist authors using the
Python documentation tools. It includes information on the source
distribution of the standard documentation, a discussion of the
document types, reference material on the markup defined in the
document classes, a list of the external tools needed for processing
documents, and reference material on the tools provided with the
documentation resources. At the end, there is also a section
discussing future directions for the Python documentation and where
to turn for more information.


4 LATEX Primer

style-guide.html | doc.html | latex-syntax.html | Documenting Python | contents.html
Previous:
3 Style Guide (style-guide.html)
Up:
Documenting Python (doc.html)
Next:
4.1 Syntax (latex-syntax.html)
---
# 4 LATEX Primer
This section is a brief introduction to LATEX concepts and
syntax, to provide authors enough information to author documents
productively without having to become ``TEXnicians.''
Perhaps the most important concept to keep in mind while marking up
Python documentation is that while TEX is unstructured, LATEX was
designed as a layer on top of TEX which specifically supports
structured markup. The Python-specific markup is intended to extend
the structure provided by standard LATEX document classes to
support additional information specific to Python.
LATEX documents contain two parts: the preamble and the body.
The preamble is used to specify certain metadata about the document
itself, such as the title, the list of authors, the date, and the
class the document belongs to. Additional information used
to control index generation and the use of bibliographic databases
can also be placed in the preamble. For most authors, the preamble
can be most easily created by copying it from an existing document
and modifying a few key pieces of information.
The class of a document is used to place a document within a
broad category of documents and set some fundamental formatting
properties. For Python documentation, two classes are used: the
`manual` class and the `howto` class. These classes also
define the additional markup used to document Python concepts and
structures. Specific information about these classes is provided in
section 5 (classes.html#classes), ``Document Classes,'' below. The first thing
in the preamble is the declaration of the document's class.
After the class declaration, a number of macros are used to
provide further information about the document and setup any
additional markup that is needed. No output is generated from the
preamble; it is an error to include free text in the preamble
because it would cause output.
The document body follows the preamble. This contains all the
printed components of the document marked up structurally. Generic
LATEX structures include hierarchical sections, numbered and
bulleted lists, and special structures for the document abstract and
indexes.


4.2 Hierarchical Structure

latex-syntax.html | latex-primer.html | classes.html | Documenting Python | contents.html
Previous:
4.1 Syntax (latex-syntax.html)
Up:
4 LATEX Primer (latex-primer.html)
Next:
5 Document Classes (classes.html)
---
## 4.2 Hierarchical Structure
LATEX expects documents to be arranged in a conventional,
hierarchical way, with chapters, sections, sub-sections,
appendixes, and the like. These are marked using macros rather
than environments, probably because the end of a section can be
safely inferred when a section of equal or higher level starts.
There are six ``levels'' of sectioning in the document classes
used for Python documentation, and the deepest two
levels1 (#foot181) are not used. The levels are:
Notes:
(1): Only used for the `manual` documents, as described in
section 5 (classes.html#classes), ``Document Classes.''
(2): Not the same as a paragraph of text; nobody seems to use this.


4.1 Syntax

latex-primer.html | latex-primer.html | latex-structure.html | Documenting Python | contents.html
Previous:
4 LATEX Primer (latex-primer.html)
Up:
4 LATEX Primer (latex-primer.html)
Next:
4.2 Hierarchical Structure (latex-structure.html)
---
## 4.1 Syntax
There are some things that an author of Python documentation needs
to know about LATEX syntax.
A comment is started by the ``percent'' character
("%") and continues through the end of the line and all
leading whitespace on the following line. This is a little
different from any programming language I know of, so an example
is in order:
```text

This is text.% comment
This is more text. % another comment
Still more text.
```
The first non-comment character following the first comment is the
letter "T" on the second line; the leading whitespace on
that line is consumed as part of the first comment. This means
that there is no space between the first and second sentences, so
the period and letter "T" will be directly adjacent in
the typeset document.
Note also that though the first non-comment character after the
second comment is the letter "S", there is whitespace
preceding the comment, so the two sentences are separated as
expected.
A group is an enclosure for a collection of text and
commands which encloses the formatting context and constrains the
scope of any changes to that context made by commands within the
group. Groups can be nested hierarchically. The formatting
context includes the font and the definition of additional macros
(or overrides of macros defined in outer groups). Syntactically,
groups are enclosed in braces:
```text

{text in a group}
```
An alternate syntax for a group using brackets, `[...]`, is
used by macros and environment constructors which take optional
parameters; brackets do not normally hold syntactic significance.
A degenerate group, containing only one atomic bit of content,
does not need to have an explicit group, unless it is required to
avoid ambiguity. Since Python tends toward the explicit, groups
are also made explicit in the documentation markup.
Groups are used only sparingly in the Python documentation, except
for their use in marking parameters to macros and environments.
A macro is usually a simple construct which is identified by
name and can take some number of parameters. In normal LATEX
usage, one of these can be optional. The markup is introduced
using the backslash character ("\"), and the name is
given by alphabetic characters (no digits, hyphens, or
underscores). Required parameters should be marked as a group,
and optional parameters should be marked using the alternate
syntax for a group.
For example, a macro named ``foo'' which takes a single parameter
would appear like this:
```text

\name{parameter}
```
A macro which takes an optional parameter would be typed like this
when the optional paramter is given:
```text

\name[optional]
```
If both optional and required parameters are to be required, it
looks like this:
```text

\name[optional]{required}
```
A macro name may be followed by a space or newline; a space
between the macro name and any parameters will be consumed, but
this usage is not practiced in the Python documentation. Such a
space is still consumed if there are no parameters to the macro,
in which case inserting an empty group (`{}`) or explicit
word space ("\ ") immediately after the macro name helps to
avoid running the expansion of the macro into the following text.
Macros which take no parameters but which should not be followed
by a word space do not need special treatment if the following
character in the document source if not a name character (such as
punctuation).
Each line of this example shows an appropriate way to write text
which includes a macro which takes no parameters:
```text

This \UNIX{} is followed by a space.
This \UNIX\ is also followed by a space.
\UNIX, followed by a comma, needs no additional markup.
```
An environment is a larger construct than a macro, and can
be used for things with more content than would conveniently fit
in a macro parameter. They are primarily used when formatting
parameters need to be changed before and after a large chunk of
content, but the content itself needs to be highly flexible. Code
samples are presented using an environment, and descriptions of
functions, methods, and classes are also marked using environments.
Since the content of an environment is free-form and can consist
of several paragraphs, they are actually marked using a pair of
macros: \begin and \end. These macros both take the
name of the environment as a parameter. An example is the
environment used to mark the abstract of a document:
```text

\begin{abstract}
This is the text of the abstract. It concisely explains what
information is found in the document.

It can consist of multiple paragraphs.
\end{abstract}
```
An environment can also have required and optional parameters of
its own. These follow the parameter of the \begin macro.
This example shows an environment which takes a single required
parameter:
```text

\begin{datadesc}{controlnames}
A 33-element string array that contains the \ASCII{} mnemonics for
the thirty-two \ASCII{} control characters from 0 (NUL) to 0x1f
(US), in order, plus the mnemonic \samp{SP} for the space character.
\end{datadesc}
```
There are a number of less-used marks in LATEX which are used
to enter characters which are not found in ASCII or which a
considered special, or active in TEX or LATEX. Given
that these are often used adjacent to other characters, the markup
required to produce the proper character may need to be followed
by a space or an empty group, or the markup can be enclosed in a
group. Some which are found in Python documentation are:


6.8 Library-level Markup

module-markup.html | special-constructs.html | table-markup.html | Documenting Python | contents.html
Previous:
6.7 Module-specific Markup (module-markup.html)
Up:
6 Special Markup Constructs (special-constructs.html)
Next:
6.9 Table Markup (table-markup.html)
---
## 6.8 Library-level Markup
This markup is used when describing a selection of modules. For
example, the Macintosh Library
Modules (../mac/mac.html) document uses this to help provide an overview of the
modules in the collection, and many chapters in the
Python Library Reference (../lib/lib.html) use it for
the same purpose.


6.2 Meta-information Markup

preamble-info.html | special-constructs.html | info-units.html | Documenting Python | contents.html
Previous:
6.1 Markup for the (preamble-info.html)
Up:
6 Special Markup Constructs (special-constructs.html)
Next:
6.3 Information Units (info-units.html)
---
## 6.2 Meta-information Markup


6.6 Miscellaneous Text Markup

inline-markup.html | special-constructs.html | module-markup.html | Documenting Python | contents.html
Previous:
6.5 Inline Markup (inline-markup.html)
Up:
6 Special Markup Constructs (special-constructs.html)
Next:
6.7 Module-specific Markup (module-markup.html)
---
## 6.6 Miscellaneous Text Markup
In addition to the inline markup, some additional ``block'' markup
is defined to make it easier to bring attention to various bits of
text. The markup described here serves this purpose, and is
intended to be used when marking one or more paragraphs or other
block constructs (such as \verbatim environments).


6.7 Module-specific Markup

misc-text-markup.html | special-constructs.html | library-markup.html | Documenting Python | contents.html
Previous:
6.6 Miscellaneous Text Markup (misc-text-markup.html)
Up:
6 Special Markup Constructs (special-constructs.html)
Next:
6.8 Library-level Markup (library-markup.html)
---
## 6.7 Module-specific Markup
The markup described in this section is used to provide information
about a module being documented. A typical use of this markup
appears at the top of the section used to document a module. A
typical example might look like this:
```text

\section{\module{spam} ---
Access to the SPAM facility}

\declaremodule{extension}{spam}
\platform{Unix}
\modulesynopsis{Access to the SPAM facility of \UNIX.}
\moduleauthor{Jane Doe}{jane.doe@frobnitz.org}
```
Python packages-- collections of modules that can
be described as a unit -- are documented using the same markup as
modules. The name for a module in a package should be typed in
``fully qualified'' form (it should include the package name).
For example, a module ``foo'' in package ``bar'' should be marked as
`\module{bar.foo}`, and the beginning of the reference
section would appear as:
```text

\section{\module{bar.foo} ---
Module from the \module{bar} package}

\declaremodule{extension}{bar.foo}
\modulesynopsis{Nifty module from the \module{bar} package.}
\moduleauthor{Jane Doe}{jane.doe@frobnitz.org}
```
Note that the name of a package is also marked using
\module.


6.1 Markup for the Preamble

special-constructs.html | special-constructs.html | meta-info.html | Documenting Python | contents.html
Previous:
6 Special Markup Constructs (special-constructs.html)
Up:
6 Special Markup Constructs (special-constructs.html)
Next:
6.2 Meta-information Markup (meta-info.html)
---
## 6.1 Markup for the Preamble


6.10 Reference List Markup

table-markup.html | special-constructs.html | indexing.html | Documenting Python | contents.html
Previous:
6.9 Table Markup (table-markup.html)
Up:
6 Special Markup Constructs (special-constructs.html)
Next:
6.11 Index-generating Markup (indexing.html)
---
## 6.10 Reference List Markup
Many sections include a list of references to module documentation
or external documents. These lists are created using the
\seealso or \seealso* environments. These environments
define some additional macros to support creating reference
entries in a reasonable manner.
The \seealso environment is typically placed in a section
just before any sub-sections. This is done to ensure that
reference links related to the section are not hidden in a
subsection in the hypertext renditions of the documentation. For
the HTML output, it is shown as a ``side bar,'' boxed off from the
main flow of the text. The \seealso* environment is
different in that it should be used when a list of references is
being presented as part of the primary content; it is not
specially set off from the text.
For each of the following macros, why should be one or more
complete sentences, starting with a capital letter (unless it
starts with an identifier, which should not be modified), and
ending with the apropriate punctuation.
These macros are only defined within the content of the
\seealso and \seealso* environments.


6.4 Showing Code Examples

info-units.html | special-constructs.html | inline-markup.html | Documenting Python | contents.html
Previous:
6.3 Information Units (info-units.html)
Up:
6 Special Markup Constructs (special-constructs.html)
Next:
6.5 Inline Markup (inline-markup.html)
---
## 6.4 Showing Code Examples
Examples of Python source code or interactive sessions are
represented as \verbatim environments. This environment
is a standard part of LATEX. It is important to only use
spaces for indentation in code examples since TEX drops tabs
instead of converting them to spaces.
Representing an interactive session requires including the prompts
and output along with the Python code. No special markup is
required for interactive sessions. After the last line of input
or output presented, there should not be an ``unused'' primary
prompt; this is an example of what not to do:
```text

>>> 1 + 1
2
>>>
```
Within the \verbatim environment, characters special to
LATEX do not need to be specially marked in any way. The entire
example will be presented in a monospaced font; no attempt at
``pretty-printing'' is made, as the environment must work for
non-Python code and non-code displays. There should be no blank
lines at the top or bottom of any \verbatim display.
Longer displays of verbatim text may be included by storing the
example text in an external file containing only plain text. The
file may be included using the standard \verbatiminput
macro; this macro takes a single argument naming the file
containing the text. For example, to include the Python source
file example.py, use:
```text

\verbatiminput{example.py}
```
Use of \verbatiminput allows easier use of special editing
modes for the included file. The file should be placed in the
same directory as the LATEX files for the document.
The Python Documentation Special Interest Group has discussed a
number of approaches to creating pretty-printed code displays and
interactive sessions; see the Doc-SIG area on the Python Web site
for more information on this topic.


6 Special Markup Constructs

classes.html | doc.html | preamble-info.html | Documenting Python | contents.html
Previous:
5 Document Classes (classes.html)
Up:
Documenting Python (doc.html)
Next:
6.1 Markup for the (preamble-info.html)
---
# 6 Special Markup Constructs
The Python document classes define a lot of new environments and
macros. This section contains the reference material for these
facilities.


8.1 Structured Documentation

futures.html | futures.html | discussion.html | Documenting Python | contents.html
Previous:
8 Future Directions (futures.html)
Up:
8 Future Directions (futures.html)
Next:
8.2 Discussion Forums (discussion.html)
---
## 8.1 Structured Documentation
Most of the small changes to the LATEX markup have been made
with an eye to divorcing the markup from the presentation, making
both a bit more maintainable. Over the course of 1998, a large
number of changes were made with exactly this in mind; previously,
changes had been made but in a less systematic manner and with
more concern for not needing to update the existing content. The
result has been a highly structured and semantically loaded markup
language implemented in LATEX. With almost no basic TEX or
LATEX markup in use, however, the markup syntax is about the
only evidence of LATEX in the actual document sources.
One side effect of this is that while we've been able to use
standard ``engines'' for manipulating the documents, such as
LATEX and LATEX2HTML, most of the actual transformations have
been created specifically for Python. The LATEX document
classes and LATEX2HTML support are both complete implementations
of the specific markup designed for these documents.
Combining highly customized markup with the somewhat esoteric
systems used to process the documents leads us to ask some
questions: Can we do this more easily? and, Can we do this
better? After a great deal of discussion with the community, we
have determined that actively pursuing modern structured
documentation systems is worth some investment of time.
There appear to be two real contenders in this arena: the Standard
General Markup Language (SGML), and the Extensible Markup Language
(XML). Both of these standards have advantages and disadvantages,
and many advantages are shared.
SGML offers advantages which may appeal most to authors,
especially those using ordinary text editors. There are also
additional abilities to define content models. A number of
high-quality tools with demonstrated maturity are available, but
most are not free; for those which are, portability issues remain
a problem.
The advantages of XML include the availability of a large number
of evolving tools. Unfortunately, many of the associated
standards are still evolving, and the tools will have to follow
along. This means that developing a robust tool set that uses
more than the basic XML 1.0 recommendation is not possible in the
short term. The promised availability of a wide variety of
high-quality tools which support some of the most important
related standards is not immediate. Many tools are likely to be
free, and the portability issues of those which are, are not
expected to be significant.
It turns out that converting to an XML or SGML system holds
promise for translators as well; how much can be done to ease the
burden on translators remains to be seen, and may have some impact
on the schema and specific technologies used.
XXX Eventual migration to XML.
The documentation will be moved to XML in the future, and tools
are being written which will convert the documentation from the
current format to something close to a finished version, to the
extent that the desired information is already present in the
documentation. Some XSLT stylesheets have been started for
presenting a preliminary XML version as HTML, but the results are
fairly rough.
The timeframe for the conversion is not clear since there doesn't
seem to be much time available to work on this, but the appearant
benefits are growing more substantial at a moderately rapid pace.


3 Style Guide

directories.html | doc.html | latex-primer.html | Documenting Python | contents.html
Previous:
2 Directory Structure (directories.html)
Up:
Documenting Python (doc.html)
Next:
4 LATEX Primer (latex-primer.html)
---
# 3 Style Guide
The Python documentation should follow the Apple Publications Style Guide (http://developer.apple.com/documentation/UserExperience/Conceptual/APStyleGuide/AppleStyleGuide2003.pdf) wherever possible. This particular
style guide was selected mostly because it seems reasonable and is
easy to get online.
Topics which are not covered in the Apple's style guide will be
discussed in this document if necessary.
Many special names are used in the Python documentation, including
the names of operating systems, programming languages, standards
bodies, and the like. Many of these were assigned LATEX macros
at some point in the distant past, and these macros lived on long
past their usefulness. In the current markup, most of these entities
are not assigned any special markup, but the preferred spellings are
given here to aid authors in maintaining the consistency of
presentation in the Python documentation.
Other terms and words deserve special mention as well; these conventions
should be used to ensure consistency throughout the documentation:
CPU: For ``central processing unit.'' Many style guides say this
should be spelled out on the first use (and if you must use it,
do so!). For the Python documentation, this abbreviation should
be avoided since there's no reasonable way to predict which occurance
will be the first seen by the reader. It is better to use the
word ``processor'' instead.
POSIX: The name assigned to a particular group of standards. This is
always uppercase. Use the macro \POSIX to represent this
name.
Python: The name of our favorite programming language is always
capitalized.
Unicode: The name of a character set and matching encoding. This is
always written capitalized.
Unix: The name of the operating system developed at AT&T Bell Labs
in the early 1970s. Use the macro \UNIX to use this
name.


6.9 Table Markup

library-markup.html | special-constructs.html | references.html | Documenting Python | contents.html
Previous:
6.8 Library-level Markup (library-markup.html)
Up:
6 Special Markup Constructs (special-constructs.html)
Next:
6.10 Reference List Markup (references.html)
---
## 6.9 Table Markup
There are three general-purpose table environments defined which
should be used whenever possible. These environments are defined
to provide tables of specific widths and some convenience for
formatting. These environments are not meant to be general
replacements for the standard LATEX table environments, but can
be used for an advantage when the documents are processed using
the tools for Python documentation processing. In particular, the
generated HTML looks good! There is also an advantage for the
eventual conversion of the documentation to XML (see section
8 (futures.html#futures), ``Future Directions'').
Each environment is named \tablecols, where cols
is the number of columns in the table specified in lower-case
Roman numerals. Within each of these environments, an additional
macro, \linecols, is defined, where cols
matches the cols value of the corresponding table
environment. These are supported for cols values of
`ii`, `iii`, and `iv`. These environments are all
built on top of the \tabular environment. Variants based on
the \longtable environment are also provided.
Note that all tables in the standard Python documentation use
vertical lines between columns, and this must be specified in the
markup for each table. A general border around the outside of the
table is not used, but would be the responsibility of the
processor; the document markup should not include an exterior
border.
The \longtable-based variants of the table environments are
formatted with extra space before and after, so should only be
used on tables which are long enough that splitting over multiple
pages is reasonable; tables with fewer than twenty rows should
never by marked using the long flavors of the table environments.
The header row is repeated across the top of each part of the
table.
An additional table-like environment is \synopsistable. The
table generated by this environment contains two columns, and each
row is defined by an alternate definition of
\modulesynopsis. This environment is not normally used by
authors, but is created by the \localmoduletable macro.
Here is a small example of a table given in the documentation for
the warnings module; markup inside the table cells is
minimal so the markup for the table itself is readily discernable.
Here is the markup for the table:
```text

\begin{tableii}{l|l}{exception}{Class}{Description}
\lineii{Warning}
{This is the base class of all warning category classes. It
is a subclass of \exception{Exception}.}
\lineii{UserWarning}
{The default category for \function{warn()}.}
\lineii{DeprecationWarning}
{Base category for warnings about deprecated features.}
\lineii{SyntaxWarning}
{Base category for warnings about dubious syntactic
features.}
\lineii{RuntimeWarning}
{Base category for warnings about dubious runtime features.}
\lineii{FutureWarning}
{Base category for warnings about constructs that will change
semantically in the future.}
\end{tableii}
```
Here is the resulting table:
Note that the class names are implicitly marked using the
\exception macro, since that is given as the col1font
value for the \tableii environment. To create a table using
different markup for the first column, use `textrm` for the
col1font value and mark each entry individually.
To add a horizontal line between vertical sections of a table, use
the standard \hline macro between the rows which should be
separated:
```text

\begin{tableii}{l|l}{constant}{Language}{Audience}
\lineii{APL}{Masochists.}
\lineii{BASIC}{First-time programmers on PC hardware.}
\lineii{C}{\UNIX{} \&\ Linux kernel developers.}
\hline
\lineii{Python}{Everyone!}
\end{tableii}
```
Note that not all presentation formats are capable of displaying a
horizontal rule in this position. This is how the table looks in
the format you're reading now:


7.1 External Tools

tools.html | tools.html | tools-internal.html | Documenting Python | contents.html
Previous:
7 Processing Tools (tools.html)
Up:
7 Processing Tools (tools.html)
Next:
7.2 Internal Tools (tools-internal.html)
---
## 7.1 External Tools
Many tools are needed to be able to process the Python
documentation if all supported formats are required. This
section lists the tools used and when each is required. Consult
the Doc/README file to see if there are specific version
requirements for any of these.
dvips: This program is a typical part of TEX installations. It is
used to generate PostScript from the ``device independent''
.dvi files. It is needed for the conversion to
PostScript.
emacs: Emacs is the kitchen sink of programmers' editors, and a damn
fine kitchen sink it is. It also comes with some of the
processing needed to support the proper menu structures for
Texinfo documents when an info conversion is desired. This is
needed for the info conversion. Using xemacs
instead of FSF emacs may lead to instability in the
conversion, but that's because nobody seems to maintain the
Emacs Texinfo code in a portable manner.
latex: LATEX is a large and extensible macro package by Leslie
Lamport, based on TEX, a world-class typesetter by Donald
Knuth. It is used for the conversion to PostScript, and is
needed for the HTML conversion as well (LATEX2HTML requires
one of the intermediate files it creates).
latex2html: Probably the longest Perl script anyone ever attempted to
maintain. This converts LATEX documents to HTML documents,
and does a pretty reasonable job. It is required for the
conversions to HTML and GNU info.
lynx: This is a text-mode Web browser which includes an
HTML-to-plain text conversion. This is used to convert
`howto` documents to text.
make: Just about any version should work for the standard documents,
but GNU make is required for the experimental
processes in Doc/tools/sgmlconv/, at least while
they're experimental. This is not required for running the
mkhowto script.
makeindex: This is a standard program for converting LATEX index data
to a formatted index; it should be included with all LATEX
installations. It is needed for the PDF and PostScript
conversions.
makeinfo: GNU makeinfo is used to convert Texinfo documents to
GNU info files. Since Texinfo is used as an intermediate
format in the info conversion, this program is needed in that
conversion.
pdflatex: pdfTEX is a relatively new variant of TEX, and is used to
generate the PDF version of the manuals. It is typically
installed as part of most of the large TEX distributions.
pdflatex is pdfTEX using the LATEX format.
perl: Perl is required for LATEX2HTML and one of the scripts used
to post-process LATEX2HTML output, as well as the
HTML-to-Texinfo conversion. This is required for
the HTML and GNU info conversions.
python: Python is used for many of the scripts in the
Doc/tools/ directory; it is required for all
conversions. This shouldn't be a problem if you're interested
in writing documentation for Python!


7.2 Internal Tools

tools-external.html | tools.html | cygwin.html | Documenting Python | contents.html
Previous:
7.1 External Tools (tools-external.html)
Up:
7 Processing Tools (tools.html)
Next:
7.3 Working on Cygwin (cygwin.html)
---
## 7.2 Internal Tools
This section describes the various scripts that are used to
implement various stages of document processing or to orchestrate
entire build sequences. Most of these tools are only useful
in the context of building the standard documentation, but some
are more general.
mkhowto: This is the primary script used to format third-party
documents. It contains all the logic needed to ``get it
right.'' The proper way to use this script is to make a
symbolic link to it or run it in place; the actual script file
must be stored as part of the documentation source tree,
though it may be used to format documents outside the
tree. Use mkhowto --help
for a list of
command line options.
mkhowto can be used for both `howto` and
`manual` class documents. It is usually a good idea to
always use the latest version of this tool rather than a
version from an older source release of Python.
XXX Need more here.


7 Processing Tools

gui-markup.html | doc.html | tools-external.html | Documenting Python | contents.html
Previous:
6.13 Graphical Interface Components (gui-markup.html)
Up:
Documenting Python (doc.html)
Next:
7.1 External Tools (tools-external.html)
---
# 7 Processing Tools


About this document ...

node50.html | ext.html | Extending and Embedding the Python Interpreter | contents.html
Previous:
B.2 Terms and conditions (node50.html)
Up:
Extending and Embedding the (ext.html)
---
# About this document ...
Extending and Embedding the Python Interpreter,
July 29, 2003, Release 2.3
This document was generated using the LaTeX2HTML (http://saftsack.fs.uni-bayreuth.de/~latex2ht/) translator.
LaTeX2HTML (http://saftsack.fs.uni-bayreuth.de/~latex2ht/) is Copyright ©
1993, 1994, 1995, 1996, 1997, Nikos
Drakos (http://cbl.leeds.ac.uk/nikos/personal.html), Computer Based Learning Unit, University of
Leeds, and Copyright © 1997, 1998, Ross
Moore (http://www.maths.mq.edu.au/~ross/), Mathematics Department, Macquarie University,
Sydney.
The application of LaTeX2HTML (http://saftsack.fs.uni-bayreuth.de/~latex2ht/) to the Python
documentation has been heavily tailored by Fred L. Drake,
Jr. Original navigation icons were contributed by Christopher
Petrilli.
---
## Comments and Questions
General comments and questions regarding this document should
be sent by email to python-docs@python.org (mailto:python-docs@python.org). If you find specific errors in
this document, either in the content or the presentation, please
report the bug at the Python Bug
Tracker (http://sourceforge.net/bugs/?group_id=5470) at SourceForge (http://sourceforge.net/).
Questions regarding how to use the information in this
document should be sent to the Python news group, comp.lang.python (news:comp.lang.python), or the Python mailing list (http://www.python.org/mailman/listinfo/python-list) (which is gated to the newsgroup and
carries the same content).
For any of these channels, please be sure not to send HTML email.
Thanks.


1.3 Back to the Example

errors.html | intro.html | methodTable.html | Extending and Embedding the Python Interpreter | contents.html
Previous:
1.2 Intermezzo: Errors and (errors.html)
Up:
1. Extending Python with (intro.html)
Next:
1.4 The Module's Method (methodTable.html)
---
# 1.3 Back to the Example
Going back to our example function, you should now be able to
understand this statement:
```text

if (!PyArg_ParseTuple(args, "s", &command))
return NULL;
```
It returns NULL (the error indicator for functions returning
object pointers) if an error is detected in the argument list, relying
on the exception set by PyArg_ParseTuple(). Otherwise the
string value of the argument has been copied to the local variable
command. This is a pointer assignment and you are not supposed
to modify the string to which it points (so in Standard C, the variable
command should properly be declared as "const char
*command").
The next statement is a call to the Unix function
system(), passing it the string we just got from
PyArg_ParseTuple():
```text

sts = system(command);
```
Our spam.system() function must return the value of
sts as a Python object. This is done using the function
Py_BuildValue(), which is something like the inverse of
PyArg_ParseTuple(): it takes a format string and an
arbitrary number of C values, and returns a new Python object.
More info on Py_BuildValue() is given later.
```text

return Py_BuildValue("i", sts);
```
In this case, it will return an integer object. (Yes, even integers
are objects on the heap in Python!)
If you have a C function that returns no useful argument (a function
returning void), the corresponding Python function must return
`None`. You need this idiom to do so:
```text

Py_INCREF(Py_None);
return Py_None;
```
Py_None is the C name for the special Python object
`None`. It is a genuine Python object rather than a NULL
pointer, which means ``error'' in most contexts, as we have seen.


distributing.html | ext.html | win-cookbook.html | Extending and Embedding the Python Interpreter | contents.html
Previous:
3.1 Distributing your extension (distributing.html)
Up:
Extending and Embedding the (ext.html)
Next:
4.1 A Cookbook Approach (win-cookbook.html)
---
# 4. Building C and C++ Extensions on Windows
This chapter briefly explains how to create a Windows extension module
for Python using Microsoft Visual C++, and follows with more
detailed background information on how it works. The explanatory
material is useful for both the Windows programmer learning to build
Python extensions and the Unix programmer interested in producing
software which can be successfully built on both Unix and Windows.
Module authors are encouraged to use the distutils approach for
building extension modules, instead of the one described in this
section. You will still need the C compiler that was used to build
Python; typically Microsoft Visual C++.
Note:
This chapter mentions a number of filenames that include an encoded
Python version number. These filenames are represented with the
version number shown as "XY"; in practive, "X" will
be the major version number and "Y" will be the minor
version number of the Python release you're working with. For
example, if you are using Python 2.2.1, "XY" will actually be
"22".


node33.html | ext.html | distributing.html | Extending and Embedding the Python Interpreter | contents.html
Previous:
2.2.6 More Suggestions (node33.html)
Up:
Extending and Embedding the (ext.html)
Next:
3.1 Distributing your extension (distributing.html)
---
# 3. Building C and C++ Extensions with distutils
Starting in Python 1.4, Python provides, on Unix, a special make
file for building make files for building dynamically-linked
extensions and custom interpreters. Starting with Python 2.0, this
mechanism (known as related to Makefile.pre.in, and Setup files) is no
longer supported. Building custom interpreters was rarely used, and
extension modules can be built using distutils.
Building an extension module using distutils requires that distutils
is installed on the build machine, which is included in Python 2.x and
available separately for Python 1.5. Since distutils also supports
creation of binary packages, users don't necessarily need a compiler
and distutils to install the extension.
A distutils package contains a driver script, setup.py. This is
a plain Python file, which, in the most simple case, could look like
this:
```text

from distutils.core import setup, Extension

module1 = Extension('demo',
sources = ['demo.c'])

setup (name = 'PackageName',
version = '1.0',
description = 'This is a demo package',
ext_modules = [module1])
```
With this setup.py, and a file demo.c, running
```text

python setup.py build
```
will compile demo.c, and produce an extension module named
"demo" in the build directory. Depending on the system,
the module file will end up in a subdirectory build/lib.system,
and may have a name like demo.so or demo.pyd.
In the setup.py, all execution is performed by calling the
"setup" function. This takes a variable number of keyword
arguments, of which the example above uses only a
subset. Specifically, the example specifies meta-information to build
packages, and it specifies the contents of the package. Normally, a
package will contain of addition modules, like Python source modules,
documentation, subpackages, etc. Please refer to the distutils
documentation in Distributing Python
Modules (../dist/dist.html) to learn more about the features of distutils; this section
explains building extension modules only.
It is common to pre-compute arguments to setup, to better
structure the driver script. In the example above,
the"ext_modules" argument to setup is a list of
extension modules, each of which is an instance of the
Extension. In the example, the instance defines an extension
named "demo" which is build by compiling a single source file,
demo.c.
In many cases, building an extension is more complex, since additional
preprocessor defines and libraries may be needed. This is demonstrated
in the example below.
```text

from distutils.core import setup, Extension

module1 = Extension('demo',
define_macros = [('MAJOR_VERSION', '1'),
('MINOR_VERSION', '0')],
include_dirs = ['/usr/local/include'],
libraries = ['tcl83'],
library_dirs = ['/usr/local/lib'],
sources = ['demo.c'])

setup (name = 'PackageName',
version = '1.0',
description = 'This is a demo package',
author = 'Martin v. Loewis',
author_email = 'martin@v.loewis.de',
url = 'http://www.python.org/doc/current/ext/building.html',
long_description = '''
This is really just a demo package.
''',
ext_modules = [module1])
```
In this example, setup is called with additional
meta-information, which is recommended when distribution packages have
to be built. For the extension itself, it specifies preprocessor
defines, include directories, library directories, and libraries.
Depending on the compiler, distutils passes this information in
different ways to the compiler. For example, on Unix, this may
result in the compilation commands
```text

gcc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -DMAJOR_VERSION=1 -DMINOR_VERSION=0 -I/usr/local/include -I/usr/local/include/python2.2 -c demo.c -o build/temp.linux-i686-2.2/demo.o

gcc -shared build/temp.linux-i686-2.2/demo.o -L/usr/local/lib -ltcl83 -o build/lib.linux-i686-2.2/demo.so
```
These lines are for demonstration purposes only; distutils users
should trust that distutils gets the invocations right.


1.9 Building Arbitrary Values

parseTupleAndKeywords.html | intro.html | refcounts.html | Extending and Embedding the Python Interpreter | contents.html
Previous:
1.8 Keyword Parameters for (parseTupleAndKeywords.html)
Up:
1. Extending Python with (intro.html)
Next:
1.10 Reference Counts (refcounts.html)
---
# 1.9 Building Arbitrary Values
This function is the counterpart to PyArg_ParseTuple(). It is
declared as follows:
```text

PyObject *Py_BuildValue(char *format, ...);
```
It recognizes a set of format units similar to the ones recognized by
PyArg_ParseTuple(), but the arguments (which are input to the
function, not output) must not be pointers, just values. It returns a
new Python object, suitable for returning from a C function called
from Python.
One difference with PyArg_ParseTuple(): while the latter
requires its first argument to be a tuple (since Python argument lists
are always represented as tuples internally),
Py_BuildValue() does not always build a tuple. It builds
a tuple only if its format string contains two or more format units.
If the format string is empty, it returns `None`; if it contains
exactly one format unit, it returns whatever object is described by
that format unit. To force it to return a tuple of size 0 or one,
parenthesize the format string.
Examples (to the left the call, to the right the resulting Python value):
```text

Py_BuildValue("") None
Py_BuildValue("i", 123) 123
Py_BuildValue("iii", 123, 456, 789) (123, 456, 789)
Py_BuildValue("s", "hello") 'hello'
Py_BuildValue("ss", "hello", "world") ('hello', 'world')
Py_BuildValue("s#", "hello", 4) 'hell'
Py_BuildValue("()") ()
Py_BuildValue("(i)", 123) (123,)
Py_BuildValue("(ii)", 123, 456) (123, 456)
Py_BuildValue("(i,i)", 123, 456) (123, 456)
Py_BuildValue("[i,i]", 123, 456) [123, 456]
Py_BuildValue("{s:i,s:i}",
"abc", 123, "def", 456) {'abc': 123, 'def': 456}
Py_BuildValue("((ii)(ii)) (ii)",
1, 2, 3, 4, 5, 6) (((1, 2), (3, 4)), (5, 6))
```


1.6 Calling Python Functions from C

compilation.html | intro.html | parseTuple.html | Extending and Embedding the Python Interpreter | contents.html
Previous:
1.5 Compilation and Linkage (compilation.html)
Up:
1. Extending Python with (intro.html)
Next:
1.7 Extracting Parameters in (parseTuple.html)
---
# 1.6 Calling Python Functions from C
So far we have concentrated on making C functions callable from
Python. The reverse is also useful: calling Python functions from C.
This is especially the case for libraries that support so-called
``callback'' functions. If a C interface makes use of callbacks, the
equivalent Python often needs to provide a callback mechanism to the
Python programmer; the implementation will require calling the Python
callback functions from a C callback. Other uses are also imaginable.
Fortunately, the Python interpreter is easily called recursively, and
there is a standard interface to call a Python function. (I won't
dwell on how to call the Python parser with a particular string as
input -- if you're interested, have a look at the implementation of
the -c command line option in Python/pythonmain.c
from the Python source code.)
Calling a Python function is easy. First, the Python program must
somehow pass you the Python function object. You should provide a
function (or some other interface) to do this. When this function is
called, save a pointer to the Python function object (be careful to
Py_INCREF() it!) in a global variable -- or wherever you
see fit. For example, the following function might be part of a module
definition:
```text

static PyObject *my_callback = NULL;

static PyObject *
my_set_callback(PyObject *dummy, PyObject *args)
{
PyObject *result = NULL;
PyObject *temp;

if (PyArg_ParseTuple(args, "O:set_callback", &temp)) {
if (!PyCallable_Check(temp)) {
PyErr_SetString(PyExc_TypeError, "parameter must be callable");
return NULL;
}
Py_XINCREF(temp); /* Add a reference to new callback */
Py_XDECREF(my_callback); /* Dispose of previous callback */
my_callback = temp; /* Remember new callback */
/* Boilerplate to return "None" */
Py_INCREF(Py_None);
result = Py_None;
}
return result;
}
```
This function must be registered with the interpreter using the
METH_VARARGS flag; this is described in section
1.4 (methodTable.html#methodTable), ``The Module's Method Table and Initialization
Function.'' The PyArg_ParseTuple() function and its
arguments are documented in section 1.7 (parseTuple.html#parseTuple), ``Extracting
Parameters in Extension Functions.''
The macros Py_XINCREF() and Py_XDECREF()
increment/decrement the reference count of an object and are safe in
the presence of NULL pointers (but note that temp will not be
NULL in this context). More info on them in
section 1.10 (refcounts.html#refcounts), ``Reference Counts.''
Later, when it is time to call the function, you call the C function
PyEval_CallObject(). This
function has two arguments, both pointers to arbitrary Python objects:
the Python function, and the argument list. The argument list must
always be a tuple object, whose length is the number of arguments. To
call the Python function with no arguments, pass an empty tuple; to
call it with one argument, pass a singleton tuple.
Py_BuildValue() returns a tuple when its format string
consists of zero or more format codes between parentheses. For
example:
```text

int arg;
PyObject *arglist;
PyObject *result;
...
arg = 123;
...
/* Time to call the callback */
arglist = Py_BuildValue("(i)", arg);
result = PyEval_CallObject(my_callback, arglist);
Py_DECREF(arglist);
```
PyEval_CallObject() returns a Python object pointer: this is
the return value of the Python function. PyEval_CallObject() is
``reference-count-neutral'' with respect to its arguments. In the
example a new tuple was created to serve as the argument list, which
is Py_DECREF()-ed immediately after the call.
The return value of PyEval_CallObject() is ``new'': either it
is a brand new object, or it is an existing object whose reference
count has been incremented. So, unless you want to save it in a
global variable, you should somehow Py_DECREF() the result,
even (especially!) if you are not interested in its value.
Before you do this, however, it is important to check that the return
value isn't NULL. If it is, the Python function terminated by
raising an exception. If the C code that called
PyEval_CallObject() is called from Python, it should now
return an error indication to its Python caller, so the interpreter
can print a stack trace, or the calling Python code can handle the
exception. If this is not possible or desirable, the exception should
be cleared by calling PyErr_Clear(). For example:
```text

if (result == NULL)
return NULL; /* Pass error back */
...use result...
Py_DECREF(result);
```
Depending on the desired interface to the Python callback function,
you may also have to provide an argument list to
PyEval_CallObject(). In some cases the argument list is
also provided by the Python program, through the same interface that
specified the callback function. It can then be saved and used in the
same manner as the function object. In other cases, you may have to
construct a new tuple to pass as the argument list. The simplest way
to do this is to call Py_BuildValue(). For example, if
you want to pass an integral event code, you might use the following
code:
```text

PyObject *arglist;
...
arglist = Py_BuildValue("(l)", eventcode);
result = PyEval_CallObject(my_callback, arglist);
Py_DECREF(arglist);
if (result == NULL)
return NULL; /* Pass error back */
/* Here maybe use the result */
Py_DECREF(result);
```
Note the placement of "Py_DECREF(arglist)" immediately after the
call, before the error check! Also note that strictly spoken this
code is not complete: Py_BuildValue() may run out of
memory, and this should be checked.


1.5 Compilation and Linkage

methodTable.html | intro.html | callingPython.html | Extending and Embedding the Python Interpreter | contents.html
Previous:
1.4 The Module's Method (methodTable.html)
Up:
1. Extending Python with (intro.html)
Next:
1.6 Calling Python Functions (callingPython.html)
---
# 1.5 Compilation and Linkage
There are two more things to do before you can use your new extension:
compiling and linking it with the Python system. If you use dynamic
loading, the details may depend on the style of dynamic loading your
system uses; see the chapters about building extension modules
(chapter 3 (building.html#building)) and additional information that pertains only
to building on Windows (chapter 4 (building-on-windows.html#building-on-windows)) for more
information about this.
If you can't use dynamic loading, or if you want to make your module a
permanent part of the Python interpreter, you will have to change the
configuration setup and rebuild the interpreter. Luckily, this is
very simple on Unix: just place your file (spammodule.c for
example) in the Modules/ directory of an unpacked source
distribution, add a line to the file Modules/Setup.local
describing your file:
```text

spam spammodule.o
```
and rebuild the interpreter by running make in the toplevel
directory. You can also run make in the Modules/
subdirectory, but then you must first rebuild Makefile
there by running `make Makefile'. (This is necessary each
time you change the Setup file.)
If your module requires additional libraries to link with, these can
be listed on the line in the configuration file as well, for instance:
```text

spam spammodule.o -lX11
```


Contents

front.html | ext.html | intro.html | Extending and Embedding the Python Interpreter
Previous:
Front Matter (front.html)
Up:
Extending and Embedding the (ext.html)
Next:
1. Extending Python with (intro.html)
---
## Contents
Table of Contents
- Front Matter (front.html)
  1. Extending Python with C or C++ (intro.html)
  - 1.1 A Simple Example (simpleExample.html)
  1.2 Intermezzo: Errors and Exceptions (errors.html)
  1.3 Back to the Example (backToExample.html)
  1.4 The Module's Method Table and Initialization Function (methodTable.html)
  1.5 Compilation and Linkage (compilation.html)
  1.6 Calling Python Functions from C (callingPython.html)
  1.7 Extracting Parameters in Extension Functions (parseTuple.html)
  1.8 Keyword Parameters for Extension Functions (parseTupleAndKeywords.html)
  1.9 Building Arbitrary Values (buildValue.html)
  1.10 Reference Counts (refcounts.html)
  - 1.10.1 Reference Counting in Python (refcountsInPython.html)
  1.10.2 Ownership Rules (ownershipRules.html)
  1.10.3 Thin Ice (thinIce.html)
  1.10.4 NULL Pointers (nullPointers.html)
  1.11 Writing Extensions in C++ (cplusplus.html)
  1.12 Providing a C API for an Extension Module (using-cobjects.html)
  2. Defining New Types (defining-new-types.html)
  - 2.1 The Basics (dnt-basics.html)
  - 2.1.1 Adding data and methods to the Basic example (node22.html)
  2.1.2 Providing finer control over data attributes (node23.html)
  2.1.3 Supporting cyclic garbage collection (node24.html)
  2.2 Type Methods (dnt-type-methods.html)
  - 2.2.1 Finalization and De-allocation (node26.html)
  2.2.2 Object Presentation (node27.html)
  2.2.3 Attribute Management (node28.html)
  2.2.4 Object Comparison (node31.html)
  2.2.5 Abstract Protocol Support (node32.html)
  2.2.6 More Suggestions (node33.html)
  3. Building C and C++ Extensions with distutils (building.html)
  - 3.1 Distributing your extension modules (distributing.html)
  4. Building C and C++ Extensions on Windows (building-on-windows.html)
  - 4.1 A Cookbook Approach (win-cookbook.html)
  4.2 Differences Between Unix and Windows (dynamic-linking.html)
  4.3 Using DLLs in Practice (win-dlls.html)
  5. Embedding Python in Another Application (embedding.html)
  - 5.1 Very High Level Embedding (high-level-embedding.html)
  5.2 Beyond Very High Level Embedding: An overview (lower-level-embedding.html)
  5.3 Pure Embedding (pure-embedding.html)
  5.4 Extending Embedded Python (extending-with-embedding.html)
  5.5 Embedding Python in C++ (embeddingInCplusplus.html)
  5.6 Linking Requirements (link-reqs.html)
  A. Reporting Bugs (reporting-bugs.html)
  B. History and License (node48.html)
  - B.1 History of the software (node49.html)
  B.2 Terms and conditions for accessing or otherwise using Python (node50.html)
  About this document ... (about.html)
End of Table of Contents


1.11 Writing Extensions in C++

nullPointers.html | intro.html | using-cobjects.html | Extending and Embedding the Python Interpreter | contents.html
Previous:
1.10.4 NULL Pointers (nullPointers.html)
Up:
1. Extending Python with (intro.html)
Next:
1.12 Providing a C (using-cobjects.html)
---
# 1.11 Writing Extensions in C++
It is possible to write extension modules in C++. Some restrictions
apply. If the main program (the Python interpreter) is compiled and
linked by the C compiler, global or static objects with constructors
cannot be used. This is not a problem if the main program is linked
by the C++ compiler. Functions that will be called by the
Python interpreter (in particular, module initalization functions)
have to be declared using `extern "C"`.
It is unnecessary to enclose the Python header files in
`extern "C" {...}` -- they use this form already if the symbol
"__cplusplus" is defined (all recent C++ compilers define this
symbol).


2. Defining New Types

using-cobjects.html | ext.html | dnt-basics.html | Extending and Embedding the Python Interpreter | contents.html
Previous:
1.12 Providing a C (using-cobjects.html)
Up:
Extending and Embedding the (ext.html)
Next:
2.1 The Basics (dnt-basics.html)
---
# 2. Defining New Types
As mentioned in the last chapter, Python allows the writer of an
extension module to define new types that can be manipulated from
Python code, much like strings and lists in core Python.
This is not hard; the code for all extension types follows a pattern,
but there are some details that you need to understand before you can
get started.
Note:
The way new types are defined changed dramatically (and for the
better) in Python 2.2. This document documents how to define new
types for Python 2.2 and later. If you need to support older
versions of Python, you will need to refer to older versions of this
documentation.


3.1 Distributing your extension modules

building.html | building.html | building-on-windows.html | Extending and Embedding the Python Interpreter | contents.html
Previous:
3. Building C and (building.html)
Up:
3. Building C and (building.html)
Next:
4. Building C and (building-on-windows.html)
---
# 3.1 Distributing your extension modules
When an extension has been successfully build, there are three ways to
use it.
End-users will typically want to install the module, they do so by
running
```text

python setup.py install
```
Module maintainers should produce source packages; to do so, they run
```text

python setup.py sdist
```
In some cases, additional files need to be included in a source
distribution; this is done through a MANIFEST.in file; see the
distutils documentation for details.
If the source distribution has been build successfully, maintainers
can also create binary distributions. Depending on the platform, one
of the following commands can be used to do so.


2.1 The Basics

defining-new-types.html | defining-new-types.html | node22.html | Extending and Embedding the Python Interpreter | contents.html
Previous:
2. Defining New Types (defining-new-types.html)
Up:
2. Defining New Types (defining-new-types.html)
Next:
2.1.1 Adding data and (node22.html)
---
# 2.1 The Basics
The Python runtime sees all Python objects as variables of type
PyObject*. A PyObject is not a very magnificent
object - it just contains the refcount and a pointer to the object's
``type object''. This is where the action is; the type object
determines which (C) functions get called when, for instance, an
attribute gets looked up on an object or it is multiplied by another
object. These C functions are called ``type methods'' to distinguish
them from things like `[].append` (which we call ``object
methods'').
So, if you want to define a new object type, you need to create a new
type object.
This sort of thing can only be explained by example, so here's a
minimal, but complete, module that defines a new type:
```text
#include <Python.h>

typedef struct {
PyObject_HEAD
/* Type-specific fields go here. */
} noddy_NoddyObject;

static PyTypeObject noddy_NoddyType = {
PyObject_HEAD_INIT(NULL)
0, /*ob_size*/
"noddy.Noddy", /*tp_name*/
sizeof(noddy_NoddyObject), /*tp_basicsize*/
0, /*tp_itemsize*/
0, /*tp_dealloc*/
0, /*tp_print*/
0, /*tp_getattr*/
0, /*tp_setattr*/
0, /*tp_compare*/
0, /*tp_repr*/
0, /*tp_as_number*/
0, /*tp_as_sequence*/
0, /*tp_as_mapping*/
0, /*tp_hash */
0, /*tp_call*/
0, /*tp_str*/
0, /*tp_getattro*/
0, /*tp_setattro*/
0, /*tp_as_buffer*/
Py_TPFLAGS_DEFAULT, /*tp_flags*/
"Noddy objects", /* tp_doc */
};

static PyMethodDef noddy_methods[] = {
{NULL} /* Sentinel */
};

#ifndef PyMODINIT_FUNC /* declarations for DLL import/export */
#define PyMODINIT_FUNC void
#endif
PyMODINIT_FUNC
initnoddy(void)
{
PyObject* m;

noddy_NoddyType.tp_new = PyType_GenericNew;
if (PyType_Ready(&noddy_NoddyType) < 0)
return;

m = Py_InitModule3("noddy", noddy_methods,
"Example module that creates an extension type.");

Py_INCREF(&noddy_NoddyType);
PyModule_AddObject(m, "Noddy", (PyObject *)&noddy_NoddyType);
}
```
Download as text (original file name: noddy.c). (noddy.txt)
Now that's quite a bit to take in at once, but hopefully bits will
seem familiar from the last chapter.
The first bit that will be new is:
```text

typedef struct {
PyObject_HEAD
} noddy_NoddyObject;
```
This is what a Noddy object will contain--in this case, nothing more
than every Python object contains, namely a refcount and a pointer to a type
object. These are the fields the `PyObject_HEAD` macro brings
in. The reason for the macro is to standardize the layout and to
enable special debugging fields in debug builds. Note that there is
no semicolon after the `PyObject_HEAD` macro; one is included in
the macro definition. Be wary of adding one by accident; it's easy to
do from habit, and your compiler might not complain, but someone
else's probably will! (On Windows, MSVC is known to call this an
error and refuse to compile the code.)
For contrast, let's take a look at the corresponding definition for
standard Python integers:
```text

typedef struct {
PyObject_HEAD
long ob_ival;
} PyIntObject;
```
Moving on, we come to the crunch -- the type object.
```text

static PyTypeObject noddy_NoddyType = {
PyObject_HEAD_INIT(NULL)
0, /*ob_size*/
"noddy.Noddy", /*tp_name*/
sizeof(noddy_NoddyObject), /*tp_basicsize*/
0, /*tp_itemsize*/
0, /*tp_dealloc*/
0, /*tp_print*/
0, /*tp_getattr*/
0, /*tp_setattr*/
0, /*tp_compare*/
0, /*tp_repr*/
0, /*tp_as_number*/
0, /*tp_as_sequence*/
0, /*tp_as_mapping*/
0, /*tp_hash */
0, /*tp_call*/
0, /*tp_str*/
0, /*tp_getattro*/
0, /*tp_setattro*/
0, /*tp_as_buffer*/
Py_TPFLAGS_DEFAULT, /*tp_flags*/
"Noddy objects", /* tp_doc */
};
```
Now if you go and look up the definition of PyTypeObject in
object.h you'll see that it has many more fields that the
definition above. The remaining fields will be filled with zeros by
the C compiler, and it's common practice to not specify them
explicitly unless you need them.
This is so important that we're going to pick the top of it apart still
further:
```text

PyObject_HEAD_INIT(NULL)
```
This line is a bit of a wart; what we'd like to write is:
```text

PyObject_HEAD_INIT(&PyType_Type)
```
as the type of a type object is ``type'', but this isn't strictly
conforming C and some compilers complain. Fortunately, this member
will be filled in for us by PyType_Ready().
```text

0, /* ob_size */
```
The ob_size field of the header is not used; its presence in
the type structure is a historical artifact that is maintained for
binary compatibility with extension modules compiled for older
versions of Python. Always set this field to zero.
```text

"noddy.Noddy", /* tp_name */
```
The name of our type. This will appear in the default textual
representation of our objects and in some error messages, for example:
```text

>>> "" + noddy.new_noddy()
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: cannot add type "noddy.Noddy" to string
```
Note that the name is a dotted name that includes both the module name
and the name of the type within the module. The module in this case is
noddy and the type is Noddy, so we set the type name
to noddy.Noddy.
```text

sizeof(noddy_NoddyObject), /* tp_basicsize */
```
This is so that Python knows how much memory to allocate when you call
PyObject_New().
```text

0, /* tp_itemsize */
```
This has to do with variable length objects like lists and strings.
Ignore this for now.
Skipping a number of type methods that we don't provide, we set the
class flags to Py_TPFLAGS_DEFAULT.
```text

Py_TPFLAGS_DEFAULT, /*tp_flags*/
```
All types should include this constant in their flags. It enables all
of the members defined by the current version of Python.
We provide a doc string for the type in tp_doc.
```text

"Noddy objects", /* tp_doc */
```
Now we get into the type methods, the things that make your objects
different from the others. We aren't going to implement any of these
in this version of the module. We'll expand this example later to
have more interesting behavior.
For now, all we want to be able to do is to create new Noddy
objects. To enable object creation, we have to provide a
tp_new implementation. In this case, we can just use the
default implementation provided by the API function
PyType_GenericNew(). We'd like to just assign this to the
tp_new slot, but we can't, for portability sake, On some
platforms or compilers, we can't statically initialize a structure
member with a function defined in another C module, so, instead, we'll
assign the tp_new slot in the module initialization function
just before calling PyType_Ready():
```text

noddy_NoddyType.tp_new = PyType_GenericNew;
if (PyType_Ready(&noddy_NoddyType) < 0)
return;
```
All the other type methods are NULL, so we'll go over them later
-- that's for a later section!
Everything else in the file should be familiar, except for some code
in initnoddy():
```text

if (PyType_Ready(&noddy_NoddyType) < 0)
return;
```
This initializes the Noddy type, filing in a number of
members, including ob_type that we initially set to NULL.
```text

PyModule_AddObject(m, "Noddy", (PyObject *)&noddy_NoddyType);
```
This adds the type to the module dictionary. This allows us to create
Noddy instances by calling the Noddy class:
```text

import noddy
mynoddy = noddy.Noddy()
```
That's it! All that remains is to build it; put the above code in a
file called noddy.c and
```text

from distutils.core import setup, Extension
setup(name="noddy", version="1.0",
ext_modules=[Extension("noddy", ["noddy.c"])])
```
in a file called setup.py; then typing
```text

$ python setup.py build
```
at a shell should produce a file noddy.so in a subdirectory;
move to that directory and fire up Python -- you should be able to
`import noddy` and play around with Noddy objects.
That wasn't so hard, was it?
Of course, the current Noddy type is pretty uninteresting. It has no
data and doesn't do anything. It can't even be subclassed.


2.2 Type Methods

node24.html | defining-new-types.html | node26.html | Extending and Embedding the Python Interpreter | contents.html
Previous:
2.1.3 Supporting cyclic garbage (node24.html)
Up:
2. Defining New Types (defining-new-types.html)
Next:
2.2.1 Finalization and De-allocation (node26.html)
---
# 2.2 Type Methods
This section aims to give a quick fly-by on the various type methods
you can implement and what they do.
Here is the definition of PyTypeObject, with some fields only
used in debug builds omitted:
```text
typedef struct _typeobject {
PyObject_VAR_HEAD
char *tp_name; /* For printing, in format "<module>.<name>" */
int tp_basicsize, tp_itemsize; /* For allocation */

/* Methods to implement standard operations */

destructor tp_dealloc;
printfunc tp_print;
getattrfunc tp_getattr;
setattrfunc tp_setattr;
cmpfunc tp_compare;
reprfunc tp_repr;

/* Method suites for standard classes */

PyNumberMethods *tp_as_number;
PySequenceMethods *tp_as_sequence;
PyMappingMethods *tp_as_mapping;

/* More standard operations (here for binary compatibility) */

hashfunc tp_hash;
ternaryfunc tp_call;
reprfunc tp_str;
getattrofunc tp_getattro;
setattrofunc tp_setattro;

/* Functions to access object as input/output buffer */
PyBufferProcs *tp_as_buffer;

/* Flags to define presence of optional/expanded features */
long tp_flags;

char *tp_doc; /* Documentation string */

/* Assigned meaning in release 2.0 */
/* call function for all accessible objects */
traverseproc tp_traverse;

/* delete references to contained objects */
inquiry tp_clear;

/* Assigned meaning in release 2.1 */
/* rich comparisons */
richcmpfunc tp_richcompare;

/* weak reference enabler */
long tp_weaklistoffset;

/* Added in release 2.2 */
/* Iterators */
getiterfunc tp_iter;
iternextfunc tp_iternext;

/* Attribute descriptor and subclassing stuff */
struct PyMethodDef *tp_methods;
struct PyMemberDef *tp_members;
struct PyGetSetDef *tp_getset;
struct _typeobject *tp_base;
PyObject *tp_dict;
descrgetfunc tp_descr_get;
descrsetfunc tp_descr_set;
long tp_dictoffset;
initproc tp_init;
allocfunc tp_alloc;
newfunc tp_new;
freefunc tp_free; /* Low-level free-memory routine */
inquiry tp_is_gc; /* For PyObject_IS_GC */
PyObject *tp_bases;
PyObject *tp_mro; /* method resolution order */
PyObject *tp_cache;
PyObject *tp_subclasses;
PyObject *tp_weaklist;

} PyTypeObject;
```
Download as text (original file name: typestruct.h). (typestruct.txt)
Now that's a lot of methods. Don't worry too much though - if
you have a type you want to define, the chances are very good that you
will only implement a handful of these.
As you probably expect by now, we're going to go over this and give
more information about the various handlers. We won't go in the order
they are defined in the structure, because there is a lot of
historical baggage that impacts the ordering of the fields; be sure
your type initializaion keeps the fields in the right order! It's
often easiest to find an example that includes all the fields you need
(even if they're initialized to `0`) and then change the values
to suit your new type.
```text

char *tp_name; /* For printing */
```
The name of the type - as mentioned in the last section, this will
appear in various places, almost entirely for diagnostic purposes.
Try to choose something that will be helpful in such a situation!
```text

int tp_basicsize, tp_itemsize; /* For allocation */
```
These fields tell the runtime how much memory to allocate when new
objects of this type are created. Python has some builtin support
for variable length structures (think: strings, lists) which is where
the tp_itemsize field comes in. This will be dealt with
later.
```text

char *tp_doc;
```
Here you can put a string (or its address) that you want returned when
the Python script references `obj.__doc__` to retrieve the
docstring.
Now we come to the basic type methods--the ones most extension types
will implement.


win-cookbook.html | building-on-windows.html | win-dlls.html | Extending and Embedding the Python Interpreter | contents.html
Previous:
4.1 A Cookbook Approach (win-cookbook.html)
Up:
4. Building C and (building-on-windows.html)
Next:
4.3 Using DLLs in (win-dlls.html)
---
# 4.2 Differences Between Unix and Windows
Unix and Windows use completely different paradigms for run-time
loading of code. Before you try to build a module that can be
dynamically loaded, be aware of how your system works.
In Unix, a shared object (.so) file contains code to be used by the
program, and also the names of functions and data that it expects to
find in the program. When the file is joined to the program, all
references to those functions and data in the file's code are changed
to point to the actual locations in the program where the functions
and data are placed in memory. This is basically a link operation.
In Windows, a dynamic-link library (.dll) file has no dangling
references. Instead, an access to functions or data goes through a
lookup table. So the DLL code does not have to be fixed up at runtime
to refer to the program's memory; instead, the code already uses the
DLL's lookup table, and the lookup table is modified at runtime to
point to the functions and data.
In Unix, there is only one type of library file (.a) which
contains code from several object files (.o). During the link
step to create a shared object file (.so), the linker may find
that it doesn't know where an identifier is defined. The linker will
look for it in the object files in the libraries; if it finds it, it
will include all the code from that object file.
In Windows, there are two types of library, a static library and an
import library (both called .lib). A static library is like a
Unix .a file; it contains code to be included as necessary.
An import library is basically used only to reassure the linker that a
certain identifier is legal, and will be present in the program when
the DLL is loaded. So the linker uses the information from the
import library to build the lookup table for using identifiers that
are not included in the DLL. When an application or a DLL is linked,
an import library may be generated, which will need to be used for all
future DLLs that depend on the symbols in the application or DLL.
Suppose you are building two dynamic-load modules, B and C, which should
share another block of code A. On Unix, you would not pass
A.a to the linker for B.so and C.so; that would
cause it to be included twice, so that B and C would each have their
own copy. In Windows, building A.dll will also build
A.lib. You do pass A.lib to the linker for B and
C. A.lib does not contain code; it just contains information
which will be used at runtime to access A's code.
In Windows, using an import library is sort of like using "import
spam"; it gives you access to spam's names, but does not create a
separate copy. On Unix, linking with a library is more like
"from spam import *"; it does create a separate copy.


win-dlls.html | ext.html | high-level-embedding.html | Extending and Embedding the Python Interpreter | contents.html
Previous:
4.3 Using DLLs in (win-dlls.html)
Up:
Extending and Embedding the (ext.html)
Next:
5.1 Very High Level (high-level-embedding.html)
---
# 5. Embedding Python in Another Application
The previous chapters discussed how to extend Python, that is, how to
extend the functionality of Python by attaching a library of C
functions to it. It is also possible to do it the other way around:
enrich your C/C++ application by embedding Python in it. Embedding
provides your application with the ability to implement some of the
functionality of your application in Python rather than C or C++.
This can be used for many purposes; one example would be to allow
users to tailor the application to their needs by writing some scripts
in Python. You can also use it yourself if some of the functionality
can be written in Python more easily.
Embedding Python is similar to extending it, but not quite. The
difference is that when you extend Python, the main program of the
application is still the Python interpreter, while if you embed
Python, the main program may have nothing to do with Python --
instead, some parts of the application occasionally call the Python
interpreter to run some Python code.
So if you are embedding Python, you are providing your own main
program. One of the things this main program has to do is initialize
the Python interpreter. At the very least, you have to call the
function Py_Initialize() (on Mac OS, call
PyMac_Initialize() instead). There are optional calls to
pass command line arguments to Python. Then later you can call the
interpreter from any part of the application.
There are several different ways to call the interpreter: you can pass
a string containing Python statements to
PyRun_SimpleString(), or you can pass a stdio file pointer
and a file name (for identification in error messages only) to
PyRun_SimpleFile(). You can also call the lower-level
operations described in the previous chapters to construct and use
Python objects.
A simple demo of embedding Python can be found in the directory
Demo/embed/ of the source distribution.
See Also:


5.5 Embedding Python in C++

extending-with-embedding.html | embedding.html | link-reqs.html | Extending and Embedding the Python Interpreter | contents.html
Previous:
5.4 Extending Embedded Python (extending-with-embedding.html)
Up:
5. Embedding Python in (embedding.html)
Next:
5.6 Linking Requirements (link-reqs.html)
---
# 5.5 Embedding Python in C++
It is also possible to embed Python in a C++ program; precisely how this
is done will depend on the details of the C++ system used; in general you
will need to write the main program in C++, and use the C++ compiler
to compile and link your program. There is no need to recompile Python
itself using C++.


1.2 Intermezzo: Errors and Exceptions

simpleExample.html | intro.html | backToExample.html | Extending and Embedding the Python Interpreter | contents.html
Previous:
1.1 A Simple Example (simpleExample.html)
Up:
1. Extending Python with (intro.html)
Next:
1.3 Back to the (backToExample.html)
---
# 1.2 Intermezzo: Errors and Exceptions
An important convention throughout the Python interpreter is the
following: when a function fails, it should set an exception condition
and return an error value (usually a NULL pointer). Exceptions
are stored in a static global variable inside the interpreter; if this
variable is NULL no exception has occurred. A second global
variable stores the ``associated value'' of the exception (the second
argument to raise). A third variable contains the stack
traceback in case the error originated in Python code. These three
variables are the C equivalents of the Python variables
`sys.exc_type`, `sys.exc_value` and `sys.exc_traceback` (see
the section on module sys in the
Python Library Reference (../lib/lib.html)). It is
important to know about them to understand how errors are passed
around.
The Python API defines a number of functions to set various types of
exceptions.
The most common one is PyErr_SetString(). Its arguments
are an exception object and a C string. The exception object is
usually a predefined object like PyExc_ZeroDivisionError. The
C string indicates the cause of the error and is converted to a
Python string object and stored as the ``associated value'' of the
exception.
Another useful function is PyErr_SetFromErrno(), which only
takes an exception argument and constructs the associated value by
inspection of the global variable errno. The most
general function is PyErr_SetObject(), which takes two object
arguments, the exception and its associated value. You don't need to
Py_INCREF() the objects passed to any of these functions.
You can test non-destructively whether an exception has been set with
PyErr_Occurred(). This returns the current exception object,
or NULL if no exception has occurred. You normally don't need
to call PyErr_Occurred() to see whether an error occurred in a
function call, since you should be able to tell from the return value.
When a function f that calls another function g detects
that the latter fails, f should itself return an error value
(usually NULL or `-1`). It should not call one of the
PyErr_*() functions -- one has already been called by g.
f's caller is then supposed to also return an error indication
to its caller, again without calling PyErr_*(),
and so on -- the most detailed cause of the error was already
reported by the function that first detected it. Once the error
reaches the Python interpreter's main loop, this aborts the currently
executing Python code and tries to find an exception handler specified
by the Python programmer.
(There are situations where a module can actually give a more detailed
error message by calling another PyErr_*() function, and in
such cases it is fine to do so. As a general rule, however, this is
not necessary, and can cause information about the cause of the error
to be lost: most operations can fail for a variety of reasons.)
To ignore an exception set by a function call that failed, the exception
condition must be cleared explicitly by calling PyErr_Clear().
The only time C code should call PyErr_Clear() is if it doesn't
want to pass the error on to the interpreter but wants to handle it
completely by itself (possibly by trying something else, or pretending
nothing went wrong).
Every failing malloc() call must be turned into an
exception -- the direct caller of malloc() (or
realloc()) must call PyErr_NoMemory() and
return a failure indicator itself. All the object-creating functions
(for example, PyInt_FromLong()) already do this, so this
note is only relevant to those who call malloc() directly.
Also note that, with the important exception of
PyArg_ParseTuple() and friends, functions that return an
integer status usually return a positive value or zero for success and
`-1` for failure, like Unix system calls.
Finally, be careful to clean up garbage (by making
Py_XDECREF() or Py_DECREF() calls for objects
you have already created) when you return an error indicator!
The choice of which exception to raise is entirely yours. There are
predeclared C objects corresponding to all built-in Python exceptions,
such as PyExc_ZeroDivisionError, which you can use directly.
Of course, you should choose exceptions wisely -- don't use
PyExc_TypeError to mean that a file couldn't be opened (that
should probably be PyExc_IOError). If something's wrong with
the argument list, the PyArg_ParseTuple() function usually
raises PyExc_TypeError. If you have an argument whose value
must be in a particular range or must satisfy other conditions,
PyExc_ValueError is appropriate.
You can also define a new exception that is unique to your module.
For this, you usually declare a static object variable at the
beginning of your file:
```text

static PyObject *SpamError;
```
and initialize it in your module's initialization function
(initspam()) with an exception object (leaving out
the error checking for now):
```text

PyMODINIT_FUNC
initspam(void)
{
PyObject *m;

m = Py_InitModule("spam", SpamMethods);

SpamError = PyErr_NewException("spam.error", NULL, NULL);
Py_INCREF(SpamError);
PyModule_AddObject(m, "error", SpamError);
}
```
Note that the Python name for the exception object is
spam.error. The PyErr_NewException() function
may create a class with the base class being Exception
(unless another class is passed in instead of NULL), described in the
Python Library Reference (../lib/lib.html) under ``Built-in
Exceptions.''
Note also that the SpamError variable retains a reference to
the newly created exception class; this is intentional! Since the
exception could be removed from the module by external code, an owned
reference to the class is needed to ensure that it will not be
discarded, causing SpamError to become a dangling pointer.
Should it become a dangling pointer, C code which raises the exception
could cause a core dump or other unintended side effects.
We discuss the use of PyMODINIT_FUNC later in this sample.


../index.html | front.html | Extending and Embedding the Python Interpreter | contents.html
Up:
Python Documentation Index (../index.html)
Next:
Front Matter (front.html)
---
# Extending and Embedding the Python Interpreter
Guido van Rossum
Fred L. Drake, Jr., editor
PythonLabs
Email: python-docs@python.org
Release 2.3
July 29, 2003


5.4 Extending Embedded Python

pure-embedding.html | embedding.html | embeddingInCplusplus.html | Extending and Embedding the Python Interpreter | contents.html
Previous:
5.3 Pure Embedding (pure-embedding.html)
Up:
5. Embedding Python in (embedding.html)
Next:
5.5 Embedding Python in (embeddingInCplusplus.html)
---
# 5.4 Extending Embedded Python
Until now, the embedded Python interpreter had no access to
functionality from the application itself. The Python API allows this
by extending the embedded interpreter. That is, the embedded
interpreter gets extended with routines provided by the application.
While it sounds complex, it is not so bad. Simply forget for a while
that the application starts the Python interpreter. Instead, consider
the application to be a set of subroutines, and write some glue code
that gives Python access to those routines, just like you would write
a normal Python extension. For example:
```text

static int numargs=0;

/* Return the number of arguments of the application command line */
static PyObject*
emb_numargs(PyObject *self, PyObject *args)
{
if(!PyArg_ParseTuple(args, ":numargs"))
return NULL;
return Py_BuildValue("i", numargs);
}

static PyMethodDef EmbMethods[] = {
{"numargs", emb_numargs, METH_VARARGS,
"Return the number of arguments received by the process."},
{NULL, NULL, 0, NULL}
};
```
Insert the above code just above the main() function.
Also, insert the following two statements directly after
Py_Initialize():
```text

numargs = argc;
Py_InitModule("emb", EmbMethods);
```
These two lines initialize the `numargs` variable, and make the
emb.numargs() function accessible to the embedded Python
interpreter. With these extensions, the Python script can do things
like
```text

import emb
print "Number of arguments", emb.numargs()
```
In a real application, the methods will expose an API of the
application to Python.


Front Matter

ext.html | ext.html | contents.html | Extending and Embedding the Python Interpreter | contents.html
Previous:
Extending and Embedding the (ext.html)
Up:
Extending and Embedding the (ext.html)
Next:
---
# Front Matter
Copyright © 2001, 2002, 2003 Python Software Foundation.
All rights reserved.
Copyright © 2000 BeOpen.com.
All rights reserved.
Copyright © 1995-2000 Corporation for National Research Initiatives.
All rights reserved.
Copyright © 1991-1995 Stichting Mathematisch Centrum.
All rights reserved.
See the end of this document for complete license and permissions
information.
### Abstract:
Python is an interpreted, object-oriented programming language. This
document describes how to write modules in C or C++ to extend the
Python interpreter with new modules. Those modules can define new
functions but also new object types and their methods. The document
also describes how to embed the Python interpreter in another
application, for use as an extension language. Finally, it shows how
to compile and link extension modules so that they can be loaded
dynamically (at run time) into the interpreter, if the underlying
operating system supports this feature.
This document assumes basic knowledge about Python. For an informal
introduction to the language, see the
Python Tutorial (../tut/tut.html). The
Python Reference Manual (../ref/ref.html) gives a more
formal definition of the language. The
Python Library Reference (../lib/lib.html) documents the
existing object types, functions and modules (both built-in and
written in Python) that give the language its wide application range.
For a detailed description of the whole Python/C API, see the separate
Python/C API Reference Manual (../api/api.html).


5.1 Very High Level Embedding

embedding.html | embedding.html | lower-level-embedding.html | Extending and Embedding the Python Interpreter | contents.html
Previous:
5. Embedding Python in (embedding.html)
Up:
5. Embedding Python in (embedding.html)
Next:
5.2 Beyond Very High (lower-level-embedding.html)
---
# 5.1 Very High Level Embedding
The simplest form of embedding Python is the use of the very
high level interface. This interface is intended to execute a
Python script without needing to interact with the application
directly. This can for example be used to perform some operation
on a file.
```text

#include <Python.h>

int
main(int argc, char *argv[])
{
Py_Initialize();
PyRun_SimpleString("from time import time,ctime\n"
"print 'Today is',ctime(time())\n");
Py_Finalize();
return 0;
}
```
The above code first initializes the Python interpreter with
Py_Initialize(), followed by the execution of a hard-coded
Python script that print the date and time. Afterwards, the
Py_Finalize() call shuts the interpreter down, followed by
the end of the program. In a real program, you may want to get the
Python script from another source, perhaps a text-editor routine, a
file, or a database. Getting the Python code from a file can better
be done by using the PyRun_SimpleFile() function, which
saves you the trouble of allocating memory space and loading the file
contents.


1. Extending Python with C or C++

contents.html | ext.html | simpleExample.html | Extending and Embedding the Python Interpreter | contents.html
Previous:
Up:
Extending and Embedding the (ext.html)
Next:
1.1 A Simple Example (simpleExample.html)
---
# 1. Extending Python with C or C++
It is quite easy to add new built-in modules to Python, if you know
how to program in C. Such extension modules can do two things
that can't be done directly in Python: they can implement new built-in
object types, and they can call C library functions and system calls.
To support extensions, the Python API (Application Programmers
Interface) defines a set of functions, macros and variables that
provide access to most aspects of the Python run-time system. The
Python API is incorporated in a C source file by including the header
`"Python.h"`.
The compilation of an extension module depends on its intended use as
well as on your system setup; details are given in later chapters.


5.6 Linking Requirements

embeddingInCplusplus.html | embedding.html | reporting-bugs.html | Extending and Embedding the Python Interpreter | contents.html
Previous:
5.5 Embedding Python in (embeddingInCplusplus.html)
Up:
5. Embedding Python in (embedding.html)
Next:
A. Reporting Bugs (reporting-bugs.html)
---
# 5.6 Linking Requirements
While the configure script shipped with the Python sources
will correctly build Python to export the symbols needed by
dynamically linked extensions, this is not automatically inherited by
applications which embed the Python library statically, at least on
Unix. This is an issue when the application is linked to the static
runtime library (libpython.a) and needs to load dynamic
extensions (implemented as .so files).
The problem is that some entry points are defined by the Python
runtime solely for extension modules to use. If the embedding
application does not use any of these entry points, some linkers will
not include those entries in the symbol table of the finished
executable. Some additional options are needed to inform the linker
not to remove these symbols.
Determining the right options to use for any given platform can be
quite difficult, but fortunately the Python configuration already has
those values. To retrieve them from an installed Python interpreter,
start an interactive interpreter and have a short session like this:
```text

>>> import distutils.sysconfig
>>> distutils.sysconfig.get_config_var('LINKFORSHARED')
'-Xlinker -export-dynamic'
```
The contents of the string presented will be the options that should
be used. If the string is empty, there's no need to add any
additional options. The LINKFORSHARED definition
corresponds to the variable of the same name in Python's top-level
Makefile.


high-level-embedding.html | embedding.html | pure-embedding.html | Extending and Embedding the Python Interpreter | contents.html
Previous:
5.1 Very High Level (high-level-embedding.html)
Up:
5. Embedding Python in (embedding.html)
Next:
5.3 Pure Embedding (pure-embedding.html)
---
# 5.2 Beyond Very High Level Embedding: An overview
The high level interface gives you the ability to execute
arbitrary pieces of Python code from your application, but
exchanging data values is quite cumbersome to say the least. If
you want that, you should use lower level calls. At the cost of
having to write more C code, you can achieve almost anything.
It should be noted that extending Python and embedding Python
is quite the same activity, despite the different intent. Most
topics discussed in the previous chapters are still valid. To
show this, consider what the extension code from Python to C
really does:
1. Convert data values from Python to C,
2. Perform a function call to a C routine using the
converted values, and
3. Convert the data values from the call from C to Python.
When embedding Python, the interface code does:
1. Convert data values from C to Python,
2. Perform a function call to a Python interface routine
using the converted values, and
3. Convert the data values from the call from Python to C.
As you can see, the data conversion steps are simply swapped to
accomodate the different direction of the cross-language transfer.
The only difference is the routine that you call between both
data conversions. When extending, you call a C routine, when
embedding, you call a Python routine.
This chapter will not discuss how to convert data from Python
to C and vice versa. Also, proper use of references and dealing
with errors is assumed to be understood. Since these aspects do not
differ from extending the interpreter, you can refer to earlier
chapters for the required information.