Bundle Example: Add a Command

This tutorial builds on the material from Bundle Example: Hello World.

This example describes how to create a ChimeraX bundle that defines two new commands, tutorial cofm and tutorial highlight. The steps in implementing the bundle are:

1. Create a bundle_info.xml containing information about the bundle,

2. Create a Python package that interfaces with ChimeraX and implements the command functionality, and

3. Install and test the bundle in ChimeraX.

The final step builds a Python wheel that ChimeraX uses to install the bundle. So if the bundle passes testing, it is immediately available for sharing with other users.

Before deciding on the name and syntax of your own command, you should peruse the command style guide.

Source Code Organization

The source code for this example may be downloaded as a zip-format file containing a folder named tut_cmd. Alternatively, one can start with an empty folder and create source files based on the samples below. The source folder may be arbitrarily named, as it is only used during installation; however, avoiding whitespace characters in the folder name bypasses the need to type quote characters in some steps.

Sample Files

The files in the tut_cmd folder are:

• tut_cmd - bundle folder

  • bundle_info.xml - bundle information read by ChimeraX

  • src - source code to Python package for bundle

    • __init__.py - package initializer and interface to ChimeraX

    • cmd.py - source code to implement two tutorial commands

    • docs/user/commands/tutorial.html - help file for the tutorial commands

The file contents are shown below.

bundle_info.xml

bundle_info.xml is an eXtensible Markup Language format file whose tags are listed in Bundle Information XML Tags. While there are many tags defined, only a few are needed for bundles written completely in Python. The bundle_info.xml in this example is similar to the one from the hello world example with changes highlighted. For explanations of the unhighlighted lines, please see Bundle Example: Hello World.

<!--
ChimeraX bundle names must start with "ChimeraX-"
to avoid clashes with package names in pypi.python.org.
When uploaded to the ChimeraX toolshed, the bundle
will be displayed without the ChimeraX- prefix.
-->

<BundleInfo name="ChimeraX-TutorialCommand"
	    version="0.1" package="chimerax.tut_cmd"
  	    minSessionVersion="1" maxSessionVersion="1">

  <!-- Additional information about bundle source -->
  <Author>UCSF RBVI</Author>
  <Email>chimerax@cgl.ucsf.edu</Email>
  <URL>https://www.rbvi.ucsf.edu/chimerax/</URL>

  <!-- Synopsis is a one-line description
       Description is a full multi-line description -->
  <Synopsis>Example for adding commands</Synopsis>
  <Description>Example code for implementing ChimeraX bundle.

Implements commands "tutorial cofm" to print the center of mass of a
set of atoms, and "tutorial highlight" to highlight the atoms nearest
the center of mass.
  </Description>

  <!-- Categories is a list where this bundle should appear -->
  <Categories>
    <Category name="General"/>
  </Categories>

  <!-- DataFiles is a list of additional files to include in bundle -->
  <DataFiles>
    <DataFile>docs/user/commands/tutorial.html</DataFile>
  </DataFiles>

  <!-- Dependencies on other ChimeraX/Python packages -->
  <Dependencies>
    <Dependency name="ChimeraX-Core" version="~=1.1"/>
  </Dependencies>

  <Classifiers>
    <!-- Development Status should be compatible with bundle version number -->
    <PythonClassifier>Development Status :: 3 - Alpha</PythonClassifier>
    <PythonClassifier>License :: Freeware</PythonClassifier>
    <!-- ChimeraX classifiers describe supplied functionality -->
    <!-- Register two commands for printing the center of mass
	 and selecting/coloring atom(s) nearest the center of mass of atoms -->
    <ChimeraXClassifier>ChimeraX :: Command :: tutorial cofm :: General ::
      Print center of mass of atoms</ChimeraXClassifier>
    <ChimeraXClassifier>ChimeraX :: Command :: tutorial highlight :: General ::
      Highlight atoms near center of mass</ChimeraXClassifier>
  </Classifiers>

</BundleInfo>

The BundleInfo, Synopsis and Description tags are changed to reflect the new bundle name and documentation (lines 8-10 and 17-25). The DataFiles tag is added to include documentation files (lines 33-35). The only other change is replacing the ChimeraXClassifier tags to declare the two commands in this bundle (lines 49-52).

Note that the two command, tutorial cofm (Center OF Mass) and tutorial highlight, are multi-word commands that share the same initial word. Most bundles that provide multiple commands should add multi-word commands that share the same “umbrella” name, e.g., tutorial in this example. All names in the command may be shortened, so tut high is an accepted alternative to tutorial highlight, which minimizes the typing burden on the user. Note also that the ChimeraXClassifier tag text may be split over multiple lines for readability. Whitespace characters around :: are ignored.

src

src is the folder containing the source code for the Python package that implements the bundle functionality. The ChimeraX devel command, used for building and installing bundles, automatically includes all .py files in src as part of the bundle. (Additional files may also be included using bundle information tags such as DataFiles as shown in Bundle Example: Add a Tool.) The only required file in src is __init__.py. Other .py files are typically arranged to implement different types of functionality. For example, cmd.py is used for command-line commands; tool.py or gui.py for graphical interfaces; io.py for reading and saving files, etc.

src/__init__.py

The command registration code is essentially the same as Bundle Example: Hello World, except that the command information, ci, is used to get the full name (as listed in bundle_info.xml) of the command to be registered, and the corresponding function and description are retrieved from the cmd module.

# vim: set expandtab shiftwidth=4 softtabstop=4:

from chimerax.core.toolshed import BundleAPI


# Subclass from chimerax.core.toolshed.BundleAPI and
# override the method for registering commands,
# inheriting all other methods from the base class.
class _MyAPI(BundleAPI):

    api_version = 1     # register_command called with BundleInfo and
                        # CommandInfo instance instead of command name
                        # (when api_version==0)

    # Override method
    @staticmethod
    def register_command(bi, ci, logger):
        # bi is an instance of chimerax.core.toolshed.BundleInfo
        # ci is an instance of chimerax.core.toolshed.CommandInfo
        # logger is an instance of chimerax.core.logger.Logger

        # This method is called once for each command listed
        # in bundle_info.xml.  Since we list two commands,
        # we expect two calls to this method.

        # We check the name of the command, which should match
        # one of the ones listed in bundle_info.xml
        # (without the leading and trailing whitespace),
        # and import the function to call and its argument
        # description from the ``cmd`` module.
        # If the description does not contain a synopsis, we
        # add the one in ``ci``, which comes from bundle_info.xml.
        from . import cmd
        if ci.name == "tutorial cofm":
            func = cmd.cofm
            desc = cmd.cofm_desc
        elif ci.name == "tutorial highlight":
            func = cmd.highlight
            desc = cmd.highlight_desc
        else:
            raise ValueError("trying to register unknown command: %s" % ci.name)
        if desc.synopsis is None:
            desc.synopsis = ci.synopsis

        # We then register the function as the command callback
        # with the chimerax.core.commands module.
        from chimerax.core.commands import register
        register(ci.name, desc, func)


# Create the ``bundle_api`` object that ChimeraX expects.
bundle_api = _MyAPI()

src/cmd.py

cmd.py contains the functions that implement the bundle commands. For example, the cofm function is called when the user issues a tutorial cofm command. To report the center of mass of a set of atoms, cofm requires several parameters supplied by the user:

1. the atoms of interest,

2. in which coordinate system to do the computation (using the atomic coordinates from the input file, or include geometric transformations relative to other models), and

3. whether the center calculation is weighted by the atomic masses.

It then takes the parameters, computes the center of mass, and reports the result to the ChimeraX log. The missing link is how the user-typed command gets translated into a call to cofm. This is the purpose of the call to chimerax.core.commands.register in the register_command method in __init__.py. The register call tells ChimeraX to associate a function and description with a command name. In this case, cofm and cofm_desc are the function and description associated with the command tutorial cofm. When the user types a command that starts with tutorial cofm (or some abbreviation thereof), ChimeraX parses the input text according to a standard syntax, maps the input words to function arguments using the command description, and then calls the function.

The standard syntax of ChimeraX commands is of the form:

command required_arguments optional_arguments

command is the command name, possibly abbreviated and multi-word. Required arguments appear immediately after the command. If there are multiple required arguments, they must be specified in a prespecified order, i.e., they must all be present and are positional. Optional arguments appear after required arguments. They are typically keyword-value pairs and, because they are keyword-based, may be in any order.

A command description instance describes how to map input text to Python values. It contains a list of 2-tuples for required arguments and another for optional arguments. The first element of the 2-tuple is a string that matches one of the command function parameter names. The second element is a “type class”. ChimeraX provides a variety of built-in type classes such as BoolArg (Boolean), IntArg (integer), AtomsArg (container of atoms), and AtomSpecArg (atom specifier). See chimerax.core.commands for the full list. The order of the required parameters list (in the command description) must match the expected order for required arguments (in the input text).

# vim: set expandtab shiftwidth=4 softtabstop=4:

from chimerax.core.commands import CmdDesc      # Command description
from chimerax.atomic import AtomsArg            # Collection of atoms argument
from chimerax.core.commands import BoolArg      # Boolean argument
from chimerax.core.commands import ColorArg     # Color argument
from chimerax.core.commands import IntArg       # Integer argument
from chimerax.core.commands import EmptyArg     # (see below)
from chimerax.core.commands import Or, Bounded  # Argument modifiers


# ==========================================================================
# Functions and descriptions for registering using ChimeraX bundle API
# ==========================================================================


def cofm(session, atoms, *, weighted=False, transformed=True):
    """Report center of mass of given atoms."""

    # ``session``     - ``chimerax.core.session.Session`` instance
    # ``atoms``       - ``chimerax.atomic.Atoms`` instance or None
    # ``weighted``    - boolean, whether to include atomic mass in calculation
    # ``transformed`` - boolean, use scene rather than original coordinates

    atoms, coords, cofm = _get_cofm(session, atoms, transformed, weighted)
    session.logger.info("%s center of mass: %s" %
                        ("weighted" if weighted else "unweighted", cofm))


cofm_desc = CmdDesc(required=[("atoms", Or(AtomsArg, EmptyArg))],
                    keyword=[("weighted", BoolArg),
                              ("transformed", BoolArg)])

# CmdDesc contains a description of how the user-typed command arguments
# should be translated into Python arguments for the function that actually
# implements the command.  There are three styles:
#   required:  the user must provide such arguments and the Python function
#       should declare them as mandatory (i.e. with no default value).
#   optional:  the user can optionally provide these arguments immediately
#       after the mandatory arguments.  The Python function should provide
#       default values for them.
#   keyword:  the user must provide a keyword to specify these arguments, but
#       they can be in any order after the required and optional arguments.
#       The Python function normally declares them after a '*,' (which
#       indicates the start of Python keyword-only arguments)
#
# Most commands should only use required and keyword arguments.  Optional
# arguments should be used in the rare case that an argument's meaning is
# obvious from its position, and it is acceptable for the argument to be
# missing.  For instance, the ChimeraX 'torsion' command requires an atom
# spec specifying four atoms as its first argument, but accepts an optional
# floating point value as a second argument.  If only the first argumeht is
# given, the torsion angle value is reported.  If both arguments are given,
# then the torsion angle is set to that value.
#
# The required/optional/keyword descriptions are passed as keyword arguments
# to the ``CmdDesc`` constructor.  Each set of descriptions is passed as a
# list of 2-tuples.  The first element of the tuple must match the name of a
# parameter of the callback function.  The second element must be a class
# describing the expected input; ChimeraX provides many such classes, e.g.,
# BoolArg for boolean values, IntArg for integer values, etc.  The order of
# the 2-tuples in the required and optional lists determine the order that
# the user must provide those arguments to the command.  That need not be the
# same order as the arguments of the callback function, though it is typically
# good programming practice to have the orders the same.

# For the "cofm" command, we declare three arguments:
#   ``atoms``       - collection of atoms (required), default: all atoms
#   ``weighted``    - boolean (keyword), default: False
#   ``transformed`` - boolean (keyword), default: True
#
# Example commands:
#   tut cofm /A                (cofm of chain A)
#   tut cofm weighted t        (weighted cofm of all atoms)
#   tut cofm :23 trans false   (cofm of input coordinates of residue 23)
#
# Note the trick used for the "atoms" argument, which may be left out to
# mean "use all atoms".  If we make "atoms" a keyword argument, the user
# would have to enter "tut cofm atoms /A" rather than "tut cofm /A".
# The trick is to make "atoms" required, so that the typed command does not
# need to include the "atoms" keyword; the value for "atoms" can be either
# an AtomsArg or an EmptyArg.  If the user enters an atom specification as
# part of the command, then "atoms" value matches AtomsArg, which
# translates to a ``chimerax.atomic.Atoms`` instance for the function
# parameter; if not, "atoms" matches EmptyArg, which translates to ``None``.
#
# The astute reader will note that the "atoms" argument could have instead 
# been declared as optional, with a parser class of AtomsArg and a Python
# default value of None.  In ChimeraX, commands that require some kind of
# atom specification typically have that as their first argument.  If the
# command has additional required arguments, then you would have to use the
# "trick" demonstrated here in order to allow the atom spec to be the first
# argument while still allowing that argument to be omitted in order to
# indicate "all atoms".


def highlight(session, atoms, color, *, weighted=False, transformed=True, count=1):
    """Highlight the atoms nearest the center of mass of given atoms."""

    # ``session``     - ``chimerax.core.session.Session`` instance
    # ``atoms``       - ``chimerax.atomic.Atoms`` instance or None
    # ``color``       - ``chimerax.core.colors.Color` instance
    # ``weighted``    - boolean, whether to include atomic mass in calculation
    # ``transformed`` - boolean, use scene rather than original coordinates

    # Compute the center of mass first
    atoms, coords, cofm = _get_cofm(session, atoms, transformed, weighted)

    # Compute the distance of each atom from the cofm
    # using the NumPy vector norm function
    from numpy.linalg import norm
    distances = norm(coords - cofm, axis=1)

    # Sort the array and get the "count" indices to the closest atoms
    if count > len(atoms):
        count = len(atoms)
    from numpy import argsort
    atom_indices = argsort(distances)[:count]

    # Create a collection of atoms from the indices
    chosen = atoms[atom_indices]

    # Update their "colors".  Assigning a single value to an
    # array means assign the same value for all elements.
    chosen.colors = color.uint8x4()


highlight_desc = CmdDesc(required=[("atoms", Or(AtomsArg, EmptyArg)),
                                   ("color", ColorArg)],
                         keyword=[("weighted", BoolArg),
                                   ("transformed", BoolArg),
                                   ("count", Bounded(IntArg, 1, 5))])


# ==========================================================================
# Functions intended only for internal use by bundle
# ==========================================================================


def _get_cofm(session, atoms, transformed, weighted):
    # ``session``     - ``chimerax.core.session.Session`` instance
    # ``atoms``       - ``chimerax.atomic.Atoms`` instance
    # ``transformed`` - boolean, use scene rather than original coordinates
    # ``weighted``    - boolean, whether to include atomic mass in calculation

    # If user did not specify the list of atoms, use all atoms
    if atoms is None:
        from chimerax.atomic import all_atoms
        atoms = all_atoms(session)

    # We can use either transformed or untransformed coordinates.
    # Transformed coordinates are "scene coordinates", which
    # takes into account translation and rotation of individual
    # models.  Untransformed coordinates are the coordinates
    # read from the data files.
    if transformed:
        coords = atoms.scene_coords
    else:
        coords = atoms.coords

    # ``coords`` is a ``numpy`` float array of shape (N, 3)
    # If we want weighted center, we have to multiply coordinates
    # by the atomic mass
    if not weighted:
        cofm = coords.mean(axis=0)
    else:
        m = atoms.elements.masses
        c = coords * m[:,None]
        cofm = c.sum(axis=0) / m.sum()

    # To get the average coordinates, we use ``numpy.mean``
    # print("DEBUG: center of mass:", cofm)
    return atoms, coords, cofm

cofm() is the function called from __init__.py when the user enters the cofm command. It retrieves the array of atoms, their coordinates, and their center of mass by calling the internal function _get_cofm() and reports the result via session.logger, an instance of chimerax.core.logger.Logger.

cofm_desc contains the description of what arguments are required or allowed for the cofm command. The details of its declaration are described in the comments in the example.

highlight() is the function called from __init__.py when the user enters the highlight command. Like cofm(), it retrieves the array of atoms, their coordinates, and their center of mass by calling _get_cofm(). It then

1. computes the distances from each atom to the center of mass using Numpy (line 104),

2. sorts the atom indices by distances so that indices of atoms that are closer to the center of mass are towards the front of the sort result (argsort(distances)), and select the first count indices (line 110),

3. turn the array of indices into an array of atoms (line 113), and

4. finally, set the color of the selected atoms (line 117). The colors attribute of the atomic array is an Nx4 array of integers, where N is the number of atoms and the rows (of 4 elements) are the RGBA values for each atom. The color argument to highlight() is an instance of chimerax.core.colors.Color, whose uint8x4() returns its RGBA value as an array of four (x4) 8-bit integers (uint8).

_get_cofm(), used by both cofm() and highlight(), is passed three arguments:

• atoms, the atoms specified by the user, if any.

• transformed, whether to retrieve transformed (scene) or untransformed (original) coordinates. Untransformed coordinates can typically be used when only a single model is involved because the atoms are fixed relative to each other. Transformed coordinates must be used when distances among multiple models are being computed (i.e., the models must all be in same coordinate system).

• weighted, whether to include atomic mass as part of the center of mass computation. Frequently, an unweighted average of atomic coordinates, which is simpler and faster to compute, is sufficient for qualitative analysis.

If the user did not choose specific atoms (when atoms is None), the usual ChimeraX interpretation is that all atoms should be used (lines 139-141). chimerax.atomic.all_atoms() returns an array of atoms from all open models. Transformed and untransformed coordinates are accessed using the scene_coords and coords attributes of the atom array, respectively (lines 148-151). If atomic mass need not be included, the “center of mass” is simply the average of the coordinates (line 157); if a weighted calculation is required, (a) the atomic masses are retrieved by atoms.elements.masses (line 159), (b) the coordinates are scaled by the corresponding atomic masses (line 160), and (c) the weighted average is computed (line 161).

For performance, ChimeraX makes use of NumPy arrays in many contexts. The container for atoms is typically a chimerax.atomic.Collection instance, as are those for bonds, residues, and atomic structures. Fetching the same attribute, e.g., coordinates, from a collection of molecular data, e.g., atoms, usually results in a NumPy array. Although code involving NumPy arrays is sometimes opaque, they are typically much more efficient than using Python loops.

src/docs/user/commands/tutorial.html

The documentation for the tutorial command should be written in HTML 5 and saved in a file whose name matches the command name and has suffix .html, i.e., tutorial.html. If the bundle command is a subcommand of an existing command (e.g. color bundlecoloring) then any spaces should be replaced by underscores. When help files are included in bundles, documentation for the commands may be displayed using the help command, the same as built-in ChimeraX commands. The directory structure is chosen to allow for multiple types of documentation for a bundle. For example, developer documentation such as the bundle API are saved in a devel directory instead of user; documentation for graphical tools are saved in user/tools instead of user/commands.

<html><head>
<link rel="stylesheet" type="text/css" href="../userdocs.css" />
<title>Command: tutorial</title>
</head><body>

<a name="top"></a>
<a href="../index.html">
<img width="60px" src="../ChimeraX-docs-icon.svg" alt="ChimeraX docs icon"
class="clRight" title="User Guide Index"/></a>

<h3><a href="../index.html#commands">Command</a>: tutorial</h3>
<h3 class="usage"><a href="usageconventions.html">Usage</a>:
<br><b>tutorial cofm</b>
&nbsp;<a href="atomspec.html"><i>spec</i></a>
[&nbsp;<b>weighted</b>&nbsp;true&nbsp;|&nbsp;false&nbsp;]
[&nbsp;<b>transformed</b>&nbsp;true&nbsp;|&nbsp;false&nbsp;]
</h3>
<h3 class="usage"><a href="usageconventions.html">Usage</a>:
<br><b>tutorial highlight</b>
&nbsp;<a href="atomspec.html"><i>spec</i></a>
&nbsp;<a href="color.html#colorname"><i>colorname</i></a>
[&nbsp;<b>weighted</b>&nbsp;true&nbsp;|&nbsp;false&nbsp;]
[&nbsp;<b>transformed</b>&nbsp;true&nbsp;|&nbsp;false&nbsp;]
[&nbsp;<b>count</b>&nbsp;<i>count</i>&nbsp;]
</h3>

<a name="cofm"/>
<p>
The <b>tutorial cofm</b> command reports the
center of mass for <a href="atomspec.html"><i>spec</i></a>.
By default, the unweighted average of the transformed coordinates
of the specified atoms is reported.
Untransformed coordinates are the values read from the input file,
while transformed coordinates include all user-applied rotations
and translations.  If atoms from multiple models are specified,
transformed coordinates reflect their relative positions correctly
while untransformed coordinates may not (unless the model coordinates
started in the same coordinate system).  To get the mass-weighted
center, use the <b>weighted true</b> option.  To get the center
for untransformed coordinates, use the <b>transformed false</b>
option.
</p>
<a name="highlight"/>
<p>
The <b>tutorial highlight</b> command changes the color
of the atom(s) nearest the center of the specified atoms.
The <b>weighted</b> and <b>transformed</b> options are
the same as in the <b><a href="#cofm">tutorial cofm</a></b>
command.  The <b>count</b> option specifies the number
of atoms whose color should be set.  The default is one
and may be as many as five.
</p>

<hr>
<address>UCSF Resource for Biocomputing, Visualization, and Informatics / 
April 2018</address>
</body></html>

While the only requirement for documentation is that it be written as HTML, it is recommended that developers write command help files following the above template, with:

• a banner linking to the documentation index,

• a usage section with a summary of the command syntax,

• text describing each command in the bundle, and

• an address for contacting the bundle author.

Note that the target links used in the HTML file are all relative to ... Even though the command documentation HTML file is stored with the bundle, ChimeraX treats the links as if the file were located in the commands directory in the developer documentation tree. This creates a virtual HTML documentation tree where command HTML files can reference each other without having to be collected together.

Building and Testing Bundles

To build a bundle, start ChimeraX and execute the command:

devel build PATH_TO_SOURCE_CODE_FOLDER

Python source code and other resource files are copied into a build sub-folder below the source code folder. C/C++ source files, if any, are compiled and also copied into the build folder. The files in build are then assembled into a Python wheel in the dist sub-folder. The file with the .whl extension in the dist folder is the ChimeraX bundle.

To test the bundle, execute the ChimeraX command:

devel install PATH_TO_SOURCE_CODE_FOLDER

This will build the bundle, if necessary, and install the bundle in ChimeraX. Bundle functionality should be available immediately.

To remove temporary files created while building the bundle, execute the ChimeraX command:

devel clean PATH_TO_SOURCE_CODE_FOLDER

Some files, such as the bundle itself, may still remain and need to be removed manually.

Building bundles as part of a batch process is straightforward, as these ChimeraX commands may be invoked directly by using commands such as:

ChimeraX --nogui --exit --cmd 'devel install PATH_TO_SOURCE_CODE_FOLDER exit true'

This example executes the devel install command without displaying a graphics window (--nogui) and exits immediately after installation (exit true). The initial --exit flag guarantees that ChimeraX will exit even if installation fails for some reason.

Distributing Bundles

With ChimeraX bundles being packaged as standard Python wheel-format files, they can be distributed as plain files and installed using the ChimeraX toolshed install command. Thus, electronic mail, web sites and file sharing services can all be used to distribute ChimeraX bundles.

Private distributions are most useful during bundle development, when circulation may be limited to testers. When bundles are ready for public release, they can be published on the ChimeraX Toolshed, which is designed to help developers by eliminating the need for custom distribution channels, and to aid users by providing a central repository where bundles with a variety of different functionality may be found.

Customizable information for each bundle on the toolshed includes its description, screen captures, authors, citation instructions and license terms. Automatically maintained information includes release history and download statistics.

To submit a bundle for publication on the toolshed, you must first sign in. Currently, only Google sign in is supported. Once signed in, use the Submit a Bundle link at the top of the page to initiate submission, and follow the instructions. The first time a bundle is submitted to the toolshed, it is held for inspection by the ChimeraX team, which may contact the authors for more information. Once approved, all subsequent submissions of new versions of the bundle are posted immediately on the site.

What’s Next

• Bundle Example: Hello World (previous topic)

• Bundle Example: Add a Command (current topic)

• Bundle Example: Add a Tool (next topic)

• Bundle Example: Read a New File Format

• Bundle Example: Save a New File Format

• Bundle Example: Fetch from Network Database

• Bundle Example: Define a Chemical Subgroup Selector

• Bundle Example: Define Presets