Tagged Data Manipulation

SD Tagged Data Manipulation

Meta information about a molecule is stored in what is known as ‘tagged data’. The most common example of this is the data fields found in SDF files. Since SD files are a common form of data storage and transfer from one system to another, OEChem TK provides several methods to manipulate this data. OEChem TK also provides round tripping SD data through the OEBinary, .oeb, and comma-separated-values, .csv file formats.

A simple class, OESDDataPair is used to set and retrieve SD data.

Data	Set method	Get method
tag	OESDDataPair.SetTag	OESDDataPair.GetTag
data	OESDDataPair.SetValue	OESDDataPair.GetValue

The following functions provide access to the SD data.

Functions to manipulate SD data

Function	Description
OEIsSDDataFormat	determines whether a file format support SD data
OESetSDData	set a tag and value data pair
OEAddSDData	add a tag and value data pair
OEHasSDData	determine whether a molecule has data with a given tag
OEGetSDData	get the value for the given tag
OEGetSDDataPairs	return an iterator over all the SD data pairs of the molecule
OECopySDData	copy the entire set of SD data from one molecule to another
OEDeleteSDData	delete all SD data items with the given tag
OEClearSDData	clear all SD data from a molecule

Since OESDDataPair stores SD data information in a string, numeric values have to be converted to a string with Python’s str() method before storing a value with either OESetSDData or OEAddSDData functions.

oechem.OESetSDData(mol, "number of atoms", str(mol.NumAtoms()))

Similarly the retrieved string can be converted to a numeric value with int() or float() functions.

if oechem.OEHasSDData(mol, "weight"):
    weight = float(oechem.OEGetSDData(mol, "weight"))
    print("weight=", weight)

The following example shows how to manipulate SD tagged data.

Listing 1: SD data manipulation

from openeye import oechem


def DumpSDData(mol):
    print("SD data of", mol.GetTitle())
    # loop over SD data
    for dp in oechem.OEGetSDDataPairs(mol):
        print(dp.GetTag(), ':', dp.GetValue())
    print()


mol = oechem.OEGraphMol()
oechem.OESmilesToMol(mol, "c1ccccc1")
mol.SetTitle("benzene")

# set some tagged data
oechem.OESetSDData(mol, "color", "brown")
oechem.OESetSDData(mol, oechem.OESDDataPair("size", "small"))
DumpSDData(mol)

# check for existence of data, then delete it
if oechem.OEHasSDData(mol, "size"):
    oechem.OEDeleteSDData(mol, "size")
DumpSDData(mol)

# add additional color data
oechem.OEAddSDData(mol, "color", "black")
DumpSDData(mol)

# remove all SD data
oechem.OEClearSDData(mol)
DumpSDData(mol)

Note

If a data with the same tag already exists:

• it is replaced, when using OESetSDData function

• it is added, when using OEAddSDData function

The output of the preceding program is the following:

SD data of benzene
color : brown
size : small

SD data of benzene
color : brown

SD data of benzene
color : brown
color : black

SD data of benzene

Note

Note that SD tagged data is specific to MDL’s SD file format. Any SD data added to a molecule will only be written out to SD files, OEBinary files, or CSV files. The SD data fields will only be filled when reading from SD files that contain SD tagged data, CSV data files, or from OEBinary files previously created to contain this data.

See also

• Generic Data chapter

CSV File Format

The CSV, comma-separated-values, file format is a comma file format used for data exchange in a wide variety of software. Over the years, the popularity of the format has out-stripped the robustness of many of its implementations leading to some subtle differences in how it is implemented between different software packages. This has led to creation and adoption of RFC 4180 as a standard for the CSV format. OEChem TK supports this standard as the base format for reading and writing molecules, OEMolBase objects, to .csv files. OEChem TK will read and write .csv files in the following layout:

1. The first line can optionally contain a header to use as the SD tag.

   • OEChem TK decides if the first line is a header by trying to interpret the first column of the first line as SMILES. See the OEIFlavor_CSV_Header flavor.

2. The first column is expected to be a SMILES string representing the molecule.

3. The second column is the molecule title, as accessed through OEMolBase.GetTitle.

4. Remaining columns will be read into SD data fields. The data is then accessible through OEGetSDData and OESetSDData.

   • If the file did not contain a header line, the SD tags will be named in the following pattern: OE_CSV_COLUMN_1, OE_CSV_COLUMN_2, …, OE_CSV_COLUMN_N.

5. New-line characters are not supported inside a CSV record. They are round-tripped using the pipe character, |, instead.

Point #5 is a subtlety of meta-data interchange between SD files and CSV files. On the one hand, many CSV implementations do not support line-breaks inside CSV fields, Microsoft Excel does not and neither does OEChem TK. However, it is common to use multiple lines inside an SD data file like the following:

>  <activ_class>
Antiarthritic
Antiinflammatory
Cyclooxygenase-2 Inhibitor

OEChem TK implements a solution to this problem that ChemAxon has implemented and documented here. The solution is to use the pipe character, |, to represent line breaks inside a data field. For example, the previous example of a multi-line SD data field would look like the following:

SMILES,TITLE,active_class
...,...,Antiarthritic|Antiinflammatory|Cyclooxygenase-2 Inhibitor

This does cause a different problem for pipe characters inside data fields: how pipe characters are escaped? OEChem TK works around this problem by embedding the same semantics for CSV parsing to parse multi-line fields, just using the pipe character as the separator and the single quote character as the quoting escape character. Listing 2 demonstrates an example that writes out a tricky set of data to the OEFormat_CSV format.

Listing 2: Writing out SD data as CSV

from openeye import oechem

ofs = oechem.oemolostream()
ofs.SetFormat(oechem.OEFormat_CSV)

mol = oechem.OEGraphMol()
oechem.OESmilesToMol(mol, "O")
mol.SetTitle("water")

oechem.OESetSDData(mol, "phases", "gas,liquid,solid")
oechem.OESetSDData(mol, "uses", "bathing\nwater guns\ntea|earl grey|hot")

oechem.OEWriteMolecule(ofs, mol)

The output of Listing 2 will look like the following:

SMILES,TITLE,phases,uses
O,water,"gas,liquid,solid",bathing|water guns|'tea|earl grey|hot'

Code Example

• Reordering CSV File OpenEye Python Cookbook recipe

PDB Tagged Data Manipulation

The OEPDBDataPair class is used to set and retrieve PDB data pairs.

Data	Set method	Get method
tag	OEPDBDataPair.SetTag	OEPDBDataPair.GetTag
data	OEPDBDataPair.SetValue	OEPDBDataPair.GetValue

If you wish to store a numeric value, use Python’s str() method to convert it to a string and then use int() or float() on the value when retrieving the data (see examples in SD Tagged Data Manipulation).

The following functions provide access to the PDB data.

Functions to manipulate PDB data

Function	Description
OESetPDBData	set a tag and value data pair
OEAddPDBData	add a tag and value data pair
OEHasPDBData	determine whether a molecule has data with a given tag
OEGetPDBData	get the value for the given tag
OEGetPDBDataPairs	return an iterator over all the PDB data pairs of the molecule
OECopyPDBData	copy the entire set of PDB data from one molecule to another
OEDeletePDBData	delete all PDB data items with the given tag
OEClearPDBData	clear all PDB data from a molecule

Note

In case of PDB header items like REMARK, each line is treated as a separate instance, Therefore these multiple lines have to be added with OEAddPDBData and can be accessed via OEGetPDBDataPairs.

The following PDB fields are stored as tagged PDB data when OEIFlavor_PDB_DATA input flavor is set:

Imported PDB data fields

AUTHOR	CAVEAT	COMPND	CRYST1	DBREF
EXPDTA	FORMUL	HEADER	HELIX	HET
HETNAM	HETSYM	JRNL	KEYWDS	MODRES
MTRIX1	MTRIX2	MTRIX3	OBSLTE	ORIGX1
ORIGX2	ORIGX3	REMARK	REVDAT	SCALE1
SCALE2	SCALE3	SEQRES	SEQADV	SHEET
SITE	SOURCE	SPRSDE	SSBOND	TITLE
TURN

Warning

The tags of PDB data are always 6 characters long and space-padded (for example "HELIX " and not "HELIX" ).

The following example shows how to manipulate PDB tagged data.

Listing 3: PDB data manipulation

from openeye import oechem
import sys

if len(sys.argv) != 2:
    oechem.OEThrow.Usage("%s <pdbfile>" % sys.argv[0])

ifs = oechem.oemolistream()
if not ifs.open(sys.argv[1]):
    oechem.OEThrow.Fatal("Unable to open %s" % sys.argv[1])

# need to set input flavor to ensure PDB data is stored on molecule
ifs.SetFlavor(oechem.OEFormat_PDB, (oechem.OEIFlavor_Generic_Default |
                                    oechem.OEIFlavor_PDB_Default |
                                    oechem.OEIFlavor_PDB_DATA))

mol = oechem.OEGraphMol()
while oechem.OEReadMolecule(ifs, mol):
    if oechem.OEHasPDBData(mol, "COMPND"):
        print("COMPND:")
        print(oechem.OEGetPDBData(mol, "COMPND"))

    if oechem.OEHasPDBData(mol, "HELIX "):
        print("HELIX:")
        print(oechem.OEGetPDBData(mol, "HELIX "))

    if oechem.OEHasPDBData(mol, "SSBOND"):
        print("SSBOND:")
        for dp in oechem.OEGetPDBDataPairs(mol):
            if dp.GetTag() == "SSBOND":
                print(dp.GetValue())

The output of the preceding program for 1D1H is the following:

COMPND:
    MOL_ID: 1;
HELIX:
   1   1 THR A   11  ASP A   14  5
SSBOND:
   1 CYS A    2    CYS A   16                          1555   1555
   2 CYS A    9    CYS A   21                          1555   1555
   3 CYS A   15    CYS A   28                          1555   1555

Note

Note that PDB tagged data is specific to PDB file format. Any PDB data added to a molecule will only be written out to PDB files or OEBinary files.

See also

• Generic Data chapter

Multi-conformer molecules

See also

For using tag data with multi-conformer molecules, see Dude, where’s my SD data?.

MMCIF Metadata Manipulation

MMCIF metadata is not stored as tagged data in the same way as PDB data is. Instead the MMCIF metadata is stored as a single string that can be retrieved, manipulated and set or appended to.

The following functions provide access to the MMCIF data.

Functions to manipulate MMCIF metadata

Function	Description
OESetMMCIFData	sets MMCIF metadata on a molecule, overwrites if any existing metadata
OEAddMMCIFData	adds MMCIF metadata on a molecule, appends to any existing metadata
OEHasMMCIFData	determine whether a molecule has MMCIF metadata
OEGetMMCIFData	get the MMCIF metadata stored on a molecule
OECopyMMCIFData	copy the MMCIF metadata from one molecule to another
OEClearMMCIFData	clear the MMCIF metadata from a molecule

See also

• Generic Data chapter