python.extractor_dict.expMetaData Class Reference

Inheritance diagram for python.extractor_dict.expMetaData:

Public Member Functions
def	__init__ (self, expname, inputfile)
def	translateKey (self, key)
def	md_gen (self, mdart, md0={})
def	getmetadata (self, md0={})
Public Member Functions inherited from python.extractor_dict.MetaData
def	__init__ (self, inputfile)
def	extract_metadata_to_pipe (self)
def	get_job (self, proc)

Public Attributes
	expname
	translateKeyf
	md
Public Attributes inherited from python.extractor_dict.MetaData
	inputfile

Additional Inherited Members
Static Public Member Functions inherited from python.extractor_dict.MetaData
def	wait_for_subprocess (jobinfo, q)
def	mdart_gen (jobtuple)
def	md_handle_application (md)

Detailed Description

Class to hold/interpret experiment-specific metadata

Definition at line 103 of file extractor_dict.py.

Constructor & Destructor Documentation

def python.extractor_dict.expMetaData.__init__	(		self,
			expname,
			inputfile
	)

Definition at line 105 of file extractor_dict.py.

    def __init__(self, expname, inputfile):
        MetaData.__init__(self, inputfile)
        self.expname = expname
        #self.exp_md_keyfile = expname + '_metadata_key'
        try:
            #translateMetaData = __import__("experiment_utilities", "MetaDataKey")
            from experiment_utilities import MetaDataKey
        except ImportError:
            print("You have not defined an experiment-specific metadata and key-translating module in experiment_utilities. Exiting")
            raise
            
        metaDataModule = MetaDataKey()
        self.metadataList, self.translateKeyf = metaDataModule.metadataList(), metaDataModule.translateKey

Member Function Documentation

def python.extractor_dict.expMetaData.getmetadata	(		self,
			md0 = {}
	)

Get metadata from input file and return as python dictionary.
Calls other methods in class and returns metadata dictionary

Definition at line 220 of file extractor_dict.py.

    def getmetadata(self, md0={}):
        """ Get metadata from input file and return as python dictionary.
        Calls other methods in class and returns metadata dictionary"""
        proc = self.extract_metadata_to_pipe()
        jobt = self.get_job(proc)
        mdart = self.mdart_gen(jobt)
        return self.md_gen(mdart, md0)  

def python.extractor_dict.expMetaData.md_gen	(		self,
			mdart,
			md0 = {}
	)

Loop through art metdata, generate metadata dictionary

Definition at line 123 of file extractor_dict.py.

    def md_gen(self, mdart, md0={}):
        """Loop through art metdata, generate metadata dictionary"""
        # define an empty python dictionary which will hold sam metadata.
        # Some fields can be copied directly from art metadata to sam metadata.
        # Other fields require conversion.
        md = {}
        
        

        # Loop over art metadata.
        mixparents = []
        for mdkey, mdval in list(mdart.items()):
            # mdval = mdart[mdkey]
            
            # Skip some art-specific fields.
            # Ignore primary run_type field (if any).
            # Instead, get run_type from runs field.
            if mdkey in ['file_format_version', 'file_format_era', 'run_type']:
                pass
            elif mdkey in ['art.file_format_version', 'art.file_format_era', 'art.run_type']:
                pass

            # Ignore new-style first/last event (old-style first/last handled below).
            elif mdkey in ['art.first_event', 'art.last_event']:
                pass

            # Ignore data_stream if it begins with "out".
            # These kinds of stream names are probably junk module labels.

            # First check if the data_stream is just "out" Catches an edge case 
            # where the stream does not have a number

            elif mdkey == 'data_stream' and mdval == 'out':
                pass

            elif mdkey == 'data_stream' and mdval[:3] == 'out' and \
                    mdval[3] >= '0' and mdval[3] <= '9':
                pass
            
            # Application family/name/version.
            elif mdkey == 'applicationFamily' or mdkey == 'application.family':
                md['application'], md['application']['family'] = self.md_handle_application(md), mdval
            elif mdkey == 'process_name' or mdkey == 'art.process_name':
                md['application'], md['application']['name'] = self.md_handle_application(md), mdval
            elif mdkey == 'applicationVersion' or mdkey == 'application.version':
                md['application'], md['application']['version'] = self.md_handle_application(md), mdval

            # Parents.
            elif mdkey == 'parents':
                md['parents'] = [{'file_name': parent} for parent in mdval]
            
            elif mdkey.startswith('mixparent'):
                mixparents.append(mdval.strip(' ,"') )  
                
            # Other fields where the key or value requires minor conversion.
            elif mdkey in ['first_event', 'last_event']:
                if (type(mdval) == type([]) or type(mdval) == type(())) and len(mdval) >= 3:
                    md[mdkey] = mdval[2]
                else:
                    md[mdkey] = mdval
            elif mdkey in self.metadataList:
                #print mdkey
                trkey = self.translateKey(mdkey)
                #print trkey
                md[trkey] = mdval
            elif mdkey == 'fclName':
                md['fcl.name'] = mdval
            elif mdkey == 'fclVersion':
                md['fcl.version'] = mdval
            
            #For all other keys, copy art metadata directly to sam metadata.
            #This works for run-tuple (run, subrun, runtype) and time stamps.
            else:
                md[mdkey] = mdval
                        
        # Merge mix parents into normal parents.
        
        for mixparent in mixparents:
           mixparent_dict = {'file_name': mixparent}
           md['parents'].append(mixparent_dict)    

        # Get the other meta data field parameters.  
        
        md['file_name'] = self.inputfile.split("/")[-1]
        if 'file_size' in md0:
            md['file_size'] = md0['file_size']
        else:
            md['file_size'] = os.path.getsize(self.inputfile)
        if 'crc' in md0:
            md['crc'] = md0['crc']
        else:
            md['crc'] = root_metadata.fileEnstoreChecksum(self.inputfile)

        # In case we ever want to check out what md is for any instance of MetaData by calling instance.md
        self.md = md
        return self.md

def python.extractor_dict.expMetaData.translateKey	(		self,
			key
	)

Returns the output of the imported translateKey function (as translateKeyf) called on key

Definition at line 119 of file extractor_dict.py.

    def translateKey(self, key):
        """Returns the output of the imported translateKey function (as translateKeyf) called on key"""
        return self.translateKeyf(key)

Member Data Documentation

python.extractor_dict.expMetaData.expname

Definition at line 107 of file extractor_dict.py.

python.extractor_dict.expMetaData.md

Definition at line 217 of file extractor_dict.py.

python.extractor_dict.expMetaData.translateKeyf

Definition at line 117 of file extractor_dict.py.

---

The documentation for this class was generated from the following file:

• larbatch/python/extractor_dict.py