SortModuleTimes Namespace Reference

Classes
class	EntryDataClass
class	EventKeyClass
class	FormatError
class	JobStatsClass
class	MaxItemLengthsClass
class	ModuleKeyClass
class	NoMoreInput
class	Stats
class	TabularAlignmentClass
class	TimeModuleStatsClass

Functions
def	signed_sqrt (value)
def	ParseTimeModuleLine (line)
def	ParseTimeEventLine (line)
def	OPEN (Path, mode='r')
def	ParseInputFile (InputFilePath, AllStats, EventStats, options)
def	CenterString (s, w, f=' ')
def	LeftString (s, w, f=' ')
def	RightString (s, w, f=' ')
def	JustifyString (s, w, f=' ')

Variables
string	Version = "%(prog)s 1.5"
string	__doc__ = "Prints statistics of the module timings based on the information from the Timing service."
	Parser = argparse.ArgumentParser(description=__doc__)
	parse command line arguments More...
	PresentMode
	metavar
	nargs
	help
	dest
	action
	const
	type
	int
	default
	version
	options = Parser.parse_args()
	CheckDuplicates
	AllStats = JobStatsClass( )
	parse all inputs, collect the information More...
	EventStats = TimeModuleStatsClass\
	bTrackEntries
int	nErrors = 0
	OutputTable = TabularAlignmentClass()
	print the results More...

Function Documentation

def SortModuleTimes.CenterString	(		s,
			w,
			f = ' '
	)

Returns the string s centered in a width w, padded by f on both sides.

Definition at line 649 of file SortModuleTimes.py.

def CenterString(s, w, f = ' '):
        """Returns the string s centered in a width w, padded by f on both sides."""
        leftFillerWidth = max(0, w - len(s)) / 2
        return f * leftFillerWidth + s + f * (w - leftFillerWidth)
# CenterString()

def SortModuleTimes.JustifyString	(		s,
			w,
			f = ' '
	)

Recomputes the spaces between the words in s so that they fill a width w.

The original spacing is lost. The string is split in words by str.split().
The character f is used to create the filling spaces between the words.
Note that the string can result longer than w if the content is too long.

Definition at line 663 of file SortModuleTimes.py.

def JustifyString(s, w, f = ' '):
        """Recomputes the spaces between the words in s so that they fill a width w.

        The original spacing is lost. The string is split in words by str.split().
        The character f is used to create the filling spaces between the words.
        Note that the string can result longer than w if the content is too long.
        """
        assert len(f) == 1
        tokens = s.split(f)
        if len(tokens) <= 1: return CenterString(s, w, f=f)

        # example: 6 words, 7 spaces (in 5 spacers)
        spaceSize = max(1., float(f - sum(map(len, tokens))) / (len(tokens) - 1))
          # = 1.4
        totalSpace = 0.
        assignedSpace = 0
        s = tokens[0]
        for token in tokens[1:]:
                totalSpace += spaceSize  # 0 => 1.4 => 2.8 => 4.2 => 5.6 => 7.0
                tokenSpace = int(totalSpace - assignedSpace) # int(1.4 1.8 2.2 1.6 2.0)
                s += f * tokenSpace + token # spaces: 1 + 1 + 2 + 1 + 2
                assignedSpace += tokenSpace # 0 => 1 => 2 => 4 => 5 => 7
        # for
        assert assignedSpace == w
        return s
# JustifyString()

def SortModuleTimes.LeftString	(		s,
			w,
			f = ' '
	)

Returns the string s in a width w, padded by f on the right.

Definition at line 655 of file SortModuleTimes.py.

def LeftString(s, w, f = ' '):
        """Returns the string s in a width w, padded by f on the right."""
        return s + f * max(0, w - len(s))

def SortModuleTimes.OPEN	(		Path,
			mode = 'r'
	)

Open a file (possibly a compressed one).

Support for modes other than 'r' (read-only) are questionable.

Definition at line 499 of file SortModuleTimes.py.

def OPEN(Path, mode = 'r'):
        """Open a file (possibly a compressed one).

        Support for modes other than 'r' (read-only) are questionable.
        """
        if Path.endswith('.bz2'): return bz2.BZ2File(Path, mode)
        if Path.endswith('.gz'): return gzip.GzipFile(Path, mode)
        return open(Path, mode)
# OPEN()

def SortModuleTimes.ParseInputFile	(		InputFilePath,
			AllStats,
			EventStats,
			options
	)

Parses a log file.

The art log file at InputFilePath is parsed.
The per-module statistics are added to the existing in AllStats (an instance
of JobStatsClass), creating new ones as needed. Similarly, per-event
statistics are added to EventStats (a TimeModuleStatsClass instance).

options class can contain the following members:
- Permissive (default: false): do not bail out when a format error is found;
  the entry is typically skipped. This often happens because the output line
  of the timing information is interrupted by some other output.
- MaxEvents (default: all events): collect statistics for at most MaxEvents
  events (always the first ones)
- CheckDuplicates (default: false): enables the single-event tracking, that
  allows to check for duplicates

It returns the number of errors encountered.

Definition at line 510 of file SortModuleTimes.py.

def ParseInputFile(InputFilePath, AllStats, EventStats, options):
        """Parses a log file.

        The art log file at InputFilePath is parsed.
        The per-module statistics are added to the existing in AllStats (an instance
        of JobStatsClass), creating new ones as needed. Similarly, per-event
        statistics are added to EventStats (a TimeModuleStatsClass instance).

        options class can contain the following members:
        - Permissive (default: false): do not bail out when a format error is found;
          the entry is typically skipped. This often happens because the output line
          of the timing information is interrupted by some other output.
        - MaxEvents (default: all events): collect statistics for at most MaxEvents
          events (always the first ones)
        - CheckDuplicates (default: false): enables the single-event tracking, that
          allows to check for duplicates

        It returns the number of errors encountered.
        """
        def CompleteEvent(CurrentEvent, EventStats, AllStats):
                """Make sure that CurrentEvent is known to all stats."""
                EventStats.complete(( CurrentEvent, ))
                for ModuleStats in AllStats:
                        ModuleStats.complete(EventStats.getEvents())
        # CompleteEvent()


        LogFile = OPEN(InputFilePath, 'r')

        nErrors = 0
        LastLine = None
        CurrentEvent = None
        for iLine, line in enumerate(LogFile):

                line = line.strip()
                if line == LastLine: continue # duplicate line
                LastLine = line

                if line.startswith("TimeModule> "):

                        try:
                                TimeData = ParseTimeModuleLine(line)
                        except FormatError, e:
                                nErrors += 1
                                msg = "Format error on '%s'@%d" % (InputFilePath, iLine + 1)
                                try: msg += " (%s)" % str(e.data['type'])
                                except KeyError: pass
                                try: msg += ", for event " + str(e.data['event'])
                                except KeyError: pass
                                try: msg += ", module " + str(e.data['module'])
                                except KeyError: pass
                                print >>sys.stderr, msg
                                if not options.Permissive: raise
                                else:                      continue
                        # try ... except

                        try:
                                ModuleStats = AllStats[TimeData.module]
                        except KeyError:
                                ModuleStats = TimeModuleStatsClass \
                                  (TimeData.module, bTrackEntries=options.CheckDuplicates)
                                AllStats[TimeData.module] = ModuleStats
                        #

                        ModuleStats.add(TimeData)
                elif line.startswith("TimeEvent> "):
                        try:
                                TimeData = ParseTimeEventLine(line)
                        except FormatError, e:
                                nErrors += 1
                                msg = "Format error on '%s'@%d" % (InputFilePath, iLine + 1)
                                try: msg += " (%s)" % str(e.data['type'])
                                except KeyError: pass
                                try: msg += ", for event " + str(e.data['event'])
                                except KeyError: pass
                                try: msg += ", module " + str(e.data['module'])
                                except KeyError: pass
                                print >>sys.stderr, msg
                                if not options.Permissive: raise
                                else:                      continue
                        # try ... except

                        EventStats.add(TimeData)
                        if (options.MaxEvents >= 0) \
                          and (EventStats.n() >= options.MaxEvents):
                                if CurrentEvent: CompleteEvent(CurrentEvent, EventStats, AllStats)
                                raise NoMoreInput
                else:
                        TimeData = None
                        continue

                if (CurrentEvent != TimeData.eventKey):
                        if TimeData and CurrentEvent:
                                CompleteEvent(CurrentEvent, EventStats, AllStats)
                        CurrentEvent = TimeData.eventKey
                # if
        # for line in log file
        if CurrentEvent: CompleteEvent(CurrentEvent, EventStats, AllStats)

        return nErrors
# ParseInputFile()


#
# output
#

def SortModuleTimes.ParseTimeEventLine

(

line

)

Parses a line to extract event timing information.

The line must be known to contain event timing information.
The function returns a EntryDataClass including the timing information, or
raises a FormatError if the line has no valid format.

Format 1 (20140226):

TimeEvent> run: 1 subRun: 0 event: 10 0.231838

Definition at line 460 of file SortModuleTimes.py.

def ParseTimeEventLine(line):
        """Parses a line to extract event timing information.

        The line must be known to contain event timing information.
        The function returns a EntryDataClass including the timing information, or
        raises a FormatError if the line has no valid format.

        Format 1 (20140226):

        TimeEvent> run: 1 subRun: 0 event: 10 0.231838
        """
        Tokens = line.split()

        EventKey = None
        time = None
        try:
                EventKey = EventKeyClass((int(Tokens[2]), int(Tokens[4]), int(Tokens[6])))
                time = float(Tokens[7])
        except Exception, e:
                raise FormatError(
                  "TimeEvent format not recognized: '%s' (%s)" % (line, str(e)),
                  type="Event", event=EventKey
                  )
        # try ... except

        if (Tokens[0] != 'TimeEvent>') \
          or (Tokens[1] != 'run:') \
          or (Tokens[3] != 'subRun:') \
          or (Tokens[5] != 'event:') \
          or (len(Tokens) != 8) \
          :
                raise FormatError("TimeEvent format not recognized: '%s'" % line,
                  type="Event", event=EventKey)
        # if

        return EntryDataClass(EventKey, time=time)
# ParseTimeEventLine()

def SortModuleTimes.ParseTimeModuleLine

(

line

)

Parses a line to extract module timing information.

The line must be known to contain module timing information.
The function returns a EntryDataClass including the timing information, or
raises a FormatError if the line has no valid format.

Format 1 (20140226):

TimeModule> run: 1 subRun: 0 event: 10 beziertrackercc BezierTrackerModule 0.231838

Definition at line 416 of file SortModuleTimes.py.

def ParseTimeModuleLine(line):
        """Parses a line to extract module timing information.

        The line must be known to contain module timing information.
        The function returns a EntryDataClass including the timing information, or
        raises a FormatError if the line has no valid format.

        Format 1 (20140226):

        TimeModule> run: 1 subRun: 0 event: 10 beziertrackercc BezierTrackerModule 0.231838
        """
        Tokens = line.split()

        ModuleKey = None
        EventKey = None
        time = None

        # Format 1 parsing:
        try:
                EventKey = EventKeyClass((int(Tokens[2]), int(Tokens[4]), int(Tokens[6])))
                ModuleKey = ModuleKeyClass((Tokens[7], Tokens[8]))
                time=float(Tokens[9])
        except Exception, e:
                raise FormatError(
                  "TimeModule format not recognized: '%s' (%s)" % (line, str(e)),
                  type="Module", event=EventKey, module=ModuleKey
                  )
        # try ... except

        # validation of Format 1
        if (Tokens[0] != 'TimeModule>') \
          or (Tokens[1] != 'run:') \
          or (Tokens[3] != 'subRun:') \
          or (Tokens[5] != 'event:') \
          or (len(Tokens) != 10) \
          :
                raise FormatError \
                  ("TimeModule format not recognized: '%s'" % line, type="Module")
        # if

        return EntryDataClass(EventKey, module=ModuleKey, time=time)
# ParseTimeModuleLine()

def SortModuleTimes.RightString	(		s,
			w,
			f = ' '
	)

Returns the string s in a width w, padded by f on the left.

Definition at line 659 of file SortModuleTimes.py.

def RightString(s, w, f = ' '):
        """Returns the string s in a width w, padded by f on the left."""
        return f * max(0, w - len(s)) + s

def SortModuleTimes.signed_sqrt

(

value

)

Returns sign(x) * sqrt(abs(x))

Definition at line 38 of file SortModuleTimes.py.

def signed_sqrt(value):
        """Returns sign(x) * sqrt(abs(x))"""
        if value >= 0.: return math.sqrt(value)
        else: return -math.sqrt(-value)
# signed_sqrt()

Variable Documentation

string SortModuleTimes.__doc__ = "Prints statistics of the module timings based on the information from the Timing service."

private

Definition at line 33 of file SortModuleTimes.py.

SortModuleTimes.action

Definition at line 870 of file SortModuleTimes.py.

SortModuleTimes.AllStats = JobStatsClass( )

parse all inputs, collect the information

Definition at line 890 of file SortModuleTimes.py.

SortModuleTimes.bTrackEntries

Definition at line 893 of file SortModuleTimes.py.

SortModuleTimes.CheckDuplicates

Definition at line 883 of file SortModuleTimes.py.

SortModuleTimes.const

Definition at line 871 of file SortModuleTimes.py.

SortModuleTimes.default

Definition at line 874 of file SortModuleTimes.py.

SortModuleTimes.dest

Definition at line 870 of file SortModuleTimes.py.

SortModuleTimes.EventStats = TimeModuleStatsClass\

Definition at line 892 of file SortModuleTimes.py.

SortModuleTimes.help

Definition at line 867 of file SortModuleTimes.py.

SortModuleTimes.int

Definition at line 874 of file SortModuleTimes.py.

SortModuleTimes.metavar

Definition at line 866 of file SortModuleTimes.py.

SortModuleTimes.nargs

Definition at line 866 of file SortModuleTimes.py.

int SortModuleTimes.nErrors = 0

Definition at line 897 of file SortModuleTimes.py.

SortModuleTimes.options = Parser.parse_args()

Definition at line 880 of file SortModuleTimes.py.

SortModuleTimes.OutputTable = TabularAlignmentClass()

print the results

Definition at line 916 of file SortModuleTimes.py.

SortModuleTimes.Parser = argparse.ArgumentParser(description=__doc__)

parse command line arguments

Definition at line 862 of file SortModuleTimes.py.

SortModuleTimes.PresentMode

Definition at line 863 of file SortModuleTimes.py.

SortModuleTimes.type

Definition at line 874 of file SortModuleTimes.py.

string SortModuleTimes.Version = "%(prog)s 1.5"

Definition at line 32 of file SortModuleTimes.py.

SortModuleTimes.version

Definition at line 878 of file SortModuleTimes.py.