MadMongers

I want to plot time series data (that is very long as it would not fit on one screen but would take many screens) horizontally by time. Events need to be annotated on the time series (with time going horizontally) and these events are to 2 decimal places starttime and endtime. There need to be 30 second sections across the whole time series that need to be color coded (from the data). The data is data from files not functions or other generated stuff.  Multiple events can overlap in time. So there would need to be the ability to have several veritcal "tracks" available. 

So I have possibly 2 problems making a giant graph and then needing a viewing program to show only only one section of a giant graph at one time _or_ a program that interactively generates only one section of the image at a time. 

Ideally times would be displayed in hh24:mi:ss.ff format (using oracle speak) when in fact they would be date-times.  or they could be unixtimes but it'd have to be a variation that allowed a decimal part, or unixtime in milliseconds, unixtime in seconds and a millisecond offset. 

Anyone know a perl plotting library that can do it? I've spent a number of hours on this already and not found anything although you can find plenty of people unhappy with gnuplot in particular for this very reason. I don't see any evidence that gnuplot gave in on the time granularity. 

I believe that python's matplotlib can do it but I don't much care for python. 

I hope I'm not getting pdl as the answer because I'm not getting anywhere with it. Can't even run a demo and installing various libraries associated with pdl have file conflicts. (Same name and location file in different libraries.) (yes this is activestate v5.14.2 and please spare me the lecture on how everything would be wonderful if it wasn't windows etc.) OK I didn't really put myself out to try to get pdl graphics to work.  The pdl docs think pdl will ask for the graphics device interactively when it fact it does not do it in my experience. They were so sure it would ask they don't make it clear what you would tell it if it didn't ask. Yeah I could work more on that but it's not clear any of the graphics options in pdl, which do not all peacefully coexist filewise, do subsecond or millisecond times. If I knew they did I'd be more excited about battling the blank graphics window problem. 

Hmm. I see enthought python is not free (except to educational email addresses).  Nonetheless I will try it. but I'd be happier with perl. 

rocky wrote:

I want to plot time series data (that is very long as it would not fit on one screen but would take many screens) horizontally by time. Events need to be annotated on the time series (with time going horizontally) and these events are to 2 decimal places starttime and endtime. There need to be 30 second sections across the whole time series that need to be color coded (from the data). The data is data from files not functions or other generated stuff.  Multiple events can overlap in time. So there would need to be the ability to have several veritcal "tracks" available. 

So I have possibly 2 problems making a giant graph and then needing a viewing program to show only only one section of a giant graph at one time _or_ a program that interactively generates only one section of the image at a time. 

Ideally times would be displayed in hh24:mi:ss.ff format (using oracle speak) when in fact they would be date-times.  or they could be unixtimes but it'd have to be a variation that allowed a decimal part, or unixtime in milliseconds, unixtime in seconds and a millisecond offset. 

Anyone know a perl plotting library that can do it? I've spent a number of hours on this already and not found anything although you can find plenty of people unhappy with gnuplot in particular for this very reason. I don't see any evidence that gnuplot gave in on the time granularity. 

I believe that python's matplotlib can do it but I don't much care for python. 

I hope I'm not getting pdl as the answer because I'm not getting anywhere with it. Can't even run a demo and installing various libraries associated with pdl have file conflicts. (Same name and location file in different libraries.) (yes this is activestate v5.14.2 and please spare me the lecture on how everything would be wonderful if it wasn't windows etc.) OK I didn't really put myself out to try to get pdl graphics to work.  The pdl docs think pdl will ask for the graphics device interactively when it fact it does not do it in my experience. They were so sure it would ask they don't make it clear what you would tell it if it didn't ask. Yeah I could work more on that but it's not clear any of the graphics options in pdl, which do not all peacefully coexist filewise, do subsecond or millisecond times. If I knew they did I'd be more excited about battling the blank graphics window problem. 

Hmm. I see enthought python is not free (except to educational email addresses).  Nonetheless I will try it. but I'd be happier with perl. 

View Online

Madison Area Perl Mongers - MadMongers
http://www.madmongers.org

rocky wrote:

I want to plot time series data (that is very long as it would not fit on one screen but would take many screens) horizontally by time. Events need to be annotated on the time series (with time going horizontally) and these events are to 2 decimal places starttime and endtime. There need to be 30 second sections across the whole time series that need to be color coded (from the data). The data is data from files not functions or other generated stuff.  Multiple events can overlap in time. So there would need to be the ability to have several veritcal "tracks" available. 

So I have possibly 2 problems making a giant graph and then needing a viewing program to show only only one section of a giant graph at one time _or_ a program that interactively generates only one section of the image at a time. 

Ideally times would be displayed in hh24:mi:ss.ff format (using oracle speak) when in fact they would be date-times.  or they could be unixtimes but it'd have to be a variation that allowed a decimal part, or unixtime in milliseconds, unixtime in seconds and a millisecond offset. 

Anyone know a perl plotting library that can do it? I've spent a number of hours on this already and not found anything although you can find plenty of people unhappy with gnuplot in particular for this very reason. I don't see any evidence that gnuplot gave in on the time granularity. 

I believe that python's matplotlib can do it but I don't much care for python. 

I hope I'm not getting pdl as the answer because I'm not getting anywhere with it. Can't even run a demo and installing various libraries associated with pdl have file conflicts. (Same name and location file in different libraries.) (yes this is activestate v5.14.2 and please spare me the lecture on how everything would be wonderful if it wasn't windows etc.) OK I didn't really put myself out to try to get pdl graphics to work.  The pdl docs think pdl will ask for the graphics device interactively when it fact it does not do it in my experience. They were so sure it would ask they don't make it clear what you would tell it if it didn't ask. Yeah I could work more on that but it's not clear any of the graphics options in pdl, which do not all peacefully coexist filewise, do subsecond or millisecond times. If I knew they did I'd be more excited about battling the blank graphics window problem. 

Hmm. I see enthought python is not free (except to educational email addresses).  Nonetheless I will try it. but I'd be happier with perl. 

View Online

Madison Area Perl Mongers - MadMongers
http://www.madmongers.org

Thanks for the info. I like javascript, since it looks a whole lot like perl, but javascript doesn't read or write files, does it? (Which is why I want server side or "client server" javascript.) Or maybe they have a server side javascript that does i/o?

rocky wrote:

Thanks for the info. I like javascript, since it looks a whole lot like perl, but javascript doesn't read or write files, does it? (Which is why I want server side or "client server" javascript.) Or maybe they have a server side javascript that does i/o?

View Online

Madison Area Perl Mongers - MadMongers
http://www.madmongers.org

Server side implies it is not running on the client in a browser meaning it can do client server type things. Like reading and writing files and operating the bar code scanner , calling excel... Whatever programs can do that are not running in a sandbox.

I want to read the data out of files in many cases but writing files is also a thing I like to do.

I have been evaluating scipy for this. Thus far I have these impressions about it. 1

. python and matplotlib, numpy et al have a prodigious number of types. This turns out to be the big annoyance with it in my book. Whatever you are trying to do, you can be pretty certain you have the wrong type in the loop somewhere (It's like C++ PLUS more types!) "expecting a string not an array of characters..."and so on type of thing. Despite all the other things I thought I'd hate about python,  the vast number of types is the thing that I most hate in practice. (hint: if perl could do the same stuff with the same accuracy without all the type issues that would be very competitive.)

2. scipy in general likes to keep things divided up. For example if you subtract two datetimes in numpy you get a "timedelta" object which has days and seconds and microseconds stored separately in it. So if you want then together you have to do that math yourself. I'm not saying it's wrong. I'm just pointing out a design decision they seemed to make across the board to 'parse once and leave it that way' type of thing. Perhaps their thinking was it's a lot easier to add things together if you want them added than to parse them out into time units. Something to get used to however.

3. It's not all that easy to find out how to do things in scipy. There are not a lot of books etc. Perl could beat them at this game except however that there are entities that seem to be supporting python so much , like enthought.

4. there are interactive shells like pylab that are useful. You can generally copy and paste to pylab and copy and paste from pylab. You can see the results of operations and query objects that have been created. It IS useful. Perhaps perl would benefit from taking a cue from pylab and creating something like it. It works and it works on windows even.

5. you can do some complex things in scipy in few lines of code. But figuring out how to do it is not easy.

example of pylab:

import numpy as np
import matplotlib.pyplot as plt
import datetime
import matplotlib.mlab as mlab
from matplotlib import dates

#define some types other choices are float, int etc

dt = [('event_time',str),('event_code',str), ('event_end',str), ('stage',str)]

#read tab delimited file with a header line, only columns 2,3,5,6 (from 0)

events=np.recfromcsv('events.dat', dtype="dt", delimiter='\t', usecols=(2,3,5,6))

events.dtype
dtype([('event_time', '|S22'), ('event_code', '|S2'), ('event_end', '|S22'), ('stage', '<i4')])

#query a field from the array:

events.event_code
chararray(['L', 'H', 'D', ..., 'D', 'LA', 'AL'],
      dtype='|S2')

len(events.event_time)
1370

 #first one:

events.event_time[0]
 '04/14/2010_21:30:32.99'
#last one:
 events.event_time[1369]
 '04/15/2010_04:17:04.27'

They do have some good ideas there that perl could perhaps benefit from stealing er emulating.