Re: raw data output

From: Christian Holm Christensen (cholm@hehi03.nbi.dk)
Date: Mon May 06 2002 - 10:30:10 EDT

  • Next message: Flemming Videbaek: "Re: raw data output"

    Hi Flemming et al, 
    
    On Mon, 6 May 2002 09:51:14 -0400
    "Flemming Videbaek" <videbaek@sgs1.hirg.bnl.gov> wrote
    concerning "Re: raw data output":
    > First as you noted this class was more of a place holder, though it
    > was used in very early days to generate some simulated data for
    > being able to digiitize data. 
    
    Which ofcourse had some validity. 
    
    > Kris, Christian and I have actual talked about removing this class
    > completely, since I cannot real imagine what you would use
    > 'generated raw data for' unless you would write some level 3 code to
    > put into the online program. 
    
    Erh, `level 3 code'?  I've never seen any real spell-out of levels, so
    here's how I see it:
    
      Level 0        Collision                     Event Generator 
                        |                              |
      Level 1        BRAHMS Experiment DAQ         BRAG cdat
                        |                              |
      Level 1a          |                          BRAT digitisation  
                         \                             /
      Level 2              `Reduction' steps (all data) 
                                        |
      Level 3                Physics Data (DST/PhD) 
                                        |
      Level 4               Analysis (ROOT interactive)
    
    > So what is it Pawel wants do do that cannot be done with the
    > 'digitzed' classes ? 
    
    
    If Pawel want's the DAQ files, he should sink 'em from HPSS.  I
    personally have no idea how to do that, execpt by making a hack that
    uses CAS to copy the files out.  A JSF like 
    
      executable=/brahms/u/stazsel/bin/cpraw 
      inputstreams=1
      inputdir[0]=/home/bramsink/raw/2001/10/12 
      inputfile[0]=run005300seq.dat 
      inputstreamtype[0]=HPSS
      outputstreams=1
      outputdir[0]=/brahms/u/stazsel/raw/2001/10/12 
      outputfile[0]=run005300seq.dat 
      outputstreamtype=UNIX 
      stderrdir=/brahms/u/stazsel/raw/log
      stderr=run005300seq.err
      stdoutdir=/brahms/u/stazsel/raw/log
      stdout=run005300seq.out
      
    and a script `/brahms/u/stazsel/bin/cpraw' like 
    
      #!/bin/sh 
    
      hpssfile=$INPUT0
      unixfile=$OUTPUT0
      
      if test -f $unixfile ; then 
         echo "File $unixfile already exists, giving up" 
         exit 1
      fi 
    
      /bin/cp -a $hpssfile $unixfile 
      if test $? -ne 0 ; then 
         echo "Failed to copy $hpssfile to $unixfile" 
         exit 2
      fi 
      exit 0
    
    One should then have a JSF for each sequence.  That's comparably easy
    to do: 
    
      #!/bin/sh 
    
      run=$1
    
      script=/brahms/u/staszel/bin/cpraw
      outdir=/brahms/u/staszel/raw
      email=staszel@nbi.sk
    
      mysqlout=`mktemp /tmp/jsfXXXXXX`
      mysql -u query -p -h pii3.brahms.bnl.gov RUNDB <<EOF > $mysqlout
      SELECT HpssFile FROM Files WHERE RunNo=$run ; 
      EOF 
    
      files=`tail -n "+2" $mysqlout` 
    
      rm $mysqlout 
    
      for f in $files ; do 
        dir=`dirname $f` 
        fin=`basename $f` 
        bas=`basename $f .dat`
        jsf=`${jsfdir}/${bas}.jsf`
        cat > $jsf <<EOF
      executable=${script}
      inputstreams=1
      inputdir[0]=/home/bramsink/raw${dir}
      inputfile[0]=${fin}
      inputstreamtype[0]=HPSS
      outputstreams=1
      outputdir[0]=${outdir}/raw/${dir}
      outputfile[0]=${fin}
      outputstreamtype=UNIX 
      stderrdir=${outdir}/raw/log
      stderr=${bas}.err
      stdoutdir=${outdir}/raw/log
      stdout=${bas}.out
      EOF   
    
        crs_submit.pl $jsf 
        sleep 1 
    
        rm -f $jsf 
      done 
    
    Execute this script from the bramreco account on rcruser[1,2]  to sink
    the data files out of HPSS into a user directory (should be sym link to
    one of the /brahms/dataX disks). 
    
    There is some utils to sink files out of HPSS  more directly, but I
    don't know how to use it.  I think Betty, Flemming, and perhaps Ian
    and Djamel knows, and maybe some one else.  I believe I once saw some
    documentation on the web, but I can't remember where.  Will the person
    that did that please stand up?
    
    Another way, is to create a bratmain configuration script that uses
    BrCopyModule to copy the full in event node to an the output.  This
    will store the data as `.root' files with a catalog of BrEvent
    objects.  This has the additional advantage, that you can ignore sync
    triggers, and similar, to store disk space.  Read it in with a normal
    BrEventIO module. 
    
    Yours, 
    
     ____ |  Christian Holm Christensen 
      |_| |	 -------------------------------------------------------------
        | |	 Address: Sankt Hansgade 23, 1. th.  Phone:  (+45) 35 35 96 91
         _|	          DK-2200 Copenhagen N       Cell:   (+45) 24 61 85 91
        _|	          Denmark                    Office: (+45) 353  25 305
     ____|	 Email:   cholm@nbi.dk               Web:    www.nbi.dk/~cholm
     | |
    



    This archive was generated by hypermail 2b30 : Mon May 06 2002 - 10:31:03 EDT