IMPORTANT!!! - Please read _carefully_ (was Re: tof calibration)

From: Christian Holm Christensen (cholm@hehi03.nbi.dk)
Date: Fri May 25 2001 - 06:08:42 EDT

  • Next message: Michael Murray: "Re: tof calibration"

    Hi Flemming, 
    
    Sorry about the length of this email, but I STRONGLY suggest EVERYONE
    reads this CAREFULLY, since it seems that quite a number of people has
    got this stuff wrong, so I once again explain this in detail. 
    
    I'll, at some later point, but the essence of this into "The BRAT
    Guide", avaliable from 
    
      http://pii3.brahms.bnl.gov/~brahmlib/brat/guide/ 
    
    (recently updated). 
    
    On Thu, 24 May 2001 20:38:17 -0400
    "Flemming Videbaek" <videbaek@sgs1.hirg.bnl.gov> wrote
    concerning ": Re: tof calibration":
    > Dear Djamel, (and others)
    > 
    > Where is this package intended to be submitted - into the tof library?
    > or in an application package. This is unclear to me, though some sounds like
    > tof stuff
    
    The class BrTofPackage (which is what Djam calls BrTofModule) can go
    into either a tof directory or an applications directroy. 
    
    > and others as if belonging to somewhere else. Maybe the 'new' structure
    > delas better with this.
    
    In BRAT2, I guess that packages should all go into the "offline/applications"
    directory. 
      
    > Why is the BrTofCalibration not called something with Parameter?
    > BrTofCalibrationParameters
    > or whatver fit into CH scheme
    
    No no.  BrTofCalibration is exactly what I wrote in the "The BRAT
    Guide", which I by the way updated a bit.  Don't change that Djam. 
     
    > > One last thought: while I was updating all that, I realized how important
    > > it was to converge towards some standard and automatic procedure. So, to
    > > second Christian's voice, I strongly recommand we use the main module
    > thing.
    > 
    > One this question there is a realy issue how to enhance to capabilties of
    > the MainModule.
    > It is not all kind of analysis that fits into this framework in
    > Event(rawdata, outputdata)
    > in particular the tracking needs more sophisticated analysis, and I think
    > PID as well when we come to
    > that. 
    
    Sorry, but no it doesn't.  See below why not. 
    
    > Essentially I think once has to be able to specify (in pesudo code)
    > something like
    > 
    > container1.Add(module1)
    > container1.Add(module2)
    > container.SetInputNode(rawevent);
    > container.SetOutputNode(reduced1)
    > 
    > cointainer2.SetInputNode(reduced1)
    > container2.SetOutputNode(rdonode)
    > container2(addmodule3)
    > container2(addmodule4)
    > 
    > outputmodule.SetInputNode(rdonode)
    
    So what's the difference to (in pseudo code): 
    
      
      container1->Add(module1) 
      container1->Add(module2) 
      
      container2->Add(module3) 
      container2->Add(module4) 
    
      container3.Add(container1) 
      container3.Add(container2) 
      container3.Add(module5) 
      container3.Add(package1) 
    
      container3.Init(); 
    
      while (moreRuns) { 
        container3.Begin(); 
        while (moreEvents) { 
          container3.Event(inNode, outNode); 
        }
        container3.End(); 
      }
      container3.Finish(); 
    
    This is how BrMainModule works (container3). 
    
    If your objection goes like "yes, but not all modules should get the
    top-level nodes", then please read on. 
    
    > And then having a module do the work
    > 
    > Any ideas, comments?
    
    No no, that's not what you do.  See the point of _all_ modules to read
    from the passed BrEventNode objects, is so that one does not need
    complicated Event loops, like explicitly setting input and so on (what
    was done in the tracking code and is the main problem to intergrate
    that code into a general form - thank (a) god Peter is working on the
    remedy - not an example to be followed).  In short: 
    
        Modules only way of getting event data, communicating results, and
        selecting events, is via the two passed pointers to BrEventNode
        objects in the Event method! 
    
    (this is all in "The BRAT Guide").  
    
    Sometimes, one may want some specific modules to only have access to a
    specific sub event node, or write it's data to a specific event node.
    The solution to that, is to make a package: 
    
      class BrFooPackage : public BrModuleContainer {
      private: 
        BrBarModule*  fBarModule; 
        BrBazModule*  fBazModule; 
      public: 
        BrFooPackage(const char* name, const char* title) 
          : BrModuleContainer(name, title) { 
          fBarModule = new BrBarModule("BAR", "Bar Module"); 
          AddModule(fBarModule); 
          fBazModule = new BrBazModule("BAZ", "Baz Modsule"); 
          AddModule(fBazModule); 
        }
    
        void Event(BrEventNode* inNode, BrEventNode* outNode) { 
          SetStatus(kEvent); 
    
          // Get a specific event node: 
          BrEventNode* ourInNode = inNode->GetEventNode("OurNode"); 
          if (!outInNode) { 
            Stop("Event", "Node 'OurNode' not found");  
            return;
          }
    
          // Make a specific output node: 
          BrEventNode* ourOutNode =  
            new BrEventNode("FooNode", "FOO output node"); 
          outNode->AddEventNode(outOutNode); 
    
          // let the sub-modules do thier job 
          BrModuleContaier::Event(ourInNode, ourOutNode); 
        }
       
        BrBarModule* GetBarModule() const { return fBarModule; }
        BrBazModule* GetBazModule() const { return fBazModule; }
    
        ClassDef(BrFooPackage, 0) // Package for FOO analysis
      };
    
    Similar a pacakge  may overload the Book method to put all it's
    sub-modules histograms into a specific directory: 
    
      class BrFooPackage : public BrModuleContainer {
        ...
        Book()  {
          // Make a directory for all modules hisotgrams
          TDirectory* saveDir = gDirectory; 
          TDirectory* ourDir  = saveDir->mkdir("ourDir");
          ourDir->cd();
      
          BrModuleContainer::Book();
      
          gDirectory = saveDir;
        }
        ...
      };
    
    
    There's only one subtlety here: Some modules may like to get data that
    is the result of some other module before that in the module pipeline.
    That would mean that the module would need the output node of the
    previous module should be the input node of the next module, which is
    not possible directly.  What one can do then, is to make a really
    simple wrapper module: 
    
    
      class SwitchModule : public BrModule 
      private:  
        BrModule* fModule; 
      public: 
        SwitchModule(const char* name, cosnt char* title) 
          : BrModule(name,title) { fModule = 0; }
    
        void      SetModule(BrModule* module) { fModule = module; } 
        BrModule* GetModule(void) const { return fModule; }
    
        void Begin() { 
          SetState(kBegin);
          if (!fModule)  
            return;
          fModule->Begin(); 
          SetStatus(fModule->GetStatus());
        }
    
        void DefineHistograms() { 
          SetState(kDefineHistograms);
          if (!fModule)  
            return;
          fModule->DefineHistograms(); 
          SetStatus(fModule->GetStatus());
        }
    
        void End() { 
          SetState(kEnd);
          if (!fModule)  
            return;
          fModule->End(); 
          SetStatus(fModule->GetStatus());
        }
    
        void Event(BrEventNode* inNode, BrEventNode* outNode) { 
          SetState(kEvent);
          if (!fModule) 
            return; 
          fModule->Event(outNode, outNode);  // The main thing here !!!!
          SetStatus(fModule->GetStatus());
        }
    
        void Finish() { 
          SetState(kFinish);
          if (!fModule)  
            return;
          fModule->Finish(); 
          SetStatus(fModule->GetStatus());
        }
    
        void Init() { 
          SetState(kInit);
          if (!fModule)  
            return;
          fModule->Init(); 
          SetStatus(fModule->GetStatus());
        }
    
        void Print(Option_t option="B") const { 
          BrModule::Print(option);
          if (!fModule)  
            return;
          fModule->Print(option); 
        }
    
        ClassDef(SwitchModule, 0) // switch output to input
      };
    
    and this can be used in the configuration script as: 
    
      //__________________________________________________________________
      // Module: BrFooModule
      BrFooModule* fooModule = 
        new BrFooModule("FOO", "A Foo module");
      SwitchModule* switchFooModule = 
        new SwitchModule("switchFoo", Switch Foo"); 
      switchFooModule->SetModule(fooModule); 
      mainModule->AddModule(switchModule);
    
    Notice that we _don't_ add the fooModule directly to the
    BrMainModule; rather we add the switch wrapper.  In this way, one
    still has all the flexibility of the module-pipeline (you can insert
    filters, checkers, and so on, where ever you like).  Ofcourse, a
    package may use such the SwitchModule to tee the passed BrEventNode
    for any module in the package 
    
      class BrFooPackage : public BrModuleContainer {
      private: 
        BrBarModule*  fBarModule; 
        BrBazModule*  fBazModule; 
        SwitchModule* fBazSwitchModule; 
      public: 
        BrFooPackage(const char* name, const char* title) 
          : BrModuleContainer(name, title) { 
          fBarModule = new BrBarModule("BAR", "Bar Module"); 
          AddModule(fBarModule); 
          fBazModule = new BrBazModule("BAZ", "Baz Modsule"); 
          fBazSwitchModule = new SwitchModule("BAZSwitch", "BAZ Switcher"); 
          fBazSwitchModule->SetModule(fBarModule); 
          AddModule(fBazSwitchModule); 
        }
    
        BrBarModule* GetBarModule() const { return fBarModule; }
        BrBazModule* GetBazModule() const { 
          return (BrBarModule*)fBarSwitchModule->GetModule(); }
    
        ClassDef(BrFooPackage, 0) // Package for FOO analysis
      };
    
    Let me outline the idea of the TestMainModule and BrMainModule
    (again!) See also 
    
      http://www.rhic.bnl.gov/brahms/WWW/private/list_hyper/brahms-dev-l/0522.html
    
    The idea is, that you make a configuration script that _only_ defines
    what you want in your job.  This is done by making modules and adding
    them to the singleton instance BrMainModule.  
    
    This configuration script is read (interpreted) by TestMainModule.
    Since BrMainModule is a singleton, this means that the instance one
    created in the configuration script, is the same (including all added
    modules) in both the configuration scripts _and_ in the compiled
    program TestMainModule. 
    
    The next thing that TestMainModule does, is to initialise all the
    modules contained in the BrMainModule instance. Then it loops over the
    number of defines run-levels (see "The BRAT Guide"), executing the
    Begin method of each module in the BrMainModule instance at the start
    of a run level, and the End method of each module in the BrMainModule
    instance at the end of a run level. For each run level, it loops over
    the the number of events, executing the Event method of each module in
    the BrMainModule instance.  At the very end of the job, the Finish
    method of each module in the BrMainModule instance is executed. 
    
    This is all that TestMainModule does (more or less), which means that
    anything that should be done at Init, Begin, Event, End, or Finish
    time _must_ be in a module, and that module _must_ respect the design
    ideas of BRAT (See above, previous mail to Djam, as well as "The BRAT
    Guide").  
    
    This design has a very VERY useful feature: Suppose you define a
    module to do some analysis, what ever that may be (dN/deta, p/pbar
    ratio, fluctuations, d2N/dydmt, and so on).  Then you simple compile
    that module into a shared library
    
      rootcint -f MyDict.cxx -c  `brat-config --cflags` MyModule.h 
      g++ `brat-config --cflags` -c -g -Werror -02 -o MyModule.o MyModule.cxx
      g++ `brat-config --cflags` -c -g -Werror -02 -o MyDict.o MyDict.cxx
      g++ -shared -Wl,-soname,MyModule.so -o MyModule.so MyModule.o MyDict.o
    
    and then you can load it directly into ROOT (i.e., your job) like 
    
      gSystem->Load("MyModule.so"); 
    
    Since the configuration script is interpreted, it means that the class
    MyModule is now known to CINT (the interpretor), so you can instantise
    it directly with out including any headers and so on: 
    
      MyModule* myModule = new MyModule("myAna", "My module for Ana"); 
    
    To have the module executed in the pipeline, you ofcourse need to add
    it the BrMainModule instance: 
    
      myModule->AddModule(myModule); 
    
    The point of this is that users may define private modules for thier
    specific task, and use them directly, without adding them to BRAT, or
    recompiling TestMainModule against the user library and so on.  Also,
    since the configuration scripts are exactly that - scripts - it's very
    easy to make corrections, since no recompilation is ever needed.   
    
    On Fri, 25 May 2001 10:29:30 +0200 (CEST)
    Djamel Ouerdane <ouerdane@nbi.dk> wrote
    concerning ": Re: tof calibration":
    > All the stuff I described yesterday will be part of the tof library
    > unless someone has some objections against it. The only thing that doesn't
    > belong to brat is the file TofCalibConfig.C which will be in my
    > brahms_app.
    
    For BRAT1, I don't give a tos.  
     
    > >
    > > Why is the BrTofCalibration not called something with Parameter?
    > > BrTofCalibrationParameters
    > > or whatver fit into CH scheme
    > 
    > I can change the name. What about BrTofCalParameter
    
    NO! See above and "The BRAT Guide" 
    
    > Concerning the main module, I agree that it's not the definitive
    > shape of it and needs some more sophisticated stuff. I was just
    > thinking that dealing with standard things makes it easier for
    > people to find their way. 
    
    Bollocks! TestMainModule is _essentially_ done.  Nothing more
    "sophisticated stuff" is needed.  Refer to what I said above. 
    
    Yours, 
    
    Christian  -----------------------------------------------------------
    Holm Christensen                             Phone:  (+45) 35 35 96 91 
      Sankt Hansgade 23, 1. th.                  Office: (+45) 353  25 305 
      DK-2200 Copenhagen N                       Web:    www.nbi.dk/~cholm    
      Denmark                                    Email:       cholm@nbi.dk
    



    This archive was generated by hypermail 2b29 : Fri May 25 2001 - 06:09:58 EDT