How To Run Phobos Data Acquisition System

Last Updated on Tuesday, August 19, 2014 at 15:51:31.

Table of contents:

 

Login

Login to the phobosx as user  phobos

Locate terminal windows:

(if windows are not exist then start it using following commands:
dtterm -name phatdaq -sl 10000 &
dtterm -name runcontrol -geometry 80x12 &)

Start DAQ system

Start Readout Controllers

Start event builder phatdaq in the phatdaq window:

Start run control macro in the runcontrol window:

After that command, the event builder will be waiting until all ROCs are connected. You should wait until it say 'Ready to start'

To enter commands you can use menu buttons (or type them directly in the runcontrol window)

Run procedure

Note. Due to known start/stop event mismatch problem, to take the normal run you need to exit all ReadOut Controllers, and restart all of them (including phatdaq) again.

If online monitoring is required, make sure that the online computer is ready to analyze data.

To start run:

Enter one of the following start run commands:

Use following command to see if events are being recorded.

rc_status()

You also should see increased CPU, Disk and Packet activity in the performance monitor.
The phatdaq will report periodically (each 10 MB) updating of the data file. When the data file approaches 2GB limit, it will be automatically closed and a new file with incremented sequence number will be open. Make sure that there is enough free space on the disk, otherwise the program will halt.

To stop run:

rc_stop_run()

If necessary, write out run comments to database using WWW browser at https://www.bnl.gov/phobos/Secure/ASP/run_logbook_comment.asp

Around the clock command sequence

Suppose you are currently taking the normal run.

  1. Stop the run:
    rc_stop_run()
  2. Wait 5 sec. Check for message "Closing everything" in phatdaq window. If there is no such message, try to stop again.
  3. Start pedestal run for 7000 events and after that the calibration run for 6000 events
    rc_start_all_runs()
    The pedestal run takes 7000/300=23 sec, the calibration run - 6000/13.5=8 min. You will have time to calculate pedestals the run will stop automatically.
  4. Use rc_status()command to check if calibration run is finished
  5. Exit all ROCs:
    rc_exit()
    answer 'y' on the first prompt
  6. Restart the Silicon ROC(in Silicon_roc window):
    <ESC>k
    sp ph_roc
  7. Restart the Plastic ROC(in Plastic_roc window):
    <ESC>k
    >sp(rocsfi,"192.168.14.1",7000,1,3,0,0)
    (always choose the largest number if asked)
  8. Restart the phatdaq (in 'phatdaq window)
    $PHATDAQ/phatdaq 2
  9. Init:
    rc_init()
  10. Start the new normal run:
    rc_start_run()
  11. Wait 1 hour and repeat from 1:
  12. Go to the Online Machine and start the Monitor Grabber (unless development is in progress)

    Calculation of the pedestals

    Find the 'pedcalc' window icon on the left side. In that window:

    1. If you don't have root prompt then you need to start phat:
      cd $PHATDAQ
      phat
    2. having the root prompt, enter:
      .! cp -p /data/#/PhoRaw000###s00#.root /data/7/peddata
      (here # is the disk number, ### is the run number for pedestal run, 00# is the file sequence number. You can check correct file name using .! ls /data/#)
    3. .x pedcalc.C(###)
      if you get an error message "Error: no symbol ...", then you are apparenly in the wrong directory, enter .q command to exit phat and continue from step 1
    4. use default values <RETURN>

    Calculation of the gains

    Find the 'pedcalc' window icon on the left side. In that window:

    1. If not in root then:
      cd $PHATDAQ
      phat
    2. having the root prompt, enter:
      .! cp -p /data/0/PhoRaw000###s00#.root /data/7/gaindata
      (here # is the disk number, ### is the run number for pedestal run, 00# is the file sequence number. You can check correct file name using .! ls /data/#)
    3. .x calcalc.C(#)
      if you get an error message "Error: no symbol ...", then you are apparenly in the wrong directory, enter .q command to exit phat and continue from step 1
    4. use default values <RETURN>

Pedestal Run

The pedestal run should be taken and calculated regularly. During the calculation phase the pedestal data will be automatically sent to the online computer for up-to-date pedestal setting

Make sure that phatdaq and roc_silicon are running.
In runcontrol window enter:

rc_start_pedestal_run()
(pedestal run with writing)

Run will be stopped automatically after 1000 events (5 sec). To check if it stopped - use rc_status() command

You can start taking calibration data now (see Calibration Run), during that time you can calculate pedestals running PHAT macro in the 'pedcalc' terminal window:

Calibration Run

The calibration run should be taken regularly. During the calculation phase the gain calibration data will be automatically sent to the online computer for up-to-date calibration setting

Make sure that phatdaq and roc_silicon are running.
In runcontrol window enter:

rc_start_calibration_run()
(calibration  run with writing)

Run will be stopped automatically after 6000 events (8 min). To check if it stopped - use rc_status() command.

Trigger connection

 

To move data files to HPSS:

from default directory:
/export/home/phobsink/bin/write_to_RCF /data/0
(
and from other disks if required)
/export/home/phobsink/bin/write_to_RCF /data/1
/export/home/phobsink/bin/write_to_RCF /data/2
/export/home/phobsink/bin/write_to_RCF /data/3
/export/home/phobsink/bin/write_to_RCF /data/4
/export/home/phobsink/bin/write_to_RCF /data/5
/export/home/phobsink/bin/write_to_RCF /data/6
/export/home/phobsink/bin/write_to_RCF /data/7

Write down to the logbook.

Useful Commands:

general commands:

Notes:

  1. If phatdaq was killed or crashed when run was in progress then one need to restart all ROCs and init phatdaq.
  2. Closing the runcontrol will exit phatdaq.
  3. Do not enter init command twice!

Troubleshooting

To contact DAQ experts:

If phatdaq crashed and file wasn't properly closed

In that case the next run might have the same run number and therefore the previous file may be overwritten. To be safe, you need to rename current file and make a short run just to increment global run number.
to rename the file:
>mv PhoRaw000###s00#.root PhoRaw000###s00#crashed.root

In case of problems with phatdaq

Try to stop the run first:
rc_stop_run()

Exit phatdaq but keep readout controllers running:

rc_exit()
Do you want to exit all ReadOut Controllers? [y/n] >
n
Do you want to exit phatdaq? [y/n]>
y

How to kill phatdaq

If phatdaq did not exit, try to <Control C> the phatdaq. If this does not work, kill it from other terminal window. To do that, locate a process ID for the /export/home/phobos/phatdaq/phatdaq, then kill it:
>ps -ef | grep phatdaq -
>kill ###
- where ### is the process ID

Restart the phatdaq again:
>$PHATDAQ/phatdaq 2

Restart, if necessary, runcontrol and init phatdaq:
.x runcontrol.C

If you get 'Ready to start' in phatdaq window - you are in business again, but for safety do following:
rc_start_run_without_writing()
rc_stop_run()

After succesful completion of above operation you can continue the regular work.

If you don't get 'Ready to start' in phatdaq window - then you need to restart or reboot ROC. When it is got up again, you will see this 'Ready to start'.

How to restart ReadOut Controllers

Exit all ReadOut Controllers using rc_exit() command and answer 'y' to first prompt.

Mercury VME crate

Locate Silicon_roc terminal window.

Software restart.

(Try it first, you may use <ESC>K to retrieve previous commands)
sp ph_roc
You should get "Waiting for Server command" message in Silicon_roc window.

Hardware restart

Press the 'RST' button MVME 2600-2 module (leftmost in the crate). The ph_roc will start automatically. The full reboot takes 3-5 minutes. After that time you should see a message in the phatdaq window saying that Silicon ROC was registered.

FASTBUS

Software restart
  1. Locate plastic_roc terminal window
  2. If rocsfi had crashed you will need to kill last rocsfi program. To do that, find task name using 'i' command. Look for rocsfi in the ENTRY column. If it exists then kill it using 'td' command, i.e:
    td t13
  3. restart rocsfi program (you can use <ESC>K command to retrieve previous command)
    sp (rocsfi,"192.168.14.1",7000,1,3,0,0)
    choose last version (highest number) if rocsfi overloaded
Hardware restart

The hardware reset is not recommended, contact Andrei first.
By the way turning on and off the FASTBUS power supply might help.The ReadOut Controller will start automatically after 2-3 minutes.

If you can't copy the pedestal/calibration file to /data/7

If you get the message 'error reading file' during the cp command, most likely case is that /data/7 is full.

If you don't get proper response from the runcontrol

How to run phatdaq without database connection

>$PHATDAQ/phatdaq 2 0

How to backup disks to DLT tape

The tape capacity is between 35GB (uncompressed) and 70GB(compressed). The full disk is about 15 GB.

To list all files on the tape:

 

How to change global run number

Normally you don't need to change it. If phatdaq encounter s that the file is already exist it will increment current run number. If after 10 attempts it still cannot open file then you, probably need to change global number.

cd $PHATDAQ
mv config/phatdaq_runlog.root config/phatdaq_runlog.root.save
phat
.x runlog_write.C(runnum)
The next time you start phatdaq the global run number will be runnum+1;