Last Updated on Tuesday, August 19, 2014 at 15:43:44. by Author: George Stephans

PCAL General Information/Help

(Expert)

Contacts (in priority order) (until Feb 1)

• Check shift schedule to see if George might be on site.
  • Try his cell phone or call housing to get his dorm number.
• (normal working hours) George Stephans @ office
• George Stephans cell phone
• (before midnight Boston time) George Stephans @ home (781)-641-2053
• (before 1 am California time) Corey Reed cell phone
• (otherwise, sigh...) George Stephans @ home (781)-641-2053
• (If no other option available) Gerrit van Nieuwenhuizen @ office or cell

Running Instructions

• What do I do when...
  • There is a VME latchup. Note that the generic term "VME Latchup" now refers to any one of the following error conditions:
    1. Persistent red channel in PCAL event display.
    2. Error messages in VME ROC window about readout failure.
    3. PCAL on-line monitor alarms, complains about missing data.
  • The PCAL displays are all purple in TriggerDisplay or PCALDisplay  and/or the terminal windows for those processes are complaining about a missing Calib set.
  • There is stable beam and I want to take data.
  • RHIC is about to dump.
  • RHIC is about to inject.
  • There will be no beam for more than 2 hours (taking cosmics).
  • PCALMon1 is alarming
• Some other condition not covered above

Reference

• CheckList Instructions 
• How to turn on/off the PCAL
• How to turn on the PCAL HV monitoring mode
• Monitoring mode description
• Starting the PCAL Event Display
• Don't see cosmics in PCAL event display
• How to contact author

Running Instructions

What do I do when...

There is stable beam and I want to take data.

1. Make sure the DAQ is not already running with writing. If it is, stop the run.
2. Load the HV file named "BM011905.DAT".
   1. If you do not have a menu on the screen, push escape. (This will get you out of monitor mode.)
   2. You do NOT need to turn HV off.
   3. Push 2 and [Enter] ("Read and apply settings from file")
   4. Type "BM011905.DAT". This is the name of the HV file you want for normal beam data taking.
   5. If HV was off, turn it on by hitting 1 and [Enter].
3. Put the PCAL HV program in monitor mode.
   1. Push 7 and [Enter] ("Monitor mode")
   2. Accept the default value (90 Volts) for the "Allowed tolerance (returned - desired)", by hitting [ENTER]
   3. Accept the default value (90 volts) for the "Allowed tolerance (returned_old - returned_new)", by hitting [ENTER]
   4. Enter 300 (i.e. 5 minutes between reading voltages)
   5. In the lower left hand corner, the computer will say "Reading voltages..." Wait for this message to go away.
   6. Find the line that looks like

      stn    drv    chn    volt0    volt1    volt2    new-old   Test #     1

   7. Make sure there are no numbers in the columns below this line. If there are, call an expert. Some tubes are not powered up properly.
4. Make sure other detectors are in the correct state and start a DAQ run.
5. Keep an eye on the PCAL event display. If you see channels in the right histogram (which displays the ADC of each channel, averaged over events) that are bright red, and stay that way, call an expert.

RHIC is about to dump.

1. Make sure the DAQ is not running with writing. If it is, stop the run.
2. Load the HV file named "UNSTBEAM.DAT".
   1. If you do not have a menu on the screen, push escape. (This will get you out of monitor mode.)
   2. Push 2 and [Enter] ("Read and apply settings from file")
   3. Type "UNSTBEAM.DAT". This is the name of the HV file you want when there may be radiation spikes.
   4. If HV was off, turn it on by hitting 1 and [Enter].
3. Put the PCAL HV program in monitor mode.
   1. Push 7 and [Enter] ("Monitor mode")
   2. Accept the default value(90 volts) for the "Allowed tolerance (returned - desired)"
   3. Accept the default value(90 volts) for the "Allowed tolerance (returned_old - returned_new)"
   4. Enter 300 (i.e. 5 minutes between reading voltages)
   5. In the lower left hand corner, the computer will say "Reading voltages..." Wait for this message to go away.
   6. Find the line that looks like

      stn    drv    chn    volt0    volt1    volt2    new-old   Test #     1

   7. Make sure there are no numbers in the columns below this line. If there are, call an expert. Some tubes are not powered up properly.

RHIC is about to inject.

1. Make sure the DAQ is not running with writing. If it is, stop the run.
2. Load the HV file named "UNSTBEAM.DAT".
   1. If you do not have a menu on the screen, push escape. (This will get you out of monitor mode.)
   2. Push 2 and [Enter] ("Read and apply settings from file")
   3. Type "UNSTBEAM.DAT". This is the name of the HV file you want when there may be radiation spikes.
   4. If HV was off, turn it on by hitting 1 and [Enter].
3. Put the PCAL HV program in monitor mode.
   1. Push 7 and [Enter] ("Monitor mode")
   2. Accept the default value(90 volts) for the "Allowed tolerance (returned - desired)"
   3. Accept the default value(90 volts) for the "Allowed tolerance (returned_old - returned_new)"
   4. Enter 300 (i.e. 5 minutes between reading voltages)
   5. In the lower left hand corner, the computer will say "Reading voltages..." Wait for this message to go away.
   6. Find the line that looks like

      stn    drv    chn    volt0    volt1    volt2    new-old   Test #     1

   7. Make sure there are no numbers in the columns below this line. If there are, call an expert. Some tubes are not powered up properly.

There will be no beam for more than 2 hours (taking cosmics)

1. Make sure the DAQ is not running with writing. If it is, stop the run.
2. Load the HV file named "CS011905.dat".
   1. If you do not have a menu on the screen, push escape. (This will get you out of monitor mode.)
   2. Push 2 and [Enter] ("Read and apply settings from file")
   3. Type "CS011905.dat". This is the name of the HV file you want for running cosmics.
   4. If HV was off, turn it on by hitting 1 and [Enter].
3. Put the PCAL HV program in monitor mode.
   1. Push 7 and [Enter] ("Monitor mode")
   2. Accept the default value(90 volts) for the "Allowed tolerance (returned - desired)"
   3. Accept the default value(90 volts) for the "Allowed tolerance (returned_old - returned_new)"
   4. Enter 300 (i.e. 5 minutes between reading voltages)
   5. In the lower left hand corner, the computer will say "Reading voltages..." Wait for this message to go away.
   6. Find the line that looks like

      stn    drv    chn    volt0    volt1    volt2    new-old   Test #     1

   7. Make sure there are no numbers in the columns below this line. If there are, call an expert. Some tubes are not powered up properly.
4. Make sure other detectors are in the correct state and start a DAQ run with run type SICOSM plus writing. The trigger configuration does not matter for this run type.
5. In the run logbook, make a note of which HV file the PCAL is running with.
6. Keep an eye on the PCAL event display. If you see channels in the right histogram (which displays the ADC of each channel, averaged over events) that are bright red, and stay that way, call an expert.
7. Keep a close eye on the beam prospects. Please do not forget to follow the instructions for "RHIC is about to inject" before RHIC puts any beam into the ring.

PCALMon1 is alarming

1. Look at the "Online Monitoring 3" computer.
2. Find the window named "PCALMon1". This is the PCAL online monitoring client.
3. Find the event counter in the status bar. Check if it is increasing (it may be increasing slowly, so give it a little time).
4. IF (a) the event number is increasing AND (b) the client has a big red bar (i.e. it is alarming on event after event), the VME ADCs may need to be reset. Follow the instructions for what to do when "There is a VME latchup". If this does not fix the problem, call an expert.
5. IF (a) the event number is increasing AND (b) the client has a big green bar (i.e. the client has stopped alarming after an event or two), send an email to the experts. If this situation happens more than once during your shift, call an expert.
6. IF the event number is NOT increasing, the monitor has crashed. Click the "restart online" button to restart it. If the monitor continues to alarm after restarting, follow the instructions for what to do when "There is a VME latchup". If this does not fix the problem, call an expert.

There is a VME latchup

Important notes:

• You may need to reset the VME more than once to fix a problem.
• You may need to kill and restart the PCAL event display, the on-line monitor, or both to fix a problem.
• (Rarely) you may need to stop and restart the DAQ.

One way to detect a VME Latchup is via the PCAL event display showing some channels as red.  This can happen in the "average" histogram in the case of a latchup and also in the case of an unusually large signal or some other atypical event.  BEFORE resetting the VME crate, check that the red channels in the average are also consistently red (or perhaps orange) in the event-by-event display.  You can also check again when the average resets itself after 200 events.  If there are channels which stay consistently red in the event-by-event display and in the average display after it resets, then there is most likely a latchup.

Go to the DAQ instructions for the latest info on resetting the VME crate.

If the PCAL Online monitor, the PCAL event display, the VME ROC window, or something else appears to indicate that the VME is latched up but you don't see the corresponding indicator on any of the ADC's, try the Reset latchup instruction first, then please contact a PCAL or SpecCal expert.

Some other condition not covered above.

Ensure that the PCAL is ON, unless otherwise instructed by PCAL expert, with HV file name "UNSTBEAM.DAT".

The PCAL displays are all purple in TriggerDisplay or PCALDisplay and/or the terminal windows for those processes are complaining about a missing Calib set.

The PCAL software has trouble with run numbers, especially the 0 when not writing to disk.  We're looking into fixing this. If you are not taking real cosmic or ALLBEAM data (and writing to disk so you have a real run number), don't worry about this. If you ARE taking real cosmic or ALLBEAM data, stop and restart both the TriggerDisplay and the PCALOnline display.  If the problem persists, contact the PCAL expert.

Reference

CheckList Instructions

Go to the "PCAL HV Control" computer.

1. Look in the top left hand corner of the screen. If you see "High Voltage: ON", go to step 2. If you see "High Voltage: OFF", go to step 8.
2. If you do not see a menu, go to step 3. If you do see a menu, do the following to start monitor mode.
   1. Push 7 and [Enter] ("Monitor mode")
   2. Accept the default value(90 volts) for the "Allowed tolerance (returned - desired)"
   3. Accept the default value(90 volts) for the "Allowed tolerance (returned_old - returned_new)
   4. Enter 300 (i.e. 5 minutes between reading voltages)
   5. In the lower left hand corner, the computer will say "Taking data..." Wait for this message to go away.
3. Look at the top of the screen and find the line that looks like

   High Voltage: ON        Input File: UNSTBEAM.DAT            Sound: ON

4. Make sure high voltage is "ON". If not, call an expert.
5. Make sure the input file is correct:
   1. If RHIC is about to end the store, make sure the input file is "UNSTBEAM.DAT". If not, follow the instructions above for "What do I do when RHIC is about to dump?"
   2. If you are taking normal beam data, make sure input file is "BM011905.DAT". If not, follow the instructions above for "What do I do when there is stable beam and I want to take data?"
   3. If you are not taking data, make sure input file is "UNSTBEAM.DAT". If not, follow the instructions above for "What do I do when RHIC is about to inject?"
   4. If you are taking PCAL cosmic data, make sure input file is "CS011905.dat". If not, call an expert.
6. Look in the middle of the screen and find the line that looks like

   stn    drv    chn    volt0    volt1    volt2    new-old   Test #     1

7. Make sure there are no numbers in the columns below this line. If there are, call an expert. Some tubes are not powered up properly.
8. Leave the program in this state.

Go to the "Online Monitoring #1" computer.

1. Find the PCAL event display. It is the window titled "PCAL Online." If you can not find the event display, start it.
2. Make sure the event number in the title of the left histogram increasing (possibly slowly). If it is not, restart the PCAL event display.
3. Look at the right histogram.
4. The title of the right histogram should be "PCAL Averaged #/#." The first number is the number of events which have been averaged so far. The second number is the number of events that will be averaged before the histogram resets.
5. Wait until at least 100 events have been averaged (i.e. the first number is 100 or greater).
6. Make sure there are no red squares. If there are, call an expert. Some channels are "latched up."
7. Again wait until at least 100 events have been averaged.
8. Count the number of purple squares. You should find 7. There are 6 in row 11 and 1 in row 10. These are unconnected channels.
9. If you counted more than 7 purple squares AND you ARE taking data, either cosmic or beam, call an expert. Some ADC modules may have tripped off or some PMTs may have died.
10. If you counted more than 7 purple squares AND you are NOT taking data, relax. You are seeing pedestals.

How to turn on/off the PCAL

ON

1. Go to "PCAL HV Control" computer
2. At the prompt, type:

   C:\TEMP\NAMO>nanom UNSTBEAM.DAT
   

3. The screen should quickly flash message "Sending data" in the lower right corner. If it doesn't and you hear beeping, call an expert.
4. You should be presented with a menu, with number of options: Type at prompt "Choose a digit",  
       1 [ENTER]
5. At the top left hand part of screen you should see
       "High Voltage: ON"
   if you don't call PCAL expert. If you do, HV is on.

OFF

If you can see a menu, do the following:

1. At prompt "Choose a digit", 0 <ENTER>
2. At the top left hand part of the screen you should see
       "High Voltage: OFF". HV is now off.

If you do not see a menu, do the following:

1. Hit the "Esc" key, this will return you to the main menu, then follow instructions above.

If this does not work, call an expert.

How to turn on the PCAL HV monitoring mode

1. If you see a DOS prompt, stop. Follow the instructions for turning on the PCAL HV.
2. If you do not see a DOS prompt but do not have a menu, stop. You are already in monitor mode.
3. If you see a menu, continue to step 4.
4. Push 7 and [Enter] ("Monitor mode")
5. Accept the default value(90 volts) for the "Allowed tolerance (returned - desired)"
6. Accept the default value(90 volts) for the "Allowed tolerance (returned_old - returned_new)
7. Enter 300 (i.e. 5 minutes between reading voltages)
8. In the lower left hand corner, the computer will say "Taking data..." Wait for this message to go away.
9. Find the line that looks like

   stn    drv    chn    volt0    volt1    volt2    new-old   Test #     1

10. Make sure there are no numbers in the columns below this line. If there are, call an expert. Some tubes are not powered up properly.

Monitoring Mode Description

    In monitoring mode the PCAL checks that the read back voltages are within tolerances of the set voltages, and the previous read back.

How to tell if a channel is out of range?

The monitoring screen will looks like

                     Monitoring Voltages

stn    drv    chn    volt0    volt1    volt2    new-old   Test #     1

If there are no entries under this table, then there is no problem, and all channels are within range.

If there are entries under this table then these channels are outside the tolerance. For example

                    Monitoring Voltages

stn    drv    chn    volt0    volt1    volt2    new-old   Test #     1

 1      3     10      800      600      600      200

Note: 

Volt0 = the set point voltage

Volt1 = Previous read back voltage

Volt 2 = Current read back voltage

new-old=Volt2-Volt1

Starting the PCAL Event Display

1. Find the computer labeled "Online Monitoring #1."
2. Try to find the window titled "PCAL Online." You may need to look at each desktop. If you are not sure how to do this, follow these steps:
   1. Find a bar at the bottom of the screen.
   2. Near the middle of the bar, toward the right, you should find a square that is divided into 4 squares.
   3. These 4 divisions are the 4 desktops. Click on a small square to view that desktop.
3. If you do NOT find the window titled "PCAL Online," go on to step 5.
4. If you DO find the window, look at the title of the left histogram. It will say "PCAL Event #." This number should be increasing (possibly slowly). If the event number is NOT INCREASING, the event distributor died. See the event distributor documentation for instructions. If the event number is increasing, stop. The PCAL Event Display is started and running properly.
5. Open a terminal. (Click the computer monitor icon that is in the left portion of the bar.)
6. Enter the following command:

~/pcalDisplay

5. Wait a few moments. The PCAL event display should start. You will see two colorful histograms updating. The left histogram shows the pedestal subtracted ADCs of the current event. The right histogram shows the pedestal subtracted ADCs averaged over a number of events. If you do not see the histograms after a few minutes, call an expert.

Don't see cosmics in PCAL event display

First note that it may take some time to see a cosmic in the event display, depending on the DAQ heartbeat rate.

1. Make sure the DAQ is running with "SICOSM" run type.
2. Make sure the PCAL HV file "CS011905.dat" is loaded and that PCAL HV is "ON".
3. Make sure the event number on the event display is increasing (possibly slowly). This number is in the title of the upper left histogram.
4. If all of the above are true, make sure the Spec Trig LED gates are not flooding the DAQ:
   1. Find the NIM bin D.
   2. Look at the third module from the left, it should be a quad coincidence module.
   3. Find the coincidence switch in the bottom square on this module. (In this square, there should be two spec trig cables plugged in.)
   4. The switch should be in "AND" mode.