Hello. What time do you go to sleep? =-O Christian Holm Christensen wrote: >Hi Hiro, > >On Fri, 2006-04-14 at 21:42 -0400, Hironori Ito wrote: > > >>Hello, I agree with you that these things should not be stored in cvs. >>My suggestion would be the following. >> >>1st method: Do similar to the file catalog. Store the information to >>locate the required file (or files) from the database. If the files are >>important (and can not be remade easily), >> >> > >This is kinda crucial. If Erik can make a small script that generates >the needed histogram(s) (perhaps in memory or on disk), then there >really should be no need to store them centrally - as log as he >documents what needs to be done. > > > >>then copy also those files to >>HPSS for archives. >> >> > >If you go for this option, I think the files should be stored in a way >that's accessible to all - that is, store the files in an (x)rootd >server. If BRAHMS is not running an (x)rootd server accessible world >wide, it's quite easy to set up one yourself. coupled with a file >catalogue, and you have a half GRID solution. > > > We are running rootd on every nodes. They are not visible from the outside world. In fact, our filecatalog already have entry to local disk space of each nodes (to be used by rootd). Therefore, we actually have distributed disk capability. We just don't need/use them(???) since we tidy up our disk spaces cautiously. ;-) Copying to HPSS is important particulary for local disks since they are never backed up (and we lose them quick often due to bad disks by excessive heat in the room.) Just like filecatalog, you can have files in HPSS and local or NFS disks. >>2nd method: Or, you can store histogram (or histograms) itself in >>database. To do that just serialize it (or them) and store in MySQL as >>blob. >> >> > >Whoha, that's probably not a good idea. In fact, if I had to redo the >BRAHMS calibration databases, I'd probably go for a file-catalogue+rootd >solution. Blobs are _evil_. > > > >>You can do very easily in RUBY with Marshal class. (or you can >>do also easily in Java, but I guess that is what our BRAT database codes >>also do for arrays of data.) >> >> > >Exactly. An array of parameters is serialised as a text string, with >additional information that indicates size and type. It's really not a >good solution, but it's the best I could come up with at the time. Live >and learn, eh? > > > >> But, I am not sure how fast this process >>is. >> >> > > >Very _very_ slow. Encoding and decoding, database connections, and so >on. > > > >>And, the response of the database might be too slow with larger >>histograms for someone's taste??? >> >> > >:-) > > > Since I have never stored serialized histograms in blob in MySQL, I have no idea how slow it is. But, the slowness might depends on the person. For example, I think that Cross Bronx highway leading upto George Washington bridge in NY city is a giant parking lot. But, New Yorkers are not quite eager to fix this "Highway". :'( If "blob" is bad, do you have any experince with OO database. That might be faster??? >There's something to be said for trying to keep high-level data like the >DST's free from external data. Perhaps you should consider getting the >data needed directly in the code, or make sure the data is used >earlier. > >Yours, > > > _______________________________________________ Brahms-dev-l mailing list Brahms-dev-l@lists.bnl.gov http://lists.bnl.gov/mailman/listinfo/brahms-dev-lReceived on Fri Apr 14 22:47:37 2006
This archive was generated by hypermail 2.1.8 : Fri Apr 14 2006 - 22:47:45 EDT