2008-09-04T00:21:22 *** kgs_evening has quit IRC 2008-09-04T00:34:33 *** dbs has quit IRC 2008-09-04T01:04:38 denials: re: broken entities, I've only recently dipped back into trunk and only saw one issue which I fixed and committed.. entities look good! 2008-09-04T01:05:37 denials: we do need to document how to install for trunk/1.4.. I had to guess and ran into weird things like needing LD_LIBRARY_PATH.. also had to use build-db.sh 2008-09-04T01:30:00 *** Mark__T has joined #openils-evergreen 2008-09-04T05:05:19 *** jeff has quit IRC 2008-09-04T07:18:30 *** jeff has joined #openils-evergreen 2008-09-04T08:37:06 *** kgs_evening has joined #openils-evergreen 2008-09-04T08:37:36 *** kgs_evening is now known as kgs 2008-09-04T08:40:19 @later tell dbs i'm getting the dupe headers in vandelay, too, and it explains a bug i had to workaround in the javascript. i'll see what i can find 2008-09-04T08:40:19 berick: The operation succeeded. 2008-09-04T08:42:33 berick: processing a 24M upload failed :( ... but, a 30 record one succeeded (with item extraction!) :) 2008-09-04T08:42:43 biab 2008-09-04T08:42:51 miker_: do you know where it failed? 2008-09-04T08:43:02 iow, any errors 2008-09-04T08:43:30 and, when you can, can i have a copy of the file? 2008-09-04T09:01:32 *** rsinger_ is now known as rsinger 2008-09-04T09:03:03 *** skmurphy_lt__ has joined #openils-evergreen 2008-09-04T09:03:05 *** skmurphy_lt__ is now known as skmurphy_lt 2008-09-04T09:10:16 goodmorning 2008-09-04T09:12:55 morning, rickd 2008-09-04T09:19:51 *** skmurphy_lt_ has quit IRC 2008-09-04T09:24:58 *** dbs has joined #openils-evergreen 2008-09-04T09:26:11 phasefx: yes, we need 1.4 install instructions but there will be more changes 2008-09-04T09:26:20 LD_LIBRARY_PATH should not be necessary, though. this was recent trunk? 2008-09-04T09:28:28 *** _dkyle1 has joined #openils-evergreen 2008-09-04T09:29:04 dbs: gimme a holler when you start testing vandelay again 2008-09-04T09:29:26 berick: just logging in to give it a shot now 2008-09-04T09:29:37 excellent... 2008-09-04T09:34:12 *** skmurphy_lt has quit IRC 2008-09-04T09:38:47 I'm getting this while trying to view a report I just ran. Not Found The requested URL /reporter/178 was not found on this server. 2008-09-04T09:38:59 is that an indication that something isn't running? 2008-09-04T09:40:58 atheos: see if you have anytyhing in /openils/var/web/reporter/ 2008-09-04T09:41:02 on your apache box 2008-09-04T09:41:57 10 directories, 1 through 8. 2008-09-04T09:42:05 is one of them "178" 2008-09-04T09:42:09 pj 2008-09-04T09:42:10 one called output, one called test. 2008-09-04T09:42:11 er, oh 2008-09-04T09:42:40 none of these folders were created today 2008-09-04T09:43:00 in your error, 178 is the ID of the template (for reference) 2008-09-04T09:43:15 I think you have to create a place for your output to go 2008-09-04T09:43:26 that should already be setup 2008-09-04T09:43:38 but, rickd, that is how it's behaving 2008-09-04T09:43:40 the defaults used to be screwed up 2008-09-04T09:44:01 we fixed that - but I can't remember if the fix is in trunk only, or 1.2.3.0, or before... 2008-09-04T09:44:06 it won't run the report, unless you select an output Rick 2008-09-04T09:44:10 dbs: yes, recent trunk, with the C services. ld.conf pointing to all the right paths... but /opensrf libs in a separate dir than /openils libs 2008-09-04T09:44:50 dbs: it was c services libs not finding opensrf libs 2008-09-04T09:45:00 phasefx: ah, at the moment I think we've all been testing with --prefix=/openils 2008-09-04T09:45:26 dbs: yeah, I think I just need a cheat sheet for how you guys install 2008-09-04T09:45:50 openils autotools probably needs to be taught to add the lib path for opensrf-libs - kbeswick? 2008-09-04T09:54:43 berick: so, maybe some light is dawning here; if I make the entry in eg_vhost.conf, I also have to change VANDELAY_URL in vandelay.js to match, yeah? 2008-09-04T09:54:55 dbs: yes 2008-09-04T09:55:20 berick: heh. I was missing that piece of my brain last night 2008-09-04T09:55:31 dbs: also, i noticed you pasting paths like /vandelay-upload/vandelay/vandelay.html .. is that a typo? 2008-09-04T09:55:34 from last night, that is 2008-09-04T09:55:46 that was the part of my brain that was messed up last night, clearly. 2008-09-04T09:56:35 now I'm back on track, somewhat - getting the dojo interface and entries added to vandelay.queue 2008-09-04T10:03:14 there's one set of marc records I can import repeatedly and get a record queue interface, but there are no records displayed 2008-09-04T10:03:46 so that's something - my other records appear to just crap out 2008-09-04T10:05:03 dbs: possible to share you data sets? 2008-09-04T10:06:25 *** Mark__T has left #openils-evergreen 2008-09-04T10:07:07 also, turning on debug in the html usually helps track down probs 2008-09-04T10:07:22 berick: data sharing? no problemo - one sec 2008-09-04T10:07:30 since dojo like to die quietly 2008-09-04T10:08:42 brb 2008-09-04T10:10:05 berick: for data sets, see http://cvs.php.net/viewvc.cgi/pear/File_MARC/tests/ - music.mrc generates entries in queued_bib_record but no visible records in the queue interface 2008-09-04T10:12:59 example.mrc and sandburg.mrc create queues but die horribly without generating rows in queued_bib_record 2008-09-04T10:19:32 I can also check in the 50 Armenian/Cyrillic records that I have - probably a good set of records for torture-testing in general 2008-09-04T10:20:45 dbs: thanks! i'm gonna give these a run over here 2008-09-04T10:21:26 mind if I create an Open-ILS/tests/ directory to start collecting sets of interesting MARC records for testing purposes? 2008-09-04T10:22:01 no objection here 2008-09-04T10:22:05 Or should I create a top-level /tests directory? 2008-09-04T10:25:39 dbs: i would prefer it to live in Open-ILS, but I have not logical justification for that ;) 2008-09-04T10:25:43 s/not/no/ 2008-09-04T10:26:15 "One subdirectory to rule them all! (Except for that horrible build/i18n/... aberration") 2008-09-04T10:26:17 sure 2008-09-04T10:31:42 *** skmurphy_lt__ has joined #openils-evergreen 2008-09-04T10:31:43 *** skmurphy_lt__ is now known as skmurphy_lt 2008-09-04T10:32:10 dbs: music.mrc generates records in the queue interface. mind turning on dojo debug and seeing if anything useful comes up? 2008-09-04T10:32:27 s/the queue/my queue/ 2008-09-04T10:33:03 sure 2008-09-04T10:33:04 * berick tries example.mrc 2008-09-04T10:33:29 it might be as simple as me missing some seed values for bib attributes 2008-09-04T10:34:10 that could definitely be it 2008-09-04T10:35:58 example.mrc is also working over here 2008-09-04T10:36:57 peeking at the vandelay_unix.log for music.mrc, I see "no mapping found for [0x99] at position 172" (where the offending char appears to be ♭) 2008-09-04T10:37:28 huh, i guess that's a MARC::Charset thing 2008-09-04T10:38:03 yep, let me paste the whole thing 2008-09-04T10:38:48 dbs pasted "MARC::Charset missing mapping? Or wrong charset in the record?" at http://paste.lisp.org/display/66296 2008-09-04T10:40:06 dbs: ahh, i'm getting those errors as well, fwiw 2008-09-04T10:40:20 hmm, we should pass an explicit $form argument to entityize in Vandelay.pm to avoid some spurious warnings 2008-09-04T10:41:46 heh, debug window is completely empty :) 2008-09-04T10:41:54 i'll look into the seed values 2008-09-04T10:45:16 dbs: form should == 'D' ? 2008-09-04T10:45:21 for all records? 2008-09-04T10:45:30 ohhhh 2008-09-04T10:45:36 to prevent empty string warnings 2008-09-04T10:45:58 that's the direction that miker_ and I were headed, but that's also why I want to build a nice big collection of challenging marc records 2008-09-04T10:46:11 right, for now just make entityize happier :) 2008-09-04T10:47:09 done 2008-09-04T10:48:19 * berick anxiously awaits miker_'s 24M MARC file 2008-09-04T10:49:24 although, until queue paging is added, i anticipate the display will crap out 2008-09-04T10:50:27 berick: I wonder whether exception-based display might be more usable in the long run for those kinds of loads 2008-09-04T10:52:24 dbs: definitely, that's a good pint 2008-09-04T10:52:29 grr.... typing-- 2008-09-04T10:54:54 *** skmurphy_lt has quit IRC 2008-09-04T10:57:00 I would like a good pint 2008-09-04T10:58:40 Freudian typo 2008-09-04T10:58:45 * berick slowly puts the pint down 2008-09-04T11:04:45 http://lcweb2.loc.gov/diglib/codetables/eacc2uni.txt has no mapping for 0x99 2008-09-04T11:06:08 neither does the xml file (just downloaded to check) 2008-09-04T11:06:27 eeevil: any bets we've got latin-1 or something like that in our unicorn? 2008-09-04T11:10:35 it's ... not latin-1 if the byte is 0x99 2008-09-04T11:12:01 weird, that flat character is x266d according to vim 2008-09-04T11:12:09 maybe it's an extraneous byte? 2008-09-04T11:12:15 could be, yeah 2008-09-04T11:12:47 Lots of unhappiness logged for the FSL data 2008-09-04T11:13:09 in latin1 it's the Single Graphic Character Introducer 2008-09-04T11:13:27 excuse me 2008-09-04T11:15:14 not in latin1 (iso-8859-1), but as an escape sequence from ISO/IEC-2022 2008-09-04T11:16:21 hrm... actually 2008-09-04T11:17:15 no ... thought it might be windows-1252 ... it's not 2008-09-04T11:17:55 I hate codepages 2008-09-04T11:19:37 * dbs saddles up and rides that hate-horse with miker_ 2008-09-04T11:23:03 *** mrpeters-isl has quit IRC 2008-09-04T11:23:58 *** mrpeters-isl has joined #openils-evergreen 2008-09-04T11:24:08 *** mrpeters-isl has joined #openils-evergreen 2008-09-04T11:26:14 unicode unicode rah rah rah! 2008-09-04T11:32:13 berick: re vandelay are you planning an import/overlay switch for each queued record? 2008-09-04T11:32:27 (and adding to an existing queue) 2008-09-04T11:35:50 eeevil: each queued record that has a match, yes. import will be the default, unless the overlay flag is set. 2008-09-04T11:35:57 haven't thought about adding to an existing queue yet 2008-09-04T11:36:59 berick: right, what I mean is a per-record switch on the queued record listing that can be used to toggle between import and overlay for that record 2008-09-04T11:38:19 as it stands, you will have to look at the matches before you can select "overlay". if we need an ovelay switch in the top-level grid, that can be added too, but if there are more than one match, they will still have to select which match to overlay 2008-09-04T11:39:16 is it a known issue that does not support onclick's ? 2008-09-04T11:39:24 ahh... I didn't see a thing that did that 2008-09-04T11:39:36 surely something else is wrong here... 2008-09-04T11:39:55 eeevil: if you have a match, the grid will show the matched field in bold red, click on that to review the matches 2008-09-04T11:39:58 I mean, I didn't see the "overlay this record" thing 2008-09-04T11:40:02 i'm adding the overlay flag now 2008-09-04T11:40:03 right 2008-09-04T11:40:07 ahh 2008-09-04T11:40:09 I see 2008-09-04T11:40:28 if i can get my radio input to respond to events 2008-09-04T11:40:40 when I looked earlier it was just showing the isbn matches in read as links ... all will be there, though? 2008-09-04T11:41:33 if you click on the ISBN, it will give you a list of matches, give you the chance to view the MARC, to select an overlay target, and to enable overlaying for this queued record 2008-09-04T11:41:43 *** David_Christense has joined #openils-evergreen 2008-09-04T11:41:52 the last 2 parts are not committed yet 2008-09-04T11:44:24 berick: right ... I saw the ISBN links, but there may not be an ISBN match ... 2008-09-04T11:44:45 s/read/red 2008-09-04T11:45:09 * berick is confused. if there's no match, it won't be red and there will be no link 2008-09-04T11:45:14 cuz there's nothing to look at 2008-09-04T11:45:33 um ... anything that's marked "ident" can cause a match 2008-09-04T11:45:45 yep 2008-09-04T11:45:56 so, there could be a tcn match, but neither record has an isbn 2008-09-04T11:46:10 then the TCN value will be a red link 2008-09-04T11:46:13 isbn isn't special, I mean 2008-09-04T11:46:24 ok, so that is the case now 2008-09-04T11:46:31 yes 2008-09-04T11:46:35 * eeevil goes to find a link :) 2008-09-04T11:46:44 please, break it 2008-09-04T11:46:50 right, I was just wondering that ... it was't before, I mean 2008-09-04T11:47:00 this morning or last night, whenever I looked last 2008-09-04T11:47:14 only the ISBN matches were red 2008-09-04T11:47:26 then there's a bug somewhere 2008-09-04T11:48:30 what's the queue id param? 2008-09-04T11:48:39 qtype=bib&qid=1 2008-09-04T11:49:11 still at ~erickson/vandelay.html? 2008-09-04T11:49:32 yep 2008-09-04T11:49:48 *** rsinger_ has joined #OpenILS-Evergreen 2008-09-04T11:49:59 ok, i see where queue 1 should have some colliding tcn values... 2008-09-04T11:50:43 hrm ... maybe I need a new ses ... 2008-09-04T11:50:52 http://dev.gapines.org/~erickson/vandelay.html?ses=22641df943c8eb5cc96671996264d92c&qtype=bib&qid=1 2008-09-04T11:51:03 just getting the progress bar, ()test and the debug box 2008-09-04T11:54:25 berick: queue 7 is full of examples, fwiw. That's records 20-49 from the dev catalog, imported this morning 2008-09-04T11:57:57 eeevil: where do the values from vandelay.bib_match.field_type come from? i had assumed they mapped to vandelay.bib_attr_definition.code 2008-09-04T11:58:07 *** David_Christense has quit IRC 2008-09-04T11:59:26 berick: those are just descriptive "looks like an isbn, looks like a tcn value" 2008-09-04T12:00:15 you want matched_attr, which points to the attr object, which in turn carries the definition pointer 2008-09-04T12:00:48 what's the purpose of the field_type column? 2008-09-04T12:03:03 if we found the matching value in an 02x field (via metabib.full_rec) on the existing record, it says isbn. found in bibio.record_entry.tcn_value, it says tcn_value. found in biblio.record_entry.id, it says id 2008-09-04T12:03:11 we don't apply the xpath to the in-EG records 2008-09-04T12:03:20 we just look in "identifier-ish" places 2008-09-04T12:03:47 the xpath is just for pulling out values from incoming records that might match one of those in-EG identifier-ish places 2008-09-04T12:04:05 sorry ... that is probably not transparent :) 2008-09-04T12:04:11 s/probably/obviously/ 2008-09-04T12:05:00 we're only looking for collisions on certain parts of the in-EG records, those that could reasonable be considered an identifying label ... id, tcn, isxn 2008-09-04T12:05:32 *** rsinger has quit IRC 2008-09-04T12:05:34 but we may need to look in many places in the incoming records for a tcn or an id, which the xpath tells us about 2008-09-04T12:05:38 does ... that make sense? 2008-09-04T12:05:44 ok, yes, that makes sense 2008-09-04T12:05:51 thanks 2008-09-04T12:06:03 out_of_my_head++ 2008-09-04T12:06:08 ;) 2008-09-04T12:10:20 dbs pasted "empty subfields galore" at http://paste.lisp.org/display/66300 2008-09-04T12:10:53 just because I like throwing curves... 2008-09-04T12:14:54 dbs: if the record says "I'm marc8" then a conversion is attempted, yes. 2008-09-04T12:15:51 dbs: also ... are you passing marcxml? 2008-09-04T12:17:09 nope, binary marc 2008-09-04T12:17:32 hrm... I assume the records did actually have data in the subfields 2008-09-04T12:17:36 originally 2008-09-04T12:17:52 eeevil: oh yes. check Open-ILS/tests/datasets/FSL.marc 2008-09-04T12:18:21 eeevil: okay, so at some point an encoding override will probably be necessary, due to the preponderance of junk data in the library world 2008-09-04T12:18:26 not to worry about for now though 2008-09-04T12:18:38 well, that's possible now, actually... 2008-09-04T12:19:07 MARC::Charset->assume_unicode(1); and MARC::Charset->ignore_errors(1); 2008-09-04T12:19:11 IIRC 2008-09-04T12:19:21 eeevil: heh - yeah, but I meant in the interface :0 2008-09-04T12:19:26 heh 2008-09-04T12:19:26 yeah 2008-09-04T12:19:59 well, it's broader than assum_unicode ... assume_encoding('latin1') is available, IIRC 2008-09-04T12:20:24 and assume_unicode(1) is a wrapper to assume_encoding('utf-8') ... I think 2008-09-04T12:20:39 or, that's how it is in my cvs version of M::Cs 2008-09-04T12:31:23 *** rsinger_ is now known as rsinger 2008-09-04T12:44:57 *** Stompro has joined #openils-evergreen 2008-09-04T12:45:42 berick: wheee ... id and tcn_value matches seem to be working now :) 2008-09-04T12:47:52 berick: we'll probably want a side-by-side comparison view of the records at some point ... should be do-able with the div-ish marc2html thing, eh? 2008-09-04T12:48:21 sure 2008-09-04T12:51:18 ah, ok, so in the list of bib matches, i'll need to display the vandelay.bib_match.field_type or the user won't where the new record matched against the existing record 2008-09-04T12:51:31 * berick just got confused by the new matches that were showing up 2008-09-04T12:52:00 for examlple, Internal ID of 10 will match against an existing record whose tcn-value is 10 2008-09-04T12:52:06 it's coming together ;) 2008-09-04T12:52:19 sk....you around? 2008-09-04T13:55:46 mrpeters-isl: I /believe/ she's doing some on-site stuff today for one of the PINES libs... jfyi... 2008-09-04T14:07:24 thanks :) 2008-09-04T14:17:55 berick: re "or examlple, Internal ID of 10 will match against an existing record whose tcn-value is 10", yes, and an id (bre.id) or isbn of 10 2008-09-04T14:20:16 eeevil: thanks 2008-09-04T14:20:33 that was my last question, actually 2008-09-04T14:38:45 *** dixonj has joined #OpenILS-Evergreen 2008-09-04T14:39:29 hi, has anyone here worked with the MARC metadata format? 2008-09-04T14:39:37 I have a giant MARC file of like 900,000 MARC records, and want to trim off the first 10,000 2008-09-04T14:39:50 anyone know how I might do that? 2008-09-04T14:44:52 dixonj: if you are familiar with Perl, this is a good place to start http://search.cpan.org/~mikery/MARC-Record-2.0.0/lib/MARC/Record.pm 2008-09-04T14:44:59 though, there may be easier ways 2008-09-04T14:48:15 *** Stompro has quit IRC 2008-09-04T14:48:19 *** Stompro has joined #openils-evergreen 2008-09-04T14:52:52 dixonj: (I'm that mikery from cpan) or you can use the indexdata yaz-marcdump tool to turn the records into marcxmlif that's easier for you to work with 2008-09-04T14:57:48 *** kados has quit IRC 2008-09-04T14:58:06 *** kados has joined #openils-evergreen 2008-09-04T15:01:02 dixonj: or even 'yaz-marcdump foobar.marc -snewfile_prefix -C10000' to split foobar.marc into separate files of 10000 records each, with the new filenames beginning iwth "newfile_prefix" 2008-09-04T15:03:37 dbs++ 2008-09-04T15:04:45 assuming yaz-marcdump doesn't corrupt the hell out of the records :) 2008-09-04T15:05:33 hey guys...i want to run a report on the demo server so i can get a list of books/barcodes that i can give to my trainers to use as examples...but alas im not much of an expert on reporting yet, could anyone fill me in on doing this? 2008-09-04T15:06:04 was able to get the patron list by barcode...but no luck getting book titles with barcodes 2008-09-04T15:06:12 Mike did I send you the notes from Shae on this? It might be helpfull. 2008-09-04T15:06:18 spell helpful 2008-09-04T15:06:48 hey rick...no, i dont think you did 2008-09-04T15:06:56 sending now 2008-09-04T15:06:59 thanks! 2008-09-04T15:07:24 sent 2008-09-04T15:08:31 rick...im going to email you a pack of all of the documentation we've been working on if you want to look at any of it 2008-09-04T15:08:45 great 2008-09-04T15:09:08 since i know you're bored at home on "vacation" ;) 2008-09-04T15:09:36 vacation is next week, I am at the library 2008-09-04T15:09:58 ah, my mistake 2008-09-04T15:10:09 everyone at ISL now thinks you were out this week...whoops! 2008-09-04T15:10:18 lol 2008-09-04T15:11:07 thanks for this info....it helps a bit...maybe ill look at the pines templates and see if there is something i can gather there 2008-09-04T15:11:19 K 2008-09-04T15:11:35 did you notice they are up for EI noW? 2008-09-04T15:11:38 will be very nice 2008-09-04T15:12:10 yup 2008-09-04T15:14:07 *** EGUEST635 has joined #openils-evergreen 2008-09-04T15:15:17 *** rsinger_ has joined #OpenILS-Evergreen 2008-09-04T15:31:13 *** rsinger has quit IRC 2008-09-04T15:31:23 *** rsinger_ is now known as rsinger 2008-09-04T16:23:29 *** rickd has quit IRC 2008-09-04T16:57:52 *** rickd has joined #OpenILS-Evergreen 2008-09-04T16:58:35 *** rickd has joined #OpenILS-Evergreen 2008-09-04T17:09:33 *** phase_bb has joined #openils-evergreen 2008-09-04T17:15:02 *** mjg_ has joined #openils-evergreen 2008-09-04T17:40:32 thanks for the suggestions guys 2008-09-04T17:40:42 I'm trying out the yaz-marcdump solution out 2008-09-04T18:13:09 I can get yaz-marcdump to dump out files of size one MARC record, but when I try to add the chunksize parameter -C, it complains I'm not using the right syntax 2008-09-04T18:13:24 can anyone see anything wrong with the way I'm using it? : 2008-09-04T18:13:28 sudo yaz-marcdump -s truncated_iter -C 10000 iter.marc 2008-09-04T18:13:40 the -C 10000 is messing it up 2008-09-04T18:13:51 if I take that out, it splits the file into chunks of size 1 2008-09-04T18:24:16 I think ubuntu gutsy might be rocking an old version of yaz-marcdump 2008-09-04T18:37:27 *** rsinger_ has joined #OpenILS-Evergreen 2008-09-04T18:38:34 *** rsinger__ has joined #OpenILS-Evergreen 2008-09-04T18:54:05 *** rsinger has quit IRC 2008-09-04T18:55:49 *** rsinger_ has quit IRC 2008-09-04T19:36:40 awesome it worked thanks again 2008-09-04T19:36:44 *** dixonj has left #OpenILS-Evergreen 2008-09-04T19:56:11 *** kgs has quit IRC 2008-09-04T20:30:03 *** agJohn has quit IRC 2008-09-04T20:34:41 *** mjg_ has left #openils-evergreen 2008-09-04T20:56:34 *** mrpeters-isl has quit IRC 2008-09-04T21:55:40 *** mib_oqf8fc has joined #openils-evergreen 2008-09-04T22:09:32 so I'm guessing there's supposed to be some seed data for config.circ_matrix_test and a CONFIG_CIRC_MATRIX_TEST_NOT_FOUND event defined, to boot 2008-09-04T22:16:09 i'll just switch back to legacy script-based circulation for now 2008-09-04T23:55:24 *** dbs has quit IRC