Morphological searching in XQuery

Morphological searching in XQuery

Postby Jonathan Robie » June 27th, 2011, 1:41 pm

I've been playing with morphological searches in XQuery. I took James Tauber's morphologically tagged SBL GNT and converted it to XML using Python. The markup uses sentences (s), clauses (cl), and words (w), words retain their morphological tags. Here's an excerpt of Luke:

Code: Select all
<file>
<s>
<cl>
<milestone book='03' chapter='01' verse='01' />
<w id='030101' pos='C-' pc='--------' lemma='ἐπειδήπερ'>Ἐπειδήπερ</w>
<w id='030101' pos='A-' pc='----NPM-' lemma='πολύς'>πολλοὶ</w>
<w id='030101' pos='V-' pc='3AAI-P--' lemma='ἐπιχειρέω'>ἐπεχείρησαν</w>
<w id='030101' pos='V-' pc='-AMN----' lemma='ἀνατάσσομαι'>ἀνατάξασθαι</w>
<w id='030101' pos='N-' pc='----ASF-' lemma='διήγησις'>διήγησιν</w>
<w id='030101' pos='P-' pc='--------' lemma='περί'>περὶ</w>
<w id='030101' pos='RA' pc='----GPN-' lemma='ὁ'>τῶν</w>
<w id='030101' pos='V-' pc='-XPPGPN-' lemma='πληροφορέω'>πεπληροφορημένων</w>
<w id='030101' pos='P-' pc='--------' lemma='ἐν'>ἐν</w>
<w id='030101' pos='RP' pc='----DP--' lemma='ἐγώ'>ἡμῖν</w>
<w id='030101' pos='N-' pc='----GPN-' lemma='πρᾶγμα'>πραγμάτων</w>
<punc>,</punc>
</cl>
<cl>
<milestone book='03' chapter='01' verse='02' />
<w id='030102' pos='C-' pc='--------' lemma='καθώς'>καθὼς</w>
<w id='030102' pos='V-' pc='3AAI-P--' lemma='παραδίδωμι'>παρέδοσαν</w>
<w id='030102' pos='RP' pc='----DP--' lemma='ἐγώ'>ἡμῖν</w>
<w id='030102' pos='RA' pc='----NPM-' lemma='ὁ'>οἱ</w>
<w id='030102' pos='P-' pc='--------' lemma='ἀπό'>ἀπ’</w>
<w id='030102' pos='N-' pc='----GSF-' lemma='ἀρχή'>ἀρχῆς</w>
<w id='030102' pos='N-' pc='----NPM-' lemma='αὐτόπτης'>αὐτόπται</w>
<w id='030102' pos='C-' pc='--------' lemma='καί'>καὶ</w>
<w id='030102' pos='N-' pc='----NPM-' lemma='ὑπηρέτης'>ὑπηρέται</w>
<w id='030102' pos='V-' pc='-AMPNPM-' lemma='γίνομαι'>γενόμενοι</w>
<w id='030102' pos='RA' pc='----GSM-' lemma='ὁ'>τοῦ</w>
<w id='030102' pos='N-' pc='----GSM-' lemma='λόγος'>λόγου</w>
<punc>,</punc>
</cl>
<cl>
<milestone book='03' chapter='01' verse='03' />
<w id='030103' pos='V-' pc='3AAI-S--' lemma='δοκέω'>ἔδοξε</w>
<w id='030103' pos='RP' pc='----DS--' lemma='κἀγώ'>κἀμοὶ</w>
<w id='030103' pos='V-' pc='-XAPDSM-' lemma='παρακολουθέω'>παρηκολουθηκότι</w>
<w id='030103' pos='D-' pc='--------' lemma='ἄνωθεν'>ἄνωθεν</w>
<w id='030103' pos='A-' pc='----DPN-' lemma='πᾶς'>πᾶσιν</w>
<w id='030103' pos='D-' pc='--------' lemma='ἀκριβῶς'>ἀκριβῶς</w>
<w id='030103' pos='D-' pc='--------' lemma='καθεξῆς'>καθεξῆς</w>
<w id='030103' pos='RP' pc='----DS--' lemma='σύ'>σοι</w>
<w id='030103' pos='V-' pc='-AAN----' lemma='γράφω'>γράψαι</w>
<punc>,</punc>
</cl>
<cl>
<w id='030103' pos='A-' pc='----VSMS' lemma='κράτιστος'>κράτιστε</w>
<w id='030103' pos='N-' pc='----VSM-' lemma='Θεόφιλος'>Θεόφιλε</w>
<punc>,</punc>
</cl>
<cl>
<milestone book='03' chapter='01' verse='04' />
<w id='030104' pos='C-' pc='--------' lemma='ἵνα'>ἵνα</w>
<w id='030104' pos='V-' pc='2AAS-S--' lemma='ἐπιγινώσκω'>ἐπιγνῷς</w>
<w id='030104' pos='P-' pc='--------' lemma='περί'>περὶ</w>
<w id='030104' pos='RR' pc='----GPM-' lemma='ὅς'>ὧν</w>
<w id='030104' pos='V-' pc='2API-S--' lemma='κατηχέω'>κατηχήθης</w>
<w id='030104' pos='N-' pc='----GPM-' lemma='λόγος'>λόγων</w>
<w id='030104' pos='RA' pc='----ASF-' lemma='ὁ'>τὴν</w>
<w id='030104' pos='N-' pc='----ASF-' lemma='ἀσφάλεια'>ἀσφάλειαν</w>
<punc>.</punc>
</cl>
</s>
!!! SNIP !!!



Then I wrote a simple XQuery library that tests individual words for part of speech, number, person, tense, voice, mood, and case. After that, I wrote an XSLT stylesheet to present results.

Here are a few sample queries. They are all written in standard XQuery, the second and third queries use the morphology library I wrote.

  • Clauses that contain the word βλέπω

    Code: Select all
    <out>{
       for $clause in doc("luke.txt.xml")//cl
       where some $word in $clause/w satisfies $word/@lemma='βλέπω'
       return $clause
    }</out>

  • Clauses that contain the word βλέπω, and also contain one noun that is nominative and one noun that is accusative

    Code: Select all
    import module namespace morph="http://ibiblio.org/bgreek/morph" at "./morph.xqlib";

    <out>{
       for $clause in doc("luke.txt.xml")//cl
       where (some $word in $clause//w satisfies $word/@lemma='βλέπω')
         and (some $word in $clause//w satisfies morph:is-noun($word) and morph:is-accusative($word))
         and (some $word in $clause//w satisfies morph:is-noun($word) and morph:is-nominative($word))
       return $clause
    }</out>

  • Clauses that contain an adjective, article, and noun in that order, where all three are nominative

    Code: Select all
    import module namespace morph="http://ibiblio.org/bgreek/morph" at "./morph.xqlib";

    <out>{
       for $adj in doc("luke.txt.xml")//w
       let $art := $adj/following-sibling::w[1]
       let $noun := $art/following-sibling::w[1]
       where morph:is-adjective($adj)
         and morph:is-nominative($adj)
         and morph:is-article($art)
         and morph:is-nominative($art)
         and morph:is-noun($noun)
         and morph:is-nominative($noun)
       return
          $adj/parent::cl
    }</out>

Is this something that would be generally useful? I could imagine setting this up on the server so that people could write queries like this and get results.

Are there particular kinds of queries we should be sure this can handle?
ἐξίσταντο δὲ πάντες καὶ διηποροῦντο, ἄλλος πρὸς ἄλλον λέγοντες, τί θέλει τοῦτο εἶναι;
Jonathan Robie
 
Posts: 1454
Joined: May 5th, 2011, 5:34 pm

Re: Morphological searching in XQuery

Postby David Lim » June 27th, 2011, 8:37 pm

Jonathan Robie wrote:Is this something that would be generally useful? I could imagine setting this up on the server so that people could write queries like this and get results.

Are there particular kinds of queries we should be sure this can handle?


I think it would definitely be useful to some people including myself, especially if it complements these:
(1) http://unbound.biola.edu/index.cfm?meth ... SearchForm
(2) using regular expressions on http://www.crosswire.org/study/powersearch.jsp?mod=Byz
since these are the only two online resources I know that allow complicated queries, but each has serious limitations.
δαυιδ λιμ
David Lim
 
Posts: 874
Joined: June 6th, 2011, 6:55 am

Re: Morphological searching in XQuery

Postby Stephen Carlson » June 27th, 2011, 9:03 pm

Jonathan Robie wrote:Are there particular kinds of queries we should be sure this can handle?


I would love to query instances of hyperbaton.

Stephen
Stephen C. Carlson, Ph.D. (Duke)
Post-Doctoral Fellow, Faculty of Theology, Uppsala
Stephen Carlson
 
Posts: 1807
Joined: May 11th, 2011, 10:51 am
Location: Uppsala University

Re: Morphological searching in XQuery

Postby Jonathan Robie » June 27th, 2011, 9:12 pm

sccarlson wrote:
Jonathan Robie wrote:Are there particular kinds of queries we should be sure this can handle?


I would love to query instances of hyperbaton.


Is there a specific sequence of things you would want to look for to find these instances? If you tell me in English, I can try writing a query.
ἐξίσταντο δὲ πάντες καὶ διηποροῦντο, ἄλλος πρὸς ἄλλον λέγοντες, τί θέλει τοῦτο εἶναι;
Jonathan Robie
 
Posts: 1454
Joined: May 5th, 2011, 5:34 pm

Re: Morphological searching in XQuery

Postby Stephen Carlson » June 27th, 2011, 9:24 pm

Jonathan Robie wrote:
sccarlson wrote:
Jonathan Robie wrote:Are there particular kinds of queries we should be sure this can handle?


I would love to query instances of hyperbaton.


Is there a specific sequence of things you would want to look for to find these instances? If you tell me in English, I can try writing a query.


Basically, I'd like see cases were one part of noun phrase is before the verb and the other part is after. The noun phrase can be noun - adjective or noun - genitive (in any order).

Example: Gal 3:7 οὗτοι υἱοί εἰσιν Ἀβραάμ. Here the discontinuous noun phrase υἱοί ... Ἀβραάμ straddles the main verb εἰσιν.

Stephen
Stephen C. Carlson, Ph.D. (Duke)
Post-Doctoral Fellow, Faculty of Theology, Uppsala
Stephen Carlson
 
Posts: 1807
Joined: May 11th, 2011, 10:51 am
Location: Uppsala University

Re: Morphological searching in XQuery

Postby MAubrey » June 27th, 2011, 10:25 pm

sccarlson wrote:Basically, I'd like see cases were one part of noun phrase is before the verb and the other part is after. The noun phrase can be noun - adjective or noun - genitive (in any order).

Example: Gal 3:7 οὗτοι υἱοί εἰσιν Ἀβραάμ. Here the discontinuous noun phrase υἱοί ... Ἀβραάμ straddles the main verb εἰσιν.

Stephen


I might have a list lying around somewhere, if you want. Let me dig around my hard drive.
Mike Aubrey
Canada Institute of Linguistics & Trinity Western University Graduate School
MAubrey
 
Posts: 622
Joined: May 6th, 2011, 8:52 pm
Location: British Columbia

Re: Morphological searching in XQuery

Postby Nigel Chapman » June 28th, 2011, 12:19 am

Hi Stephen,

By what criteria would a search distinguish between discontinuous nouns and nouns that are simply unrelated? I don't think a purely grammatical dataset allows that. You might need something like OpenText, where the verbs already "know" their subjects. The construction you describe stands out visually (see Gal 3.7 below), but you probably won't want to scroll through every chapter looking for it unless you're working in only a small corpus like the Paulines.

http://www.opentext.org/texts/NT/Gal/vi ... h3.v0.html

Nigel.
"When eras die their legacies are left to strange police." -- Clarence Day
Nigel Chapman | http://chapman.id.au
Nigel Chapman
 
Posts: 63
Joined: June 3rd, 2011, 4:55 pm
Location: Sydney Australia

Re: Morphological searching in XQuery

Postby Jonathan Robie » June 28th, 2011, 7:29 am

But you could query the OpenText site too, if there were a precise formulation of the question in terms of the structures available there. (It would require a different library to be written, not a huge task.)

On the other hand, if Mike already has a list on his hard drive ;->
ἐξίσταντο δὲ πάντες καὶ διηποροῦντο, ἄλλος πρὸς ἄλλον λέγοντες, τί θέλει τοῦτο εἶναι;
Jonathan Robie
 
Posts: 1454
Joined: May 5th, 2011, 5:34 pm


Return to Bible Study Software

Who is online

Users browsing this forum: No registered users and 0 guests