Sense and Reference on the Web

Harry Halpin, <H.Halpin@ed.ac.uk>

Textorized Parthenon

Sense and Reference on the Web

What are semantics in the Web?

Is is something to do with meaning? Reference? Formal Semantics?

To a computer, then, the web is a flat, boring world devoid of meaning...This is a pity, as in fact documents on the web describe real objects and imaginary concepts, and give particular relationships between them...Adding semantics to the web involves two things: allowing documents which have information in machine-readable forms, and allowing links to be created with relationship values.TimBL, WWW1994

The Problem

The Semantic Web uses URIs not just to refer to web-pages, but to things.

For the Semantic Web to work, there must be agreement from decentralized agents on what "things" a URI refers to.

Everything Must Have a URI?: The Eiffel Tower versus a web-page about the Eiffel Tower: Isn't there something fundamentally different? And what URI do you use for the Eiffel Tower qua Eiffel Tower?

Eiffel Tower Homepage

Eiffel Tower Picture

The URI of the web-page is clearly http://www.tour-eiffel. fr/.

The Thesis Question

Thesis

How can in an agent determine what does a URI refer to or mean? This is trivial for the hypertext web, as it simply can be given by whatever web-page is accessed by the URI. But on the Semantic Web, URIs are used to refer!

We're going to have use an interdisciplinary approach from two rarely connected fields: philosophy and the World Wide Web.

Information, Encoding, Content

information

Information: whatever in common between two things, realized by a message.

Encoding: a set of precise regularities that can be realized by a message. (English syntax, volatages)

Content: whatever is held in common between the source and the receiver as a result of the conveyance of a particular information-bearing message. (Semantics of a sentence, pictures)

Frege and Sense

Frege, the father of logic, posits that the actual thing in the world is the referent, and a name is a symbol that identifies a referent(s). The sense is the mode of presentation, a type of public, objective(?), knowledge about that private concept among a shared community. The third party of sense (meaning) mediates the reference relationship.Gottlieb Frege

The priority of meaning over reference

Example:Hesperus has a sense ("the morning star") different from that of Phosphorus ("the evening star"), yet both have the same referent, the planet Venus.

Sense = Content of Information (the result of an interpretation of an encoding).

Referential Chain

referential chain

Representations are just information with distal content.

By virtue of some distant causal connection, some representation does work as a "stand-in" for a distal referent in order to accomplish some purpose.

Web Architecture

resource

Resource: Any thing capable of having identity. A resource is typically not a particular encoding of the information but the sense that can be given by many encodings (media types).

Web Representation: The encoding of the content given by a resource given in response to a request to a URI.

RDF data...

a set of circles and arrows

...merges just like that.

more circles and arrows superim

Subject and object node using same URIs

The Semantic Web’s real selling point is URI-based data integration.

Links Join Across Different Sources of Information

 Links between column headings

Subject/Property/Object of RDF are just like semantic networks, but with URIs

The Identity Crisis

Infamous argument between Pat Hayes and Tim Berners-Lee.

Social Meaning and RDF: "the meaning of an RDF document includes the social meaning, the formal meaning, and the social meaning of the formal entailments" so that "when an RDF graph is asserted in the Web, its publisher is saying something about their view of the world" and "such an assertion should be understood to carry the same social import and responsibilities as an assertion in any other format" RDF Concepts and Abstract Syntax Draft

Berners-Lee said "a single meaning is given to each URI" which is summarized by the slogan that a URI "identifies one thing."

Hayes retorted that "I'm not saying that the `unique identification' condition is an unattainable ideal: I'm saying that it doesn't make sense, that it isn't true, and that it could not possibly be true. I'm saying that it is crazy" since it goes against the "basic results in 20th century linguistic semantics"

Berners-Lee responded that "`we are not experimental philosophers, we are philosophical engineers" such that "we are not analysing a world, we are building it"

Two positions

  1. Logicist Position: the meaning of a URI is given by whatever model(s) satisfy the formal semantics of the Semantic Web
    1. "the Semantic Web languages would operate exactly unchanged if the identifiers in them were not URIs at all, and if the Web did not exist" (Hayes, In Defense of Ambiguity).
    2. Corresponds to Russell's descriptivist theory of reference.
  2. Direct Reference position: the meaning of a URI is whatever was intended by the owner
    1. Corresponds to Kripke's causual theory of reference.
    2. The position held by Berners-Lee, seems to make sense trivially on the hypertext Web (a URI identifies the web-page it accesses!)

Russell's Descriptivist Theory of Names

Bertrand Russell

Patches of sense-data known through direct acquaintance allow one to ground the atoms of logical statements or create descriptions that can form the basis for names.

The priority of reference over sense We are also assuming the real world (as opposed to any logical idealization of it) is divided neatly into individuals.

Russell deals with referents in the past, or imaginary referents by having a name be a short-cut to a set of logical expressions.

Names are shorthand for logical expressions - so the name "Eiffel Tower" includes a host of facts about it such as "in Paris" and "completed in 1889", as well as an existential quantification!

The Logicist Position

Tarski removed the quaint Russellian sense-data epistemology, grounding out in just a mathematical model defined by extentional satisfication, composition, and axioms.

Pat HAYES This formal semantics became the foundation of the Semantic Web via the RDF Formal Semantics specified by Pat Hayes.

Ambiguity is built in: The web of logical statements is the bearer of meaning, and whatever satisfies the model could be a referent.

The First Generation Semantic Web

But...RDF has little inferential power, so almost anything satisifies formal semantics. Pat HAYES

From what does one build the model of a RDF graph? All the triples that use a URI on the Web? Just the triples at hand? Triples accessible via the URI?

In this reading, URIs are no different than any other symbol, and access via a URI is orthogonal to its referential use. Therefore, the Semantic Web is not interestingly different from any other KR language!

Kripke's Causal Theory of Names

A name refers to same referent in all possible worlds and is so independent of any logical description.

A name can be transmitted through time via historical and causal chains, with a name being given its original referent through a process Kripke calls baptism. Saul Kripke

Putnam's doctrine of natural kinds being assigned names by scientists is an expansion of the causal theory of reference to deal with more than proper names.

Berners-Lee's Direct Reference Position

causal

A URI is given its meaning by a baptism by its owner, such as registering a domain name.

Only the intended referent of the name matters and is primary over any accessible Web representations, and each URI should be given to an individual referent.

Non-Information Resources

causal

Linked Data

The application of Web architecture and the 303 decision to the Semantic Web is the second-generation Web of linked data.

Claims billions of RDF triples! causal

Are there URIs in Practice

Question:Are there too many or no URIs of interest?

Answer: : Sample the Semantic Web using a query log, prune any query with less than 10 repetitions.

Use brute-force simple rules and gazetteers (U.S. Census for names, Alexandra project for places, Wordnet and hyponym with hypernyms) to discover named entities with low recall and high precision.

Data Set: Microsoft Live Search Query Log for 1 month from 2007

Too Many URIs!

There are too many URIs for the same thing...

An average of 1,339 URIs (S.D. 8,000) returned per query. That's a lot, but most may obviously be URIs that just mention the term...

return frequency

Top Entities and Concepts

Number of Searches (RDF Hits) Entity:
  1. 7311 (99) david blaine
  2. 2997 (134) jessica alba
  3. 2100 (16723) nick
  4. 1280 (178) michael hayden
  5. 1098 (10) marcus vick
  6. 1092 (199) keith urban
  7. 1015 (43) lane bryant
  8. 990 (55) desmond dekker
  9. 922 (312) jennifer white
  10. 900 (100) clay aiken
  11. 883 (359) bill cosby
    Concepts:
  1. 11383 (10767) weather
  2. 10321 (7777) dictionary
  3. 3675 (434333) people
  4. 3217 (189115) music
  5. 3117 (7196) monster
  6. 2192 (1444) autism
  7. 1468 (149436) map
  8. 1198 (17562) travel
  9. 1191 (12067) pregnancy
  10. 1104 (82074) news

Look, a power-law!

Very few empirical studies have been done on Linked Data, and often of the type "Look, it's a power-law!" without any proper statistical tests. See Clauset, Newman et al. paper referenced in my paper for a better method! correlation frequencies

Entity Queries: alpha = 2.31, with long tail behavior starting around a frequency of 17 and a Kolmogorov-Smirnov D-statistic of .0241, indicating a significant good fit.

Concept Queries: The alpha= of the queries for concept queries was calculated to be 2.12, with long tail behavior starting around a frequency of 36 with a Kolmogorov-Smirnov D-statistic of .017.

Hypertext Query Frequency and Linked Data?

Question: Is the amount of Linked Data returned correlated with the popularity of the query?

No: Spearman's rank correlation statistic was the insignificant .0077 ($p > .05), while for concept queries, the correlation was the still insignificant at .0125 ($p > .05$)

correlation frequencies

RDF and OWL Constructs in Crawled Triples

Number Percent Language Construct
73,451 30.31% rdfs:Class
47,044 19.30% rdfs:comment
44,113 18.10% rdfs:subClassOf
8,630 3.54% owl:Ontology
7,256 2.97% rdfs:label
6,618 2.14% rdf:Subject
5,107 2.09% owl:ObjectProperty
3,642 1.49% rdfs:subPropertyOf
1,157 0.47% owl:sameAs
535 0.29% rdfs:range

The owl:sameAs Debate

One of the most popular OWL constructs is indeed the controversial owl:sameAs term used to declare equivalence.

One critique holds that it is declaring equivalence between things that are not equivalent, so equivalence is being used too much.

Only 0.47% of overall Semantic Web modelling term usage, it is far from insignificant, with 1,157 occurrences.

Given the amount of Semantic Web URIs returned by the queries, it appears that the manual discovery and publication of co-referential URIs using owl:sameAs falls far behind the actual growth of Linked Data.

Surprise! Likely owl:sameAs is not being used enough.

A Wittegensteinian Theory of Reference

Also could be considered Neo-Fregean, as the Fregean "sense" is the "objective" sense shared in a language. The Semantic Web is just a language that allows co-ordinated action. Ludwig Wittgenstein

The amount of co-ordination needed does not need full agreement or lack of ambiguity, but only the minimal agreement needed to get the task at hand completed.

All words in a language (even knowledge representation languages) are given meaning by their use with other words (language-game), and these parasite off the meaning of natural language terms.

Sense can be reconstructed to be construed in terms of the socially-grounded objective use that is necessary in order to grasp the use of a name across a language.

The sense (and secondarily, possibly reference can only be detected, by the use of the language in an embodied form of life, an agent accomplishing real-world tasks.

The Public Language Position

How can we boot-straps URIs for things and re-use these URIs in a way that people will actually use?

Answer: Use a form-of-life average Web users are familiar with, such as a search paradigm.

Since knowledge representation languages and hypertext web-pages use the same sense, they share a representational nexus

We can then use relevance feedback from the hypertext web-pages to improve the results of the Semantic Web search.

Referent via Serach

referent

An Algorithm

  1. Given a term, retrieve a set of web-pages (Using Yahoo!).
  2. Given a term, retrieve a set of Semantic Web URIs and all triples (facts) associated with them, using FALCON-S.
  3. Human searches through web-sites.
  4. For each web-page the human clicks:
    1. Strip out HTML and reduce to words.
  5. For each Semantic Web returned by the query log
    1. Extract all text and typed data from each RDF fact
    2. Decompose RDF into "a bag of words" with lemmatization and removal of words from end of URI.
    3. match converted RDF to HTML using information retrieval text.
  • Pick the URI with the best ranking score given by IR techniques.
  • Relevance Feedback

    Have a human judge actually figure out what web-pages are relevant, and then use those to feed back and expand the query, in order to re-rank the results.

    In their instructions, relevance was defined can be determined by whether or not accurate information about the information need is expressed by the result. This excludes both link farms, non-standard redirects, and legitimate hubs.

    Experiment: Had 200 queries from previous work, retrieved top 10 hypertext Web (Yahoo!) results and top 10 (FALCON-S) results each judged by 3 judges for relevancy. Fleiss's Kappa=0.5724$ (p < .05, 95% Confidence interval [0.5678,0.5771]), indicating the rejection of the null hypothesis and moderate agreement.

    referent

    Results

    Results of Relevance Judgements:

    Results: Hypertext Semantic Web
    Resolved: 197 (98%) 132 (66%)
    Unresolved: 3 (2%) 68 (34%)
    Top Relevant: 121 (61%) 76 (58%)
    Top Non-Relevant: 76 (39%) 56 (42%)

    Relevance Results: Hypertext

    referent

    Results of Querying the Hypertext Web

    Relevance Results: Semantic Web

    referent

    Results of Querying the Semantic Web

    Finding Right IR Technique and Parameters

    referent

    Average Precision Scores for Vector-space Model Parameters: Relevance Feedback From Hypertext to Semantic Web

    Next result was BM25 with the slight performance-enhancing modifications used in the InQuery system and comparison function used with standard Rocchio relevance feedback with slight modifications as used by Okapi and window size = 100 was best.

    Finding the Right URI is now Acceptable!

    referent

    Why?

    Text automatically extracted from hypertext documents is `messy,' being of low quality and bursty, with highly varying document lengths.

    So, it is unwise to normalise the models, as that will almost certainly dampen the effect of valuable features like crucial keywords.

    The reason BM25-based vector models in particular perform so well is that they are able to effectively keep track of both term frequency and inverse term frequency accurately.

    BM25 provides a slight amount of rather unprincipled non-linearity in the importance of the various variables, effectively keep track of both term frequency and inverse term frequency accurately while massively lowering the power of another document length.

    Run it in reverse!

    Now apply relevance feedback from Semantic Web search engines to the hypertext Web! referent

    Average Precision Scores for Language Model Parameters: Relevance Feedback From Hypertext to Semantic Web

    The best relevant models sampled over top 10,000 words with a cross entropy smoothing factor set to .5. Relevance models over all concatenated relevant documents beats relevance models with documents sampled individually and then combined, as well as all vector-space models.

    Looking at results

    Results: Feedback FALCON-S
    Top Relevant: 118 (89%) 76 (58%)
    Non-Relevant Top: 14 (11%) 56 (42%)
    Non-Relevant Top Entity: 9 (64%) 23 (41%)
    Non-Relevant Concept: 5 (36%) 33 (59%)

    A respectable 19% in average precision over the engine FALCON-S, intuitively makes the system's ability to place a relevant URI in the top rank acceptable for most users.

    The Semantic Web can help Hypertext Search

    referent

    Why?

    As the Semantic Web data is mostly manually high-quality curated data from sources so the actual natural language fragments on the Semantic Web (found for example in Wikipedia abstracts) are much better samples of natural language than the natural language samples found in hypertext.

    The distribution of `natural' language terms extracted from RDF terms, while often irregular, will either be repeated very heavily or fall into the sparse long tail, which can then be dealt with by relevance models.

    The hypertext search engine is being `seeded' with a high-quality accurate description of the information need expressed by the query to be used for query expansion.

    Please see paper for actual equations describing all the IR frameworks we used, i.e. relevance models, BM25, Ponte's method, local content analysis, and more.

    Summary (Part 1)

    The Web is an autonomous subject-matter from artificial intelligence, one that is built on Licklider and Engelbart's vision of humans and machines "collective intelligence."

    Not only natural languages but knowledge representation languages can have issues of sense and reference.

    The primary building block of the Web is the URI, which identifies a resource with a distinct sense that may have multiple encodings.

    The real substantive claim of the Semantic Web is that the use of URIs for non-Web accessible entities and concepts will let knowledge representation globalize

    Yet there is no working theory of reference for the Semantic Web, thus URIs are not re-used and shared globally

    Summary (Part 2)

    There are currently too many URIs for a single entity or queries on the Semantic Web

    A neo-Wittgensteian theory of reference would ground meaning, sense, and reference in actual forms of life.

    The primary form of life on the Web is keyword-based searching of natural language hypertext web-pages, so the Semantic Web should build its theory of meaning on that.

    There are too many URIs returned for a single sense as given by a set of keywords

    We can use relevance feedback from hypertext web-pages to select the "best" URI from the Semantic Web.

    Conclusions and Future Work

    Is this a unified picture?

    Thesis Result: Finding and giving meaning to URIs on the Semantic Web can be built out of the semantics implicitly given by natural language the searching behavior of ordinary users - a public language position inspired by Wittgenstein.

    Future work includes:/p>

    1. Technical: Scaling up, multimedia and non-English languages, using "wisdom of crowds", and creating new URIs and associated descriptions from text for queries for entities and concepts without URIs.
    2. Philosophical: Connect this work better with the Extended Mind hypothesis and theories of embodiment, as well as emergent work in "Web Science."

    Any questions?