What is semantic accuracy and what are the most common reasons why knowledge graphs are semantically inaccurate?

Semantic accuracy is defined as the degree to which the semantic assertions of a model are accepted to be true. Some key reasons why a semantic model may contain wrong assertions are the following:

  • Inaccuracy of automatic information extraction (IE) methods: This is by far the most common reason and has to do with the less than 100% accuracy of the algorithms that are typically used to extract semantic assertions from data sources in an automatic way.
  • Inaccuracy of the data source from which assertions are extracted: In many cases the data from which we get our assertions (either automatically or manually) can contain errors.
  • Misunderstanding of modeling elements’ semantics and intended usage: Just because a semantic modeling language defines its elements with a specific meaning and behavior in mind, it does not necessarily mean that people will follow this meaning when using the language in the real world.
  • Lack of domain knowledge and expertise: This is the case when we build a semantic model for a specialized domain and we can’t (or don’t) involve in the process the right people with the right kind of knowledge.
  • Vagueness: Vague assertions can be considered true by one group of users and false by another, without any of them being necessarily wrong. Still, however, if we build a model with input from the one group but have it used by the other group, then we should expect that the latter is pretty likely to treat the model as inaccurate

Semantic accuracy is akin to measuring perspective for an individual. The challenge is partially semantic and partially motivational and contextual as all 3 influence the nature of understanding. Algorithmic approaches fall short if they rely on correlation without causality. Causal models are a key part of what provides assurance and trust, so I would want to see understand how folks are able to overcome spurious results with current approaches.

1 Like