- From: Michiel Hildebrand <Michiel.Hildebrand@cwi.nl>
- Date: 2008年8月18日 10:30:24 +0200
- To: Andreas Harth <andreas.harth@deri.org>
- Cc: semantic-web@w3c.org
- Message-Id: <4977F404-6784-4631-8C70-0165CA9703B8@cwi.nl>
On 17 aug 2008, at 23:22, Andreas Harth wrote: > > Hi Michiel, all, > > Michiel Hildebrand wrote: >> Designing user interfaces for "new" ways of exploration is indeed >> difficult. Evaluating these interfaces is even more difficult. > > I agree with you on the difficulty of evaluation new UIs. Unlinke > traditional fields > such as IR, there is no corpus or evaluation method available for > Semantic > Web data (at least didn't find any). > > To help assessing new UIs for Semantic Web data, we've published a > medium-size > corpus at [1] (~25m triples, ~5GB), together with a set of real- > world user tasks. > There's also ratings which can be used for recommendations. Thanks Andreas, A good initiative. I will load the data into our tools and see what it looks like. From the information I get from the website it looks like you have been using the data (and other materials) for evaluation. I can't find any results though. The link to the technical report is not working. In other words what are your experiences with the use of this dataset and your evaluation methodology? I do not completely get the queries. You list three kinds of queries, directed search, simple browsing and complex browsing. - Why this distinction? - The description of the queries seem a bit limited. - Do you have a golden standard for the answers to these queries? > We've choosen books since there is a public domain data available > and there > has been work in the digital library area, so there are existing > systems to > compare to. ...including evaluations? Michiel
Received on Monday, 18 August 2008 08:30:39 UTC