Please use this identifier to cite or link to this item: http://hdl.handle.net/2307/4555
Title: User-assisted synergic crawling and wrapping for entity discovery in vertical domains
Authors: Badr, Celine
metadata.dc.contributor.advisor: Merialdo, Paolo
Keywords: active learning
entity discovery
data extraction
semi structured
Issue Date: 9-Jun-2014
Publisher: Università degli studi Roma Tre
Abstract: Large data-intensive web sites publish considerable quantities of information stored in their structured repositories. Data is usually rendered in numerous data- rich pages using templates or automatic scripts. This wealth of information is of wide interest to many applications and online services that do not have direct access to the structured data repositories. Therefore, there’s a great need to lo- cate such pages, accurately and efficiently extract data on them, and store it in a structured format more adapted to automatic processing than HTML. In this context, we exploit intra- and inter-web site information redundancy to address the problem of locating relevant data-rich pages and inferring wrappers on them, while incurring a minimum user overhead. In the first part, we propose to model large data-intensive web sites, to crawl only the subset of pages pertaining to one vertical domain, and then build effective wrappers for attributes of interest on them, with minimum user effort. Our methodology for synergic specification and execution of crawlers and wrappers is supported by a working system devoted to non-expert users, built over an active-learning inference engine. In the second part, we use the information gathered during inference on the training site, to automatically discover new similar sources on the same type of entities of the vertical domain, which can be useful to complement, enrich, or ver- ify the collected data. Our proposed approach performs an automated search and filter operation by generating specific queries and analyzing the returned search engines results. It combines exploiting existing attributes, template, and page information with a semantic, syntactic, and structural evaluation of newly discov- ered pages to identify relevant semi-structured sources. Both techniques are validated with extensive testing on a variety of sources from different vertical domains.
URI: http://hdl.handle.net/2307/4555
Access Rights: info:eu-repo/semantics/openAccess
Appears in Collections:T - Tesi di dottorato
Dipartimento di Ingegneria

Files in This Item:
File Description SizeFormat
cbadr-phd-thesis.pdf3.06 MBAdobe PDFView/Open
SFX Query Show full item record Recommend this item

Page view(s)

9
Last Week
0
Last month
8
checked on Sep 29, 2020

Download(s)

12
checked on Sep 29, 2020

Google ScholarTM

Check


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.