INSTANCE-LEVEL ATTRIBUTE ALIGNMENT FOR HETEROGENEOUS PRODUCT SOURCES

Abstract

This thesis focuses on Big Data integration, a foundational area in data management research. We describe in particular the integration of product specifications from multiple sources of data, with the final goal of building a complete and reliable product graph. Exploiting multiple data sources has the advantage to provide information about rare and niche products and uncommon properties, and having enough redundancy to solve potential conflicts. On the other hand, it involves several challenges due to the heterogeneity of Web sources. We described a complete pipeline for product data integration, involving Web extraction and integration steps, which, unlike traditional approaches, performs the record linkage step (group specifications by product) before attribute alignment step (group attributes with equivalent semantics and define mappings). Indeed, record linkage in product context is simplified by the presence of general product identifiers, while attribute alignment is a very complex task due to presence of a lot of properties about a product, some rarer and some more common, with many different representations. We provided an extensive analysis of the state of the art on these two tasks. We formulated a novel problem of computing attribute alignment at the instance level. Traditional schema-level alignment methods, which critically rely on local homogeneity within a source, are unable to effectively solve this problem due to the significant heterogeneity exhibited by product specifications, both across and within sources. We take advantage of the opportunities arising from the richness and redundancy of information across sources, and propose an iterative solution, called RaF-AIA, that consists of three key steps: (i) First, it uses a Bayesian model to analyze overlapping information across sources to match the most locally homogeneous attributes; (ii) Second, inspired by NLP techniques, it uses a tagging approach to create (virtual) homogeneous attributes from tagged portions of heterogeneous attribute values; (iii) Third, it makes creative use of classical alignment techniques based on matching of attribute names and domains. We developed a publicly available benchmark (Alaska Benchmark) for the tasks of attribute alignment and record linkage, which we also used to run experiments for evaluating the RaF-AiA approach, demonstrating its effectiveness and efficiency, and its superiority over alternative approaches adapted from the literature.