DIPF database of publications

Detailansicht Treffer

DIPF database of publications

Show results

Author
Remus, Steffen; Biemann, Chris:

Title:
Domain-specific corpus expansion with focused webcrawling

Source:
In: European Language Resources Association (Hrsg.): Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC 2016) Portoroz : European Language Resources Association (2016) , 3607-3611

URL of full text:
http://www.lrec-conf.org/proceedings/lrec2016/pdf/316_Paper.pdf

Language:
Englisch

Document type
4. Beiträge in Sammelwerken; Tagungsband/Konferenzbeitrag/Proceedings

Schlagwörter:
Algorithmus, Automatisierung, Bildung, Computerlinguistik, Data Mining, Hypertext, Modell, Sprache, Text, Textanalyse


Abstract(original):
This work presents a straightforward method for extending or creating in-domain web corpora by focused webcrawling. The focused webcrawler uses statistical N-gram language models to estimate the relatedness of documents and weblinks and needs as input only N-grams or plain texts of a predefined domain and seed URLs as starting points. Two experiments demonstrate that our focused crawler is able to stay focused in domain and language. The first experiment shows that the crawler stays in a focused domain, the second experiment demonstrates that language models trained on focused crawls obtain better perplexity scores on in-domain corpora. We distribute the focused crawler as open source software. (DIPF/Orig.)


DIPF-Departments:
Information Center for Education

Notes: