extracting specific text from documents using machine learning algorithms
abstract
increasing use of portable document format (pdf) files has promoted research
in analyzing the files' layout for text extraction purpose. for this reason, it is important
to have a system in place to analyze these documents and extract required
text. the purpose of this research fulfills this need by extracting specific text from
pdf documents while considering the document layout. this approach is used to
extract learning outcomes from academic course outlines. our algorithm consists of
a supervised leaning algorithm and white space analysis. the supervised algorithm
locates the relevant text followed by white space analysis to understand document
layout before extraction. the supervised learning approach used for detecting relevant
text does so by looking for relevant headings, which mimics the approach used
by humans while going through a document.
the data set used for this research consists of 500 course outlines randomly sampled
from the internet. to show the capability of our text detection algorithm to
work with documents other than course outlines, it is also tested on 25 reports and
articles sampled from the internet. the implemented system has shown promising
results with an accuracy of 81.8% and remediated the limitation shown by the current
literature by supporting documents with unknown format. the algorithm has a wide
scope of applications and takes a step towards automating the task of text extraction
from pdf documents.