the analysis of canada's health through social media using machine learning
abstract
real-time online data processing is quickly becoming an essential tool in the
analysis of social media for political trends, advertising, public health awareness
programs and policy making. traditionally, processes associated with offline analysis
are productive and efficient only when the data collection is a one-time process.
currently, cutting edge research requires real-time data analysis that comes
with a set of challenges, particularly the efficiency of continuous data fetching
within the context of present nosql and relational databases. in this thesis, i
demonstrate a solution to effectively address the challenges of real-time analysis
using a configurable elasticsearch search engine. we are using a distributed
database architecture, pre-build indexing and standardizing the elasticsearch framework
for large scale text mining. the results from the elasticsearch engine is visualized
in almost real-time.
we focused on taking our solution to the challenges of real-time data processing
is to apply it on social media to conduct a large scale health analaysis in canada.
social media a crucial database that provides information on a variety of topics
such as health, food, feedback on products, and many others. at present, people
utilize social media to share their daily lifestyles, for example, where they are
going, what exercise are they doing, or what are they eating. by analyzing the
information, collected from these individuals, the health of the population can be
gauged. this analysis can become an integral part of the government’s efforts to
study the health of people on a large scale. this is because public health is becoming
the primary concern for many governments around the world, and they
believe it is necessary to analyze the present scenario within the population before
creating any new policies. traditionally, governments use a door to door survey,
for example, a census, or hospital information to decide their health policies. this information is limited and sometimes takes a long time to collect and analyze sufficiently
enough to aid in decision making. our approach is to try to solve such
problems through the advancement of natural language processing algorithms and
large scale data analysis. results show, the proposed method provides the solution
in less time with the same accuracy when compared to the traditional one.