In this project, a word counter class is written in python.
It provides two ways to do word count given text input.
- One way is pandas and python native library based.
- Another way is pyspark based.
To run the code, the following python libraries are required:
* pandas (pip install pandas)
* findspark (pip install findspark)
Spark has be installed. For more information, please check the following link:
* https://spark.apache.org/docs/latest/
And put the following code into .bashrc:
export SPARK_HOME=/path/to/spark
exprt PATH=$SPARK_HOME/bin:$PATH