Using word vectors to classify spam messages
This is a project done during the Kik hackathon 2017.
In this project we demonstrate a way to classify spam messages on the client while protecting user privacy.
A client generates a "hash" from the message sending it to the server. The server then compares the "hash" to a bank of known reported messages.
The bank of known reported messages is created from spam reports. The server compares a given reported message to the previous bank of reported messages. If the message is similar to a previously reported message, a report count is incremented. Otherwise the message is added to the bank with a count of 1.
A message in the bank of reported messages is considered a spam message once it was reported more than 3 times.
We used 2 datasets for creating sentence vectors:
We played around with a few configurations to get the best results for short user messages:
the
should weigh less than camera
)This project includes a single makefile to help with the initialization, dependency installation and corpus download. You can invoke a help message by running:
make
Or you can manually run the server and client apps:
In the server
directory install the pip dependencies in a virtualenv
:
pip install -r requirements.txt
and run the server:
python app.py
To use the web client go into the webclient
directory in your terminal and then:
npm install
npm run dev
That should install all dependencies and kick start the project, if it all works you should see something like:
Project is running at http://localhost:3333/ webpack output is served from /
Now load http://localhost:3333/ in your browser
There are 3 different "view modes" which can be switched using the select box at the top right corner of the page.
The 3 views are:
IM Sender
) and the ability to report each message as spam.