A URL shortener written in Go, with a Mongo based backend, Prometheus and Grafana based monitoring, Memcached based write-through caching. The whole application is also dockerized.
This repo contains implementation of a Url Shortner written in Go.
Wanted to Learn Go and system design, by building a project. Hence goRubu.
NOTE - Url Shortner is a read heavy system (read:write = 100:1), and these load tests are done on a single Machine (8Gb Ram, i5 Processor).
Check this out for more info.
http_req_duration..........: avg=2.39s min=0s med=2.27s max=7.51s p(90)=3.65s p(95)=4.16s
http_reqs..................: 49587 413.224692/s
http_req_duration..........: avg=8.6s min=0s med=8.6s max=19.03s p(90)=10.33s p(95)=10.69s
http_reqs..................: 13345 111.207074/s
git clone https://github.com/rv404674/goRubu.git
cd goRubu
make docker
Note - I haven't dockerized prometheus and grafana with it. Containerized goRubu and locally installed prometheus and grafana will work fine as prometheus is only listening to "local:8080/metrics".
Check the Api's Section afterwards.
Ensure you have the following installed. Mongodb Memcached Make
On Macos, simply use thse
$ brew update
$ brew install mongodb/brew/mongodb-community
$ brew install memcached
$ brew install make
Note: After that check whether these have been started, else you will get connection error.
$ brew services list
if any service is not up do
$ brew service start service_name
Then we need to download the tar files for prometheus, grafana and node exporter. We will need to edit their config files, hence we are not using brew install for these. Also it becomes easy to run the server and do changes.
For Grafana
wget https://dl.grafana.com/oss/release/grafana-6.7.3.darwin-amd64.tar.gz
tar -zxvf grafana-6.7.3.darwin-amd64.tar.gz
mv grafana-6.7.3 /usr/local/bin
cd /usr/local/bin/grafana-6.7.3/bin
./grafana-server # this will run the grafana server
Note - By default prometheus server runs on 9090, and grafana on 3000.
Similarly download tar files for prometheus and node exporter and run there servers as well.
Go to the dir where prometheus is installed and change the prometheus default .yml file to this one new_yml, run the prometheus server, and node_exporter server.
do
$git clone https://github.com/rv404674/goRubu.git
make deps
It will install all the go dependencies.
make install
make execute
Note: To see what these commands do check out this makefile
{
"Url": "https://www.redditgifts.com/exchanges/manage"
}
The endpoint will return a shortened URL.
{
"Url": "https://goRubu/MTAxMTA="
}
Working:
Grafana on Top of Prometheus
Peformance Improvements, bug fixes, better design approaches are welcome. Please Discuss your change by raising an issue, beforehand.
MIT © Rahul Verma