Digital Forensics Investigation Platform
Digital Investigation Platform
Kuiper is a digital investigation platform that provides a capabilities for the investigation team and individuals to parse, search, visualize collected evidences (evidences could be collected by fast triage script like Hoarder). In additional, collaborate with other team members on the same platform by tagging artifacts and present it as a timeline, as well as setting rules for automating the detection. The main purpose of this project is to aid in streamlining digital investigation activities and allow advanced analytics capabilities with the ability to handle a large amounts of data.
Today there are many tools used during the digital investigation process, though these tools help to identify the malicious activities and findings, as digital analysts there are some shortages that needs to be optimized:
With a large number of cases and a large number of team members, it becomes hard for team members collaboration, as well as events correlation and building rules to detect malicious activities. Kuiper solve these shortages.
Create cases and upload artifacts
Investigate parsed artifacts in Kuiper
Kuiper use the following components:
Flask: A web framework written in Python, used as the primary web application component.
Elasticsearch: A distributed, open source search and analytics engine, used as the primary database to store parser results.
MongoDB: A database that stores data in JSON-like documents that can vary in structure, offering a dynamic, flexible schema, used to store Kuiper web application configurations and information about parsed files.
Redis: A in-memory data structure store, used as a database, cache and message broker, used as a message broker to relay tasks to celery workers.
Celery: A asynchronous task queue/job queue based on distributed message passing, used as the main processing engine to process relayed tasks from redis.
Gunicorn: Handle multiple clients HTTPs requests
Notes
# Install Docker
sudo apt-get update
sudo apt-get install ca-certificates curl gnupg lsb-release
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin
sudo docker -v
# Install Docker Compose
sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
sudo docker-compose -v
Starting from version 2.2.0, Kuiper run over dockers, there are 7 docker images:
To run the docker use the following command:
sysctl -w vm.max_map_count=262144
git clone https://github.com/DFIRKuiper/Kuiper.git
cd Kuiper
docker-compose pull
docker-compose up -d
1 - Note: when you first run the dockers, Elasticsearch will fail to run and give the following error
ERROR: [1] bootstrap checks failed
[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
To solve the issue run the command
sysctl -w vm.max_map_count=262144
2- Note: if you faced the following issue
Creating network "kuiper_kuiper" with driver "bridge"
Creating kuiper_es01 ... done
Creating kuiper_mongodb ... done
Creating kuiper_redis ... done
Creating kuiper_flask ... error
Creating kuiper_nfs ... done
Creating kuiper_celery ...
ERROR: for kuiper_flask Cannot start service flask: error while mounting volume '/var/lib/docker/volumes/kuiper_kuiper_nfs/_data': failed to mount local volume: mount :/:/var/lib/docker/vCreating kuiper_celery ... done
ERROR: for flask Cannot start service flask: error while mounting volume '/var/lib/docker/volumes/kuiper_kuiper_nfs/_data': failed to mount local volume: mount :/:/var/lib/docker/volumes/kuiper_kuiper_nfs/_data, data: addr=172.30.250.10: permission denied
ERROR: Encountered errors while bringing up the project.
To solve the issue, run the command again
docker-compose up -d
To check the dockers, run the command
docker-compose ps -a
It should show the results
Name Command State Ports
------------------------------------------------------------------------------------------------------------
kuiper_celery /bin/sh -c cron && python ... Up
kuiper_es01 /bin/tini -- /usr/local/bi ... Up 0.0.0.0:9200->9200/tcp,:::9200->9200/tcp, 9300/tcp
kuiper_flask /bin/sh -c cron && gunicor ... Up 0.0.0.0:5000->5000/tcp,:::5000->5000/tcp
kuiper_mongodb docker-entrypoint.sh /bin/ ... Up 0.0.0.0:27017->27017/tcp,:::27017->27017/tcp
kuiper_nfs /usr/bin/nfsd.sh Up 0.0.0.0:2049->2049/tcp,:::2049->2049/tcp
kuiper_nginx /docker-entrypoint.sh ngin ... Up 0.0.0.0:443->443/tcp,:::443->443/tcp, 80/tcp
kuiper_redis docker-entrypoint.sh /bin/ ... Up 0.0.0.0:6379->6379/tcp,:::6379->6379/tcp
if anyone failed, check the logs for the service that failed
docker-compose logs -f --tail=100 <service>
Kuiper has a limited feature API, check the repo DFIRKuiperAPI.
We are happy to receive any issues, contribution, and ideas.
we appreciate sharing any parsers you develop, please send a pull request to be able to add it to the parsers list.
Each parser has its own license, all parsers placed in the following folder /kuiper/parsers/
.
All files in this project under GPL-3.0 license, unless mentioned otherwise.
Saleh Muhaysin, Twitter (@saleh_muhaysin),
Muteb Alqahtani, Twitter(@muteb_alqahtani)
Abdullah Alrasheed, Twitter(@abdullah_rush)