The goal of this program is to quickly pull and install repos from its list
Quickly install your favorite Security repos on a new system
Kick off multiple concurrent git clone tasks utilizing Python
Add/remove Security related Github repos to the default.xt list as needed in order to address indivudual use case/project needs
Clone repos to review offline while traveling without internet connectivity
This program was tested with Python3 version 3.8.9 64-bit
Ensure the Python3 virtual environment package is installed (Ubuntu)
sudo apt-get install python3-venv
Ensure git is installed (Ubuntu)
sudo apt-get install git
Clone code repo
git clone https://github.com/tbalz2319/RapidRepoPull.git
Change directory into RapidRepoPull
cd RapidRepoPull
The script will run in a minimal Alpine Docker container (126 MB) and extract the dirs in the current working dir
docker-compose up --build
Clone code repo
git clone https://github.com/tbalz2319/RapidRepoPull.git
Change directory into RapidRepoPull
cd RapidRepoPull
Execute the script below
./install.sh
Clone code repo
git clone https://github.com/tbalz2319/RapidRepoPull.git
Change directory into RapidRepoPull
cd RapidRepoPull
Create a virtual Python3 environment to run this code
python3 -m venv venv
Activate the virual enivornment
source venv/bin/activate
Install requirements
pip install -r requirements.txt
Run program
python rapid.py
Clone code repo
git clone https://github.com/tbalz2319/RapidRepoPull.git
Change directory into RapidRepoPull
cd RapidRepoPull
Execute the script below
./option4.sh
The script will exit while leaving the program in a python virtual env indicated by (venv)
Use the help menu below to navigate options
Usage: rapid.py [OPTIONS]
Options:
-v, --verbose Will print verbose messages.
-f, --file TEXT Specify a text file with a list of user selected Github repos
-t, --thread TEXT Specify the number of CPU threads to use
-u, --url TEXT Specify a url to scrape containing Github repos to clone
--help Show this message and exit.```
Example specifying a custom text file with a list of repos and how many threads the script should use
The text file must reside in the same directory as rapid.py and can be called anything
The example file "bob.txt" is there to testing an example purposes
python rapid.py -f bob.txt -t 25
The example below with scrape a url containing Github to clone and provide verbose output, utilizing 75 threads
python rapid.py -v -u https://apsdehal.in/awesome-ctf -t 75
Run the following script
./update.sh
Run the command to maintain all existing repos by attempting to pull latest version
./update_repos.sh
Add error handling
Clean up code
Implement new features to enhance functionality via cli
The code for this project is constantly being cleaned up, refactored and refined, there are most likely lots of bugs that need to fixed and identified.
Any help with the project is greatly appreciated
Pull requests welcomed