Resources to fully understand how autonomous drones work.
I originally created this as a short to-do list of study topics for becoming a software engineer, I am creating this list to share my 3 year list of topics that I studied on the side during the curriculum of a Computer Engineering and Informatics degree. This is a list of short and medium length study topics to obtain knowledge regarding autonomous rotorcraft. The items listed here will give you enough knowledge to be able to understand how they work, their limitations and effort required to make them fly. Happy studying!
Format of this is hugely inspired by jwasham/coding-interview-university
Contributions are welcome, so please open a PR if you can improve this list or something is missing!
This is my big study plan for going from simple programmer dude to software engineer who can understand how autonomous drones work.
There are references and video of various popular open source flight controller firmware and guidance projects in order to give the reader a sense of realism and real-sense and real-world value of operations, as well as act as get-to-know-the-market info.
This is meant for software engineers or people who already know how to code and also got basic knowledge of computer science topics including math (probablility, calculus, linear algebra). If you have many years of experience this can be easier to read through, but this is not guaranteed.
When I started this project, I didn't know how a drone could see, how the math behind the control algorithm worked, what the limitations are or anything like that. I started skimming through any resources I could after being inspired by the newly released Skydio r1 and it's CEO's numerous guest lectures at youtube. I slowly started building enough sensors to be able to grasp every sub-component of such flying systems. I have not learned everything yet, but the list is big enough to share and will always be evolving with more stuff added.
It's a long plan. It may take you months. If you are familiar with a lot of this already it will take you a lot less time.
Everything below is an outline, and you should tackle the items in order from top to bottom.
I'm using Github's special markdown flavor, including tasks lists to check progress.
Create a new branch so you can check items like this, just put an x in the brackets: [x]
Fork a branch and follow the commands below
git checkout -b progress
git remote add zarkopafilis https://github.com/zarkopafilis/awesome-dronecraft
git fetch --all
Mark all boxes with X after you completed your changes
git add .
git commit -m "Marked x"
git rebase zarkopafilis/master
git push --force
More about Github-flavored markdown
You can't use only 1 language you are comfortable in to do the coding of the drones. You'd have to rely on many tools. Here are some for example.
You may also need to learn a bit of other tools in the proccess. This is NOT something that you should feel worried about. With studying consistency there is nothing you can't achieve!
This list grew over many months, and yes, it kind of got out of hand.
Here are some mistakes I made so you'll have a better experience.
Take breaks, write down possible things you need to re-review and watch them later or in a different way. This is a lot to take in and breaks are mandatory.
Some concepts can take much longer to understand fully. So if you don't understand it immediately do not worry. I suggest re-studying some of the parts that seemed hard to you after doing one pass on this study list.
There are a lot of distractions that can take up valuable time. Focus and concentration are hard.
Learn to code. This is only required for you to understand how the different algorithms and techniques that will be presented are actually implemented. You need this knowledge to be able to understand the source code of the popular data structure implementations, open source flight controllers and more. Basic data structure and algorithmic complexity should be included.
The next part is needed to be able to understand the math behind the stuff we are going to use. It's not at all harder compared to other fields of study. Hang on and in the end, you'll be surprised by how easy it is to understand everything. This is not the only path to learn these, it's just what I would take. (Yes, I like university lectures and whitepapers.)
Optional
These can get expensive, but here is a list of the popular ones. Most are cheap or free.
This is the basis of rotorcraft that is going to get covered first. These few resources will make you understand what a drone needs to fly bad, good, with the help of extra autonomy engines or with the help of a pilot.
After this part, the corks and screws of each subsystem is going to be investigated thoroughly.
At this point, you might wonder: This is only for four rotors. Don't worry, the extra ones are only used to have resilience. In the future this is going to be populated with more types of rotorcraft like submarines, VTOL drones and wings.
Now that you have got a rough understanding of how you are going to make things fly, it's time to take a better look into the mathematical concepts behind the control of systems.
In order to get the control theory to work, we have to obtain a best-effort state estimate by observing our rotorcraft system.
Kalman Filters
Map Projections
Inertia Measurement Units
Cameras and Optical Sensors
Corrections and Calibration
Communication Protocols
The sensor subsystem is complete. We can now procceed to construct a map of the environment as well as figure out where our UAV is located in the world.
Octomap: An Efficient Probabilistic 3D Mapping Framework Based on Octrees (site)
Autonomous Driving: Localization and Deep Learning Powered Mapping
After proper localization, we are ready to path or mission plan, to make the UAV complete a predefined course of trajectory along with actions
3DVFH+: Real-Time Three-Dimensional Obstacle Avoidance Using an Octomap (paper)
For your own interest. Take a look into the source and issue tracker, maybe join the weekly dev calls.
Autopilots:
SDKs:
Ground Control Stations:
Simulators:
Middleware:
Protocols:
Design Tools:
Flight Evaluation - Rule and Constraint based airframe design and evaluation
For your own interest as well.
In order to do that, you have to find a set of required hardware that is compatible, burn and configure your desired flight controller firmware.
Minimum Hardware
Optional Hardware
Software and Further Configuration
Experienced racing pilots recommend that you stard by flying 'acro'/'freestyle' mode straight from the start. Do not do that with real drone hardware, the probability of crashing within 2 seconds is almost 100%. Take your time on a simulator first. There are many ones that you can find on the most popular gaming platforms like Steam. For example: Liftoff, DRL Sim, etc...
Flying 'angle'/'aided' mode is much easier and the same to flying your typical DJI/Parrot/etc drone around. You can fly this 'line of sight' even if you dont have goggles.
Building Fully Autonomous Rotorcraft is much harder and more resource intensive than building a computer-aided one. You need to do better calibration because there is no pilot directly flying it-only supervising it.
Basic Hardware. Basic in the sense that you need at least those for this to fly autonomously. You could maybe leave the GPS out of that, but you need at least optical flow for exact stabilization.
Optional Hardware
Useful Co-Proccessor Compute Boards
Usually, you start of from the 'job' requirements ex. Mapping, Crop Spraying, Inspection and then decide what hardware is capable of doing this job on a cost-benefit scenario. Configure everything and you should be ready to go.
Congratulations!
Keep learning.
You're never really done. But still, good job :).
How to Make an Infared Night Vision Camera From a Regular Digital Camera
The Reconfigurable Aerial Robotic Chain: Modeling and Control
UAV Swarms
Automatic Resupply and Mission Resum
Deep Learning
Now that you have made it so far, here are the last resources I am able to point you to. From this point on you should have a good understanding of how drones work and how to have a digital perception of the environment. The last parts are exactly a few missing pieces and how to pull everything together:
Lastly, I hope that enjoyed reading and learning from this simple text file as much as I enjoyed researching and writing about it.