Concurrency in Go

Summary

Coding Examples

• Introduction to Concurrency
• Go Routines
• Channels
• Select
• Concurrency Patterns
• Atomic(s) - sync/atomic
• WaitGroup(s) - sync.WaitGroup
• Mutexes - sync.Mutex
• Pool - sync.Pool
• Map - sync.Map
• Cond - sync.Cond
• Once - sync.Once
• Race Conditions
• Threads
• GOMAXPROCS
• Testing
• Profiling
• HTTP REST API (Error Handling)
• AWS S3 Bucket Clone

Presentation Notes

• Concurrency in Go #1 - Introduction to Concurrency
• Concurrency in Go #2, #3, #4 - WaitGroups
• Concurrency in Go #5, #6 - Atomic(s)
• Concurrency in Go #7 - Mutexes

Go routines

A go routines can block for one of these reasons:

• Sending/Receiving on channel
• Network or I/O
• Blocking System Call
• Timers
• Mutexes

Here's the full list of Go routines statuses:

• Gidle, // 0
• Grunnable, // 1 runnable and on a run queue
• Grunning, // 2 running
• Gsyscall, // 3 performing a syscall
• Gwaiting, // 4 waiting for the runtime
• Gmoribund_unused, // 5 currently unused, but hardcoded in gdb scripts
• Gdead, // 6 goroutine is dead
• Genqueue, // 7 only the Gscanenqueue is used
• Gcopystack, // 8 in this state when newstack is moving the stack

Feel free to check the rest of the statuses in the runtime source code

Fairness

• Infinite loop — preemption (~10ms time slice)
• Local Run queue — preemption (~10ms time slice)
• Global run queue starvation is avoided by checking the global run queue for every 61 scheduler tick
• Network Poller Starvation Background thread poll network occasionally if not polled by the main worker thread

Channels

Here are couple of simple rules to make sure channels are used correctly

• Before writing to a channel, make sure someone else is reading from it (deadlock)
• Before reading from a channel, make sure someone else is writing to it (deadlock)
• When ranging over a channel, ALWAYS make sure the producer closes the channel eventually (deadlock)
• Writing to a closed channel will result in a runtime panic
• Reading from a closed channel won't have any effects
• A channel close, is considered a write operation

Poll Order

• Local Run queue
• Global Run queue
• Network Poller
• Work Stealing

Deadlocks

The Coffman Conditions are known as the techniques/conditions to help detect, prevent and correct deadlocks. The Coffman Conditions are as follows:

• Mutual Exclusion

A concurrent process holds exclusive rights to a resource, at any one time.

• Wait for Condition

A concurrent process must simultaneously hold a resource and be waiting for an additional resource.

• No Preemption

A resource held by a concurrent process can only be released by that process

• Circular Wait

A concurrent process (P1) must be waiting on a chain of other concurrent processes (P2), which are in turn waiting on it (P1)

Go Scheduler

Primarily the Go scheduler has the opportunity to get triggered on these events:

• The use of the keyword go
• Garbage collection
• System calls (i.e. open file, read file, e.t.c.)
• Synchronization and Orchestration (channel read/write)

P, M, G

G - goroutine M - worker thread, or machine P - processor, a resource that is required to execute Go code. M must have an associated P to execute Go code

Once the syscall exists Go tries to apply one of the rules:

• try to acquire the exact same P, and resume the execution
• try to acquire a P in the idle list and resume the execution
• put the goroutine in the global queue and put the associated M back to the idle list

Goroutines do not go in the global queue only when the local queue is full; it is also pushed in it when Go inject a list of goroutines to the scheduler, e.g. from the network poller or goroutines asleep during the garbage collection

Spinning Threads

Net Poller

SysMon

sysmon is smart enough to not consume resources when there is nothing to do. Its cycle time is dynamic and depends on the current activity of the running program. The initial pace is set at 20 nanoseconds, meaning the thread is constantly looking to help. Then, after some cycles, if the thread is not doing anything, the sleep between two cycles will double until it reaches 10ms. If your application does not have many system calls or long-running goroutines, the thread should back off to a 10ms delay most of its time, giving a very light overhead to your application.

For the implementation details checkout sysmon source code

The thread is also able to detect when it should not run. Here are two cases:

• The garbage collector is going to run. sysmon will resume when the garbage collector ends.
• All the threads are idle, nothing is running.

Work Stealing

Here's how Go makes sure to equally distribute & balance work and make use of computer resources as efficient as possible:

• pull work from the local queue
• pull work from the global queue
• pull work from network poller
• steal work from the other P’s local queues

Tracing

GOMAXPROCS=2 GODEBUG=schedtrace=1000,scheddetail=1 go run main.go

Concurrency Patterns

Pipelines

In general terms a pipeline is a mechanism for inter-process communication using message passing, where the output of a pipeline is the input for the next pipeline.

Suppose that assembling one car requires three tasks that take 20, 10, and 15 minutes, respectively. Then, if all three tasks were performed by a single station, the factory would output one car every 45 minutes. By using a pipeline of three stations, the factory would output the first car in 45 minutes, and then a new one every 20 minutes.

Resources

• OSX - number of CPUs
• Windows - number of CPUs
• OSX - osinit
• Windows - osinit
• Linux - osinit
• Go Scheduler by rakyll
• Go Scheduler by morsmachine
• Illustrated Tales of Go Runtime Scheduler
• Go Scheduler Implementation
• Main Goroutine
• Go Scheduler Implementation
• G - Source Code
• M - Source Code
• P - Source Code
• GRQ Check - Source Code
• Force Preempt Duration for G - Source Code
• Go Scheduler Design Doc
• Scheduling in Go (Part 2) - Ardan Labs
• Scheduling in Go (Part 3 - Concurrency) - Ardan Labs
• Scheduler Tracing in Go - Ardan Labs
• The Scheduler Saga
• Stack size
• Golang Net Poller Source Code
• Golang Net Poller
• Preemptive vs Non-Preemptive Scheduling
• Preemptive vs Cooperative
• Go 1.14 Release Notes - Runtime
• Go Asynchronous Preemption
• Go Routine & Preemption
• Go Routine, OS Thread and CPU Management
• Go Work Stealing - Go Scheduler
• Guarded Command Language
• System Monitor - sysmon
• Go SysMon Runtime Monitoring
• How does Go routine stack size evolve? - Medium
• SysMon - Source Code
• Garbage Collector Period - Source Code
• Preemption - suspend
• Preemption - resume
• Preemption - asyncPreempt
• Preemption - preemptPark
• Scheduling - schedule
• CSP
• Visualising Concurrency in Go
• NUMA Deep Dive
• How to Reduce Lock Contention with Atomic Package
• Go Mutex and Starvation
• Linux Kernel Mutex Lock Hand Off
• Sync Map - Medium
• Pipelines
• Nil Channels
• Context Cancellation
• Concurrency in Go - O Reilly
• Pipeline - Wiki
• Pipeline - Unix
• Mutex of Channel - Golang Docs
• Understand the Design of Sync Pool - Medium
• clearpools - sync.Pool, GC - Go source code
• poolCleanup - sync.Pool - Go source code
• Using Sync Pool
• Concurrency in Go eBook - Amazon