IncludeOS Versions Save

A minimal, resource efficient unikernel for cloud services

v0.9.1

7 years ago
  • Fixed DHCP
  • Relaxed too strict IP filter
  • Major improvement to testrunner
  • Various bugfixes and minor improvements

v0.9.2

7 years ago
  • Fixed CPU usage issue caused by PIT timeouts
  • Added optional file system caching
  • Various fixes and cleanups.

v0.9.3

7 years ago

Distro support

  • Support for Arch (@GuzTech)
  • Support for Fedora (@andreashappe)
  • Support for LinuxMint (@h4tr3d)

Examples

  • 256-color mode VGA (@lefticus)
  • Snake rewrite (@Voultapher)

Kernel

  • Stack switching utility (context). Create and use custom area in memory as stack for a provided delegate.
  • CPUID (@gurka). Includes KVM feature support

Misc

  • Minor fixes and refactoring (@RicoAntonioFelix)
  • Move ELF symbols (yet again) into its own section, but pre-pruned
  • Further reductions in binary sizes
  • Move stack to the old 640k conventional memory hole, reducing memory usage
  • Updated logotype and READMEs

Heap debugging

  • Validate new and frees
  • Buffer overflow checks (@gurka)
  • Verbose mode for tracking allocations

Utility

  • Added Logger - ring buffer log utility
  • Added Timer - makes it easier to start, stop and restart a task

Profiling

  • Add mode toggle for stack sampler allowing Switch between caller mode (show who calls function) and current mode (show current function).

Networking

  • Packet now travels through the network stack as a unique_ptr instead of shared_ptr (avoiding control block allocation)
  • Isolated network layering a bit by moving ethernet address to hw::MAC_addr
  • TCP optimizations
    • Fixed bug where read would result in double free
    • No longer sets up default callbacks, saving allocations
    • Minor reduction in size

Testing

  • Improvements to test automation

v0.8.1

7 years ago

A few bugs were found and fixed after running a new stress test. The OS seems very stable now, having survived a continous flood of Arp, ICMP and HTTP packets, run in parallel over ~9 hours. The VM showed no signs of having problems and gave a HTTP request rate of 3327.1 conn/s. after the flood had ended.

Test results:

  • Ping-flood: $ sudo ping -f 10.0.0.42 101095327 packets transmitted, 101095294 received, 0% packet loss, time 31206908ms
  • Arp-flood: $ sudo arping -w 100 10.0.0.42 1715415846 packets transmitted, 857201224 packets received, 50% unanswered (0 extra)
  • httperf: $ time httperf --hog --server 10.0.0.42 --num-conn 100000000
Total: connections 382691 requests 382690 replies 382690 test-duration 31343.418 s` 
...
Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0

real    522m53.450s
user    74m18.515s
sys     448m57.559s

The connection rate on http was very poor, 12.2 req/s., but I guess that’s quite understandable. Asking httperf to do a 100 million requests as fast as it possible will very likely just suck up all the available outgoing sockets / file descriptors.

  • VM diagnostics: Current memory usage: 1122304 b, (1.122304 MB) Recv: 23726780 Sent: 277016620

Experiment details:

  • The three tests were run individually in parallell, from separate processes / screens, towards the same, singular VM with one CPU core. The tests were stopped manually after having run overnight.
  • After ending the flood, without restarting the VM, it gave the following response: $ time httperf --hog --server 10.0.0.42 --num-conn 1000
Connection rate: 3327.1 conn/s (0.3 ms/conn, <=1 concurrent connections)
Connection time [ms]: min 0.3 avg 0.3 max 1.6 median 0.5 stddev 0.0
Connection time [ms]: connect 0.2
Connection length [replies/conn]: 1.000

Request rate: 3327.1 req/s (0.3 ms/req)
Request size [B]: 62.0

Reply rate [replies/s]: min 0.0 avg 0.0 max 0.0 stddev 0.0 (0 samples)
Reply time [ms]: response 0.1 transfer 0.0
Reply size [B]: header 234.0 content 489.0 footer 0.0 (total 723.0)
Reply status: 1xx=0 2xx=1000 3xx=0 4xx=0 5xx=0

CPU time [s]: user 0.09 system 0.21 (user 29.5% system 70.2% total 99.7%)
Net I/O: 2553.3 KB/s (20.9*10^6 bps)

Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0

NOTE: We've seen much higher connection rates on other platforms (e.g. 16000 conn/s. has been reported), so the numbers above should not be taken as measure of IncludeOS performance.

v0.8.0

7 years ago

Block device, filesystem and memdisk

  • Finished filesystem interface with sync and async calls
  • Support for custom filesystems
  • FAT 12/16/32 filesystem implementation
  • Added the Memdisk disk device, linked in as binary section
  • Added Virtio Block device driver
  • Added Virtio Console device driver
  • Automated MemDisk builder from disk image file
  • Async class for automatic handling of async file transfers

TCP - more or less complete rewrite

  • State logic is now separated into classes
  • "Socket" replaced with "Connection".
  • Fully implemented RFC 793 "TransmissionControl Protocol".
  • Retransmission, round trip time calculation (RFC 6298)
  • Support options - MSS implemented.
  • Congestion Control (New Reno) (RFC 5681 & 6582)
  • Fixed checksum bug causing instability
  • Send queue - queue several write requests which will be stored until fully acknowledged to be used for retransmission. Possible to avoid any copy by assigning a shared pointer to data.
  • Fair packet distribution among connections.
  • Packets with payload now released immediately after copying data once to a shared buffer, released to the user when subscribed.
  • More events to subscribe to - OnConnect, OnDisconnect among others.

Time

  • Async timers, one-shot and repeating (using PIT)
  • We finally have a wall clock! (RTC via CMOS)

Test

  • Added tests covering the major kernel and networking features
  • New system for automating tests written in python
  • Added support for LEST unit test framework (used in e.g. ./test/GSL)

Misc

  • UDP now has send queue to help maximize bandwidth
  • Improvements in packet- and buffer-management increasing bandwidth
  • Added an optional async terminal with TCP and Serial backend, support for Telnet
  • C++ GSL now included with project, with some limited usage in the kernel
  • New safer virtio interface with GSL
  • Pretty boot
  • Pretty boot screen
  • Tons of bugfixes
  • Updated to llvm 3.8, both libc++ and clang, also for OS X.

Caveats:

  • You currently can't use virtio-block devices together with virtio-net in Qemu, as they'll get the same IRQ number. This has been sorted in dev, but it's involved.
  • The CMOS wall clock is super expensive - I counted about 200k cycles (!). This is because it has to do a lot of single-byte in- and out-instructions which all cause vm exits. We're working on a better way to keep time, possibly synchronizing with CMOS only now and then.

Up next:

On our dev-branch we've replaced the classic PIC-based interrupt system with APIC / IOAPIC and MSI-x. We expect this to give a major boost to resource efficiency and it does make it possible to run virtio-net together with virtio-block. We're also working on a lot of web service stuff, but that's mostly in stealth-mode as of yet - we'll open it up when we have something we think you'll like.

Thanks a lot to the budding community of contributors - we have a long way to go but it feels like we're getting somewhere :-)

v0.8.0-rc.2

8 years ago

Detailed release notes will come with the actual release.

v0.8.0-rc.1

8 years ago

Detailed release notes will come with the actual release.

v0.7.0-proto

8 years ago

In this pre-release:

  • DHCP
  • ARP-resolution
  • UDP Sockets (TCP sockets will be changed to match)
  • New network buffer management system
  • Rewritten the Dev-class with templatized getters
  • Improved virtualbox run-script (etc/vboxrun.sh)
  • Jenkins integration
  • VGA output (somewhat sketchy - rely on the serial port)
  • Successful build on Apple OSX (try ./etc/install_mac.sh)
  • Lots of cleanups, and pretty printing
  • Apache v2.0 license

v0.6.3-proto

8 years ago
  • Partial TCP - enough to show a static webpage
  • Asynchronous timers using the PIT (Polite ones - no context switches). You'll find them in "class_pit.hpp".
  • Major directory structure cleanup, mostly split contents in ./src into folders.
  • Lots of work towards IPv6
  • VirtualBox support is back - but note, ARP is not done, so don't expect proper networking.
  • Install-script finally works - way differently
    • We now install to - and build from $HOME/IncludeOS_install. Much better than installing by default to /usr/local/IncludeOS/. You can move that folder anywhere - just update the first line in the makefiles. (Yes, environment vars are probably what we'll do later, but we're also getting a package manager...)
    • The contents of the install-folder gets gzipped into a binary release bundle. This bundle can be extracted as-is, and there you have everything you need to build IncludeOS (except for local dependencies such as clang). No need to build a cross-compiler!

v0.3.0-proto

8 years ago
  • The OS now builds into a static library, os.a, which gets installed for re-use. This also has the effect of allowing the linker to only extract the OS-objects required by the service.
  • A new service can be created by copying the ./seed, which contains everything necessary to start developing.
  • The "public" ABI is separated from the OS-specific headers.