2同行回答
请参考：http://chunqi.li/2015/11/15/Battlefield-Calico-Flannel-Weave-and-Docker-Overlay-Network/
Calico: A Solution of Multi-host Network For Docker
Flannel for Docker Overlay Network
Weave: Network Management for Docker
Docker Multi-host Overlay Networking with Etcd
This post provides a battlefiled for these 4 Docker multi-host network solutions, including features and performances.
If you want to see the results directly, directly jump to the Conclusion chapter.
Docker Multi-host Networking Introduction
Docker kicked off with a simple single-host networking from the very beginning. Unfortunately, this prevents Docker clusters from scale out to multiple hosts. A number of projects put their focus on this problem such as Calico, Flannel and Weave, and also since Nov. 2015, Docker support the Multi-host Overlay Networking itself.
What these projects have in common is trying to control the container’s networking configurations, thus to capture and inject network packets. Consequently, every containers located on different hosts can get IPs in the same subnet and communicate with each other as if they are connected to the same L2 switch. In this way, containers could spread out on multiple hosts, even on multiple data centers.
While there are also a lot of differences between them from technical models, network topology and features. This post will mainly focus on the differences between Calico, Flannel, Weave and Docker Overlay Network, and you could choose the right solution which fits best to your requirements.
Battlefield Overview
According the features these Big Four support, I will compare them in the following aspects:
Network Model - What kind of network model are used to support multi-host network.
Application Isolation - Support what level and kind of application isolation of containers.
Name Service - DNS lookup with simple hostname or DNS rules.
Distributed Storage Requirements - Whether an external distributed storage is required, e.g. etcd or consul.
Encryption Channel - Whether data and infomation tranvers can put in an encryption channel.
Partially Connected Network Support - Whether the system can run on a partially connected host network.
Seperate vNIC for Container - Whether a seperate NIC is generated for container.
IP Overlap Support - Whether the same IP can be allocated to different containers.
Container Subnet Restriction - Whether container’s subnet should not be the same as host’s.
Protocol Support - What kind of Layer-3 or Layer-4 protocols are supported.
Now let’s see more details of these aspects on Calico, Flannel, Weave and Docker Overlay Network.
Network Model
Multi-host networking means aggregating containers on different hosts to a same virtual network, and also these networking providers (Calico, etc.) are organized as a clustering network, too. The cluster organizations are called network model in this post. Technically, these four solutions uses different network model to organize their own network topology.
Calico implements a pure Layer 3 approach to achieve a simpler, higher scaling, better performance and more efficient multi-host networking. So Calico can not be treated as an overlay network. The pure Layer 3 approach avoids the packet encapsulation associated with the Layer 2 solution which simplifies diagnostics, reduces transport overhead and improves performance. Calico also implements BGP protocl for routing combined with a pure IP network, thus allows Internet scaling for virtual networks.
Flannel has two different network model to choose. One is called UDP backend, which is a simple IP-over-IP solutions which uses a TUN device to encapsulate every IP fragment in a UDP packet, thus forming an overlay network; the other is a VxLAN backend, which is same as Docker Overlay Network. I have run a simple test for these two models, VxLAN is much more faster than UDP backend. The reason, I suggest, is that VxLAN is well supported by Linux Kernel, while UDP backend implements a pure software-layer encapsulation. Flannel requires a Etcd cluster to store the network configuration, allocate subnets and auxiliary data (such as host’s IP). And the packet routing also requires the cooperation of Etcd cluster. Besides, Flannel runs a seperate process flanneld on host environment to support packet switching. Apart from Docker, flannel can also used for traditional VMs.
Weave also has two different connection modes. One is called sleeve, which implements a UDP channel to tranverse IP packets from containers. The main differences between Weave sleeve mode and Flannel UDP backend mode is that, Weave will merge multiple container’s packet to one packet and transfer via UDP channel, so technically Weave sleeve mode will be a bit faster than Flannel UDP backend mode in most cases. The other connection mode of Weave is called fastdp mode, which also implements a VxLAN solutions. Though there’s no official documents clarifying the VxLAN usage, we still can found the usage of VxLAN from Weave codes. Weave runs a Docker container performing the same role as flanneld.
Docker Overlay Network implements a VxLAN-based solution with the help of libnetwork and libkv, and, of course, is integrated into Docker succesfully without any seperate process or containers.
So a brief conclusion of network model is in the following table:
Calico Flannel Weave Docker Overlay Network
Network Model Pure Layer-3 Solution VxLAN or UDP Channel VxLAN or UDP Channel VxLAN
Application Isolation
Since containers are connected to each other, we need a method to put containers into different groups and isolate containers in different group.
Flannel, Weave and Docker Overlay Network uses the same application isolation schema - the traditional CIDR isolation. The traditional CIDR isolation uses netmask to identify different subnet, and machines in different subnet cannot talk to each other. For example, w1/w2/w3 has IP 192.168.0.2/24 192.168.0.3/24 and 192.168.1.2/24 seperately. w1 and w2 can talk to each other since they are in the same subnet 192.168.0.0/24, but w3 cannot talk to w1 and w2.
Calico implements another type of application isolation schema - profile. You can create a batch of profiles and append containers with Calico network into different profiles. Only containers in the same profile could talk to each other. Containers in differen profile cannot access to each other even though they are in the same CIDR subnet.
Brief conclusion:
Calico Flannel Weave Docker Overlay Network
Application Isolation Profile Schema CIDR Schema CIDR Schema CIDR Schema
Protocol Support
Since Calico is a pure Layer-3 solution, not all Layer-3 or Layer-4 protocols are supported. From the official github forum, developers of Calico declaims only TCP, UDP, ICMP ad ICMPv6 are supported by Calico. It does make sense that supporting other protocols are a bit harder in such a Layer-3 solution.
Other solutions support all protocols. It’s easy for them to achieve so because either udp encapsulation or VxLAN can support encapsulate L2 packets over L3. So it doesn’t matter what kind of protocol the packet holds.
Brief conclusion:
Calico Flannel Weave Docker Overlay Network
Protocol Support TCP, UDP, ICMP & ICMPv6 ALL ALL ALL
Name Service
Weave supports a name service between containers. When you create a container, Weave will put it into a DNS name service with format {hostname}.weave.local. Thus you can access to any container with {hostname}.weave.local or simply use {hostname}. The suffix (weave.local) can be changed to other strings, and the DNS lookup service can also be turned off.