cri-o 与 cni的集成分析

 2023-09-10 阅读 16 评论 0

摘要:// 创建pod时,network的设置 1、// cri-o/server/sandbox.go // RunPodSandbox creates and runs a pod-level sandbox func (s *Server) RunPodSandbox(ctx context.Context, req *pb.RunPodSandboxRequest) (*pb.RunPodSandboxResponse, error) CRI是什么意思。在该函

// 创建pod时,network的设置

1、// cri-o/server/sandbox.go

// RunPodSandbox creates and runs a pod-level sandbox

func (s *Server) RunPodSandbox(ctx context.Context, req *pb.RunPodSandboxRequest) (*pb.RunPodSandboxResponse, error)

CRI是什么意思。在该函数中先后调用了:

(0)、首先调用req.GetConfig().GetLinux().GetSecurityContext().GetNamespaceOptions().GetHostNetwork()

如果为true,则调用g.RemoveLinuxNamespace("network")和netNsPath, err = hostNetNsPath()

否则,Create the sandbox network namespace,先调用sb.netNsCreate(),再Pass the created namespace path to the runtime,即g.AddOrReplaceLinuxNamespace("network", sb.netNsPath())

(1)、container, err := oci.NewContainer(containerID, containerName, podSandboxDir, podSandboxDir, labels, nil, id, false)

CRI全称、sb.infraContainer = container

(2)、设置podNamespace := "",调用s.netPlugin.SetUpPod(netnsPath, podNamespace, id, containerName),为容器创建network

(3)、调用s.runContainer(container)

 

// pod创建完成,创建内部的container时,network的设置

// CreateContainer creates a new container in specified PodSandbox

1、// cri-o/server/container_create.go

func (s *Server) CreateContainer(ctx context.Context, req *pb.CreateContainerRequest) (res *pb.CreateContainerResponse, err error)

....

container, err := s.createSandboxContainer(containerID, containerName, sb, containerDir, containerConfig)

...

 

2、// cri-o/server/container_create.go

func (s *Server) createSandboxContainer(containerID string, containerName string, sb *sandbox, containerDir string, containerConfig *pb.ContainerConfig) (*oci.Container, error)

(1)、调用netNsPath := sb.netNsPath()

(2)、当netNsPath为""时,the sandbox does not have a permanent namespace, it's on the host,则调用netNsPath = fmt.Sprintf("/proc/%d/ns/net", podInfraState.Pid)

(3)、调用specgen.AddOrReplaceLinuxNamespace("network", netNsPath)

 

-------------------------------------------------------------------------- cni 初始化 --------------------------------------------------------------------------------------------------

cniNetworkPlugin的数据结构如下所示:

type cniNetworkPlugin struct {loNetwork    *cniNetworksync.RWMutexdefaultNetwork  *cniNetworknsenterPath    stringpluginDir       stringvendorCNIDirPrefix  string
}

 

cniNetwork的结构如下所示:

type cniNetwork struct {name      stringNetworkConfig  *libcni.NetworkConfigCNIConfig    libcni.CNI
}

 

NetworkConfig的结构如下所示:

type NetworkConfig struct {Network  *types.NetConfBytes    []byte
}

  

 

CNI的结构如下所示:

type CNI interface {AddNetwork(net *NetworkConfig, rt *RuntimeConf) (*types.Result, error)DelNetwork(net *NetworkConfig, rt *RuntimeConf) error}

  

常量如下所示:

const (DefaultInterfaceName = "eth0"CNIPluginName     = "cni"DefaultNetDir      = "/etc/cni/net.d"DefaultCNIDir      = "/opt/cni/bin"VendorCNIDirTemplate = "%s/opt/%s/bin")

  

2、// cri-o/vendor/src/github.com/rajatchopra/ocicni.go

server的netPlugin字段初始化为:netPlugin, err := ocicni.InitCNI("")

func InitCNI(pluginDir string) (CNIPlugin, error)

(1)、首先调用plugin := probeNetworkPluginsWithVendorCNIDirPrefix(pluginDir, ""),返回一个默认的cniNetworkPlugin{}

(2)、plugin.nsenterPath, err = exec.LookPath(”nsenter“)

(3)、检查默认的network是否存在,如果不存在则停止CNI的查找,直接返回一个noop plugin,调用_, err = getDefaultCNINetwork(plugin.pluginDir, plugin.vendorCNIDirPrefix)

(4)、当有默认的network存在时,周期性地从pluginDir中读取网络配置的更新。即生成一个goroutine,每隔10s调用一次plugin.syncNetworkConfig()

 

3、// cri-o/vendor/src/github.com/rajatchopra/ocicni.go

func probeNetworkPluginsWithVendorCNIDirPrefix(pluginDir, vendorCNIDirPrefix string) (*cniNetworkPlugin)

配置获得 plugin := &cniNetworkPlugin {

  defaultNetwork:    nil,

  loNetwork:      getLoNetwork(vendorCNIDirPrefix),

  pluginDir:        pluginDir,

  vendorCNIDirPrefix:  vendorCNIDirPrefix,

}

最后调用plugin.syncNetworkConfig()并返回 return plugin,其中syncNetworkConfig首先调用network, err := getDefaultCNINetwork(plugin.pluginDir, plugin.vendorCNIDirPrefix),然后调用plugin.setDefaultNetwork(network)设置为plugin.defaultNetwork为network

 

4、//cri-o/vendor/src/github.com/rajachopra/ocicni.go

func getLoNetwork(vendorDirPrefix string) *cniNetwork

(1)、手动添加loConfig, err := libcni.ConfFromBytes([]byte(`{"cniVersion": "0.1.0", "name": "cni-loopback", "type": "loopback"}`))

(2)、调用cninet := &libcni.CNIConfig{Path: []string{vendorCNIDir(vendorDirPrefix, loConfig.Network.Type), DefaultCNIDir}}

并返回 reutrn loNetwork := &cniNetwork{name: "lo", NetworkConfig: loConfig, CNIConfig: cninet}

 

5、//cri-o/vendor/src/github.com/rajatchopra/ocicni.go

func getDefaultCNINetwork(pluginDir, vendorCNIDirPrefix string) (*cniNetwork, error)

(1)、当pluginDir为空时,将pluginDir设置为DefaultNetDir,为/etc/cni/net.d

(2)、调用files, err := libcni.ConfFiles(pluginDir),加载配置文件

(3)、若files不为空,则调用for循环,for _, confFile := range files

  对于confFile,先调用conf, err := libcni.ConfFromFile(confFile)

  再调用vendorDir := vendorCNIDir(vendorCNIDirPrefix, conf.Network.Type),cninet := &libcni.CNIConfig{Path: []string{DefaultCNIDir, vendorDir }}

  其中vendorDir为"/opt/pluginType/bin"

  最后,返回 return network := &cniNetwork{name: conf.Network.Name, NetworkConfig: conf, CNIConfig: cninet}

 

6、// cri-o/vendor/src/github.com/rajatchopra/ocicni.go

func vendorCNIDir(prefix, pluginType string) string

  该函数仅仅return fmt.Sprintf(VendorCNIDirTemplate, prefix, pluginType)

 

7、// cri-o/vendor/src/github.com/rajatchopra/ocicni.go

func (plugin *cniNetworkPlugin)  syncNetworkConfig()

(1)、调用network, err := getDefaultCNINetwork(plugin.pluginDir, plugin.vendorCNIDirPrefix)

(2)、再调用plugin.setDefaultNetwork(network)

 

8、// cri-o/vendor/src/github.com/rajatchopra/ocicni.go

func (plugin *cniNetworkPlugin) setDefaultNetwork(n *cniNetwork)

调用plugin.Lock(),再让plugin.defaultNetwork = n

 

------------------------------------------------------------------------------ 设置Pod的network ---------------------------------------------------------------------------------------------------

6、//cni-o/vendor/src/github.com/rajatchopra/ocicni.go

func (plugin *cniNetworkPlugin) SetUpPod(netnsPath string, namespace string, name string, id string) error

(1)、调用 plugin.checkInitialized(),判断plugin.defaultNetwork是否为空,若为空,返回错误

(2)、分别调用plugin.loNetwork.addToNetwork(name, namespace, id, netnsPath)和plugin.getDefaultNetwork().addToNetwork(name, namespace, id, netnsPath)

 

7、//cni-o/vendor/src/github.com/rajatchopra/ocicni.go

func (network *cniNetwork) addToNetwork(podName string, podNamespace string, podInfraContainerID string, podNetnsPath string) (*cnitypes.Result, error)

(1)、调用rt, err := buildCNIRuntimeConf(podName, podNamespace, podInfaraContainerID, podNetnsPath)

(2)、再调用netconf, cninet := network.NetworkConfig, network.CNIConfig,最后调用res, err := cninet.AddNetwork(netconf, rt)

(3) 、返回return res, nil

 

8、//cni-o/vendor/src/github.com/rajatchopra/ocicni.go

func buildCNIRuntimeConf(podName string, podNs string, podInfraContainerID string, podNetnsPath string) (*libcni.RuntimeConf, error)

该函数只是简单地填充libcni.RuntimeConf并返回

rt := &libcni.RuntimeConf {

  ContainerID:    podInfraContainerID,

  NetNS:       podNetnsPath,

  IfName:      DefaultInterfaceName,

  Args:     [][2]string {

    {"IgnoreUnknown", "1"},

    {"K8S_POD_NAMESPACE", podNs},

    {"K8S_POD_NAME", podName},

    {"K8S_POD_INFRA_CONTAINER_ID", podInfraContainerID},

  }

}

 

--------------------------------------------------------------sandboxNetNs相关-------------------------------------------------------

sandboxNetNs结构如下所示:

type sandboxNetNs struct {sync.Mutexns    ns.NetNSsymlink  *os.Fileclosed  boolrestored bool
}

// cri-o/server/sandbox.go

1、func (s *sandbox) netNsCreate() error

(1)、调用netNS, err := ns.NewNS()

(2)、创建s.netns = &sandboxNetNs{ns: netNS, closed: false}

(3)、最后调用s.netns.symlinkCreate(s.name)

 

// cri-o/server/sandbox.go

2、func (ns *sandboxNetNs) symlinkCreate(name string) error

(1)、随机产生一个四位的byte b,nsName := fmt.Sprintf("%s-%x", name, b)以及symlinkPath := filepath.Join(nsRunDir, nsName)  ---> nsRunDir默认为"/var/run/netns"

(2)、调用os.Symlink(ns.ns.Path(), symlinkPath)

(3)、fd, err := os.Open(symlinkPath),并且ns.symlink = fd

 

3、、func (s *sandbox) netNsPath() string

(1)、当s.netns == nil时,返回 ""

(2)、否则return s.netns.symlink.Name(),即地址"/var/run/netns/$nsName"

 

转载于:https://www.cnblogs.com/YaoDD/p/6043418.html

版权声明:本站所有资料均为网友推荐收集整理而来,仅供学习和研究交流使用。

原文链接:https://hbdhgg.com/1/37567.html

发表评论:

本站为非赢利网站,部分文章来源或改编自互联网及其他公众平台,主要目的在于分享信息,版权归原作者所有,内容仅供读者参考,如有侵权请联系我们删除!

Copyright © 2022 匯編語言學習筆記 Inc. 保留所有权利。

底部版权信息