Two ways to retrieve process id while startup a process via Windows command line

一个特殊的机缘, 需要通过cmd.exe启动一个进程, 并且获取该启动后进程的Process ID, 搜罗到两种方法:

1. 通过wmic process call create

  • wmic如果创建进程成功,将返回一个ReturnValue为0类JSON结构的输出, 从中获取ProcessId:
C:\app>cmd.exe /C wmic process call create "c:\app\cluster\GatewayServer.exe start -id 6", "c:\app"
Executing (Win32_Process)->Create()
Method execution successful.
Out Parameters:
instance of __PARAMETERS
{
        ProcessId = 19420;
        ReturnValue = 0;
};
  • wmic如果创建进程失败, 将返回一个ReturnValue非0的输出:
C:\app>wmic process call create "cluster\GatewayServer.exe start -id 6", "c:\app"
Executing (Win32_Process)->Create()
Method execution successful.
Out Parameters:
instance of __PARAMETERS
{
        ReturnValue = 9;
};

但wmic有一个很大的问题: 不使用当前用户上下文和系统的环境变量.

2. 通过powershell的Start-Process启动进程, 然后取Start-Process返回对像的Id属于得到Process ID:

C:\app>cmd.exe /C powershell -Command "try{$app = Start-Process -PassThru -FilePath \"cluster\GatewayServer.exe\" -WorkingDirectory \"C:\app\" -ArgumentList \"start -id 5\";echo $app.Id} catch {throw}"

How to install specific hotfix on Windows Server

Windows容器环境有个特点, Host与Container的OS Builder Number必须匹配, 有点场景甚至要求Revision Number匹配, 所以经常要为K8s Node安装指定Revision 的hotfix,  用powershell在线安装时下载过程缓慢而不可控, 体验最好的路径还是直接找到相应Revision Number的msu安装包,直接安装:

1. 从Windows Update History网站找到版本对应的KB. 如: Windows Server 1809 OS Build 10.0.17763.1158
https://support.microsoft.com/en-us/help/4549949

2. 在Windows Update Catelog按KB搜索: https://www.catalog.update.microsoft.com/
找到相应的下载包. 如17763.1158对应的KB4549949: https://www.catalog.update.microsoft.com/Search.aspx?q=KB4549949

3. 下载msu安装包后使用wusa指令安装即可:

wusa windows10.0-kb4549949-x64_90e8805e69944530b8d4d4877c7609b9a9e68d81.msu

附:

为了防止Windows Node版本变更, 还要关闭Windows Auto Update, 防止Node OS自己变更版本:

a). 查看Auto Update 状态:

%systemroot%\system32\Cscript %systemroot%\system32\scregedit.wsf /AU /v

b). 禁用 Windows Auto Update:

Net stop wuauserv 
%systemroot%\system32\Cscript %systemroot%\system32\scregedit.wsf /AU 1 
Net start wuauserv

PS: 可使用wmic qfe list查看已安装的hostfix

Reference:
https://docs.microsoft.com/en-us/windows-server/administration/server-core/server-core-servicing

For Windows Container, you need to set –image-pull-progress-deadline for kubelet

Windows镜像动则几个G, 基于Windows Server Core的镜像5~10G, Windows节点上的kubelet在下载镜像的时候经常会cancel掉:

Failed to pull image "XXX": rpc error: code = Unknown desc = context canceled

 

造成这个问题的原因是因为默认的image pulling progress deadline是1分钟, 如果1分钟内镜像下载没有任何进度更新, 下载动作就会取消, 比较大的镜像就无法成功下载. 见官方文档:

If no pulling progress is made before this deadline, the image pulling will be cancelled. This docker-specific flag only works when container-runtime is set to docker. (default 1m0s)

 

解决方法是为kubelet配置–image-pull-progress-deadline参数, 比如指定为30分钟:

"c:/k/kubelet.exe ... --image-pull-progress-deadline=30m"

 

对于Windows服务, 使用sc指令修改kubelet的binPath:

sc config kubelet binPath= " --image-pull-progress-deadline=30m

然后重启kubelet及依赖服务:

sc stop kubeproxy && sc stop kubelet && sc start kubelet && sc start kubeproxy && sc query kubelet && sc query kubeproxy

Refer to: https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/

 

DOS中通过chcp修改console code page

UTF-8格式的日志文件, DOS中通过type命令查看时显示乱码:

C:\app\Manager\Log\RootManagerServer>type Root_20200527_135730_Trace.html
1590559051 0 0 Trace 0 0 Bind Port:Listen:8800<br>                                                                                                                                                                                                                
1590559051 0 0 Trace 0 0 鏈嶅姟鍣ㄧ洃鍚鍙?8800<br>                                                                                                                                                                                                              
1590559051 0 0 Trace 0 0 Bind Port:Listen:9800<br>                                                                                                                                                                                                                
1590559051 0 0 Trace 0 0 瀹㈡埛绔洃鍚鍙?9800<br>

通过chcp命令修改code page为65001后显示正常:

C:\app\Manager\Log\RootManagerServer>type Root_20200527_135730_Trace.html
1590559051 0 0 Trace 0 0 Bind Port:Listen:8800<br>                                                                                                                                                                                                                
1590559051 0 0 Trace 0 0 服务器监听端口:8800<br>                                                                                                                                                                                                                  
1590559051 0 0 Trace 0 0 Bind Port:Listen:9800<br>                                                                                                                                                                                                                
1590559051 0 0 Trace 0 0 客户端监听端口:9800<br>

 

DOS命令行中支持的Code Page:

代码页 国家/地区或语言
437 United States
850 多语言 (拉丁文我)
852 西里尔语 (俄语)
855 西里尔语 (俄语)
857 土耳其语
860 葡萄牙语
861 冰岛语
863 加拿大法语
865 北欧
866 俄语
869 现代希腊语
65001 UTF-8

chcp文档: https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/chcp

 

Using diskpart to create partition in Windows

DISKPART
DISKPART> list disk
  磁盘 ###  状态           大小     可用     Dyn  Gpt
  --------  -------------  -------  -------  ---  ---
  磁盘 0    联机               50 GB      0 B
  磁盘 1    脱机               50 GB    50 GB
DISKPART> select disk 1
DISKPART> online disk 1
DISKPART> list disk

  磁盘 ###  状态           大小     可用     Dyn  Gpt
  --------  -------------  -------  -------  ---  ---
  磁盘 0    联机               50 GB      0 B
* 磁盘 1    联机               50 GB    50 GB
DISKPART> create partition primary

DiskPart 遇到错误: 介质受写入保护。
有关详细信息,请参阅系统事件日志。
DISKPART> ATTRIBUTES DISK CLEAR READONLY

已成功清除磁盘属性。

DISKPART> create partition primary

DiskPart 成功地创建了指定分区。
DISKPART> list disk

  磁盘 ###  状态           大小     可用     Dyn  Gpt
  --------  -------------  -------  -------  ---  ---
  磁盘 0    联机               50 GB      0 B
* 磁盘 1    联机               50 GB      0 B

DISKPART> list partition

  分区 ###       类型              大小     偏移量
  -------------  ----------------  -------  -------
* 分区      1    主要                  49 GB  1024 KB
DISKPART> FORMAT FS=NTFS QUICK

  100 百分比已完成

DiskPart 成功格式化该卷。
DISKPART> list volume

  卷 ###      LTR  标签         FS     类型        大小     状态       信息
  ----------  ---  -----------  -----  ----------  -------  ---------  --------
  卷     0     D                       DVD-ROM         0 B  无介质
  卷     1         系统保留         NTFS   磁盘分区         549 MB  正常         系统
  卷     2     C                NTFS   磁盘分区          49 GB  正常         启动
* 卷     3                      NTFS   磁盘分区          49 GB  正常
DISKPART> select volume 0

卷 0 是所选卷。

DISKPART> assign letter=e

DiskPart 成功地分配了驱动器号或装载点。

DISKPART> select volume 3

卷 3 是所选卷。

DISKPART> assign letter=d

DiskPart 成功地分配了驱动器号或装载点。

DISKPART> list volume

  卷 ###      LTR  标签         FS     类型        大小     状态       信息
  ----------  ---  -----------  -----  ----------  -------  ---------  --------
  卷     0     E                       DVD-ROM         0 B  无介质
  卷     1         系统保留         NTFS   磁盘分区         549 MB  正常         系统
  卷     2     C                NTFS   磁盘分区          49 GB  正常         启动
* 卷     3     D                NTFS   磁盘分区          49 GB  正常

Implementing Graceful Shutdown in Windows Container

Kubernetes Linux Pod中,当通过kubectl删除一个Pod或rolling update一个Pod时, 每Terminating的Pod中的每个Container中PID为1的进程会收到SIGTERM信号, 通知进程进行资源回收并准备退出. 如果在Pod spec.terminationGracePeriodSeconds指定的时间周期内进程没有退出, 则Kubernetes接着会发出SIGKILL信号KILL这个进程。

通过 kubectl delete –force –grace-period=0 … 的效果等同于直接发SIGKILL信号.

但SIGTERM和SIGKILL方式在Windows Container中并不工作, 目前Windows Container的表现是接收到Terminating指令5秒后直接终止。。。

参见:https://v1-18.docs.kubernetes.io/docs/setup/production-environment/windows/intro-windows-in-kubernetes/#v1-pod

  • V1.Pod.terminationGracePeriodSeconds – this is not fully implemented in Docker on Windows, see: reference. The behavior today is that the ENTRYPOINT process is sent CTRL_SHUTDOWN_EVENT, then Windows waits 5 seconds by default, and finally shuts down all processes using the normal Windows shutdown behavior. The 5 second default is actually in the Windows registry inside the container, so it can be overridden when the container is built.

基于社区的讨论结果及多次尝试, 目前Windows容器中行之有效的Graceful Shutdown方法是:

1. Build docker image时通过修改注册表延长等待时间

...
RUN reg add hklm\system\currentcontrolset\services\cexecsvc /v ProcessShutdownTimeoutSeconds /t REG_DWORD /d 300 && \
    reg add hklm\system\currentcontrolset\control /v WaitToKillServiceTimeout /t REG_SZ /d 300000 /f
...

上面两个注册表位置, 第1个单位为秒, 第2个为毫秒

2. 在应用程序中注册kernel32.dll中的SetConsoleCtrlHandler函数捕获CTRL_SHUTDOWN_EVENT事件, 进行资源回收

以一个.net framework 的Console App为例说明用法:

using System;
using System.Runtime.InteropServices;
using System.Threading;

namespace Q1.Foundation.SocketServer
{
    class Program
    {
        internal delegate bool HandlerRoutine(CtrlType CtrlType);
        private static HandlerRoutine ctrlTypeHandlerRoutine = new HandlerRoutine(ConsoleCtrlHandler);

        private static bool cancelled = false;
        private static bool cleanupCompleted = false;

        internal enum CtrlType
        {
            CTRL_C_EVENT = 0,
            CTRL_BREAK_EVENT = 1,
            CTRL_CLOSE_EVENT = 2,
            CTRL_LOGOFF_EVENT = 5,
            CTRL_SHUTDOWN_EVENT = 6
        }

        [DllImport("Kernel32")]
        internal static extern bool SetConsoleCtrlHandler(HandlerRoutine handler, bool add);

        static void Main()
        {
            var result = SetConsoleCtrlHandler(handlerRoutine, true);

            // INITIAL AND START APP HERE

            while (true)
            {
                if (cancelled) break;
            }

            // DO CLEANUP HERE
            ...
            cleanupCompleted = true;
        }

        private static bool ConsoleCtrlHandler(CtrlType type)
        {
            cancelled = true;

            while (!cleanupCompleted)
            {
                // Watting for clean-up to be completed...
            }

            return true;
        }
    }
}

代码解释:

  • 引入Kernel32并声明extern函数SetConsoleCtrlHandler
  • 创建static的HandlerRoutine.
  • 调用SetConsoleCtrlHandler注册处理函数进行事件捕获
  • 捕获后在HandlerRoutine应用程序中进行资源清理
  • 清理完成后在HandlerRoutine中返回true允许应用程序退出

上述两个步骤即完成了Graceful Shutdown.

 

需要注意的点是:

1. 传统.net Console App中的事件捕获( 比如: Console.CancelKeyPressSystemEvents.SessionEnding )在容器中都不会生效,AppDomain.CurrentDomain.ProcessExit的触发时间又太晚, 只有SetConsoleCtrlHandler可行. 更多的尝试代码请参见: https://github.com/moby/moby/issues/25982#issuecomment-250490552

2. 要防止程序退出前HandlerRoutine实例被回收, 所以上面示例中使用了static的HandlerRoutine. 这点很重要, 如果HandlerRoutine在应用程序未结束的时候被回收掉, 就会引发错误, 看如下代码:

static void Main()
{
    // Initialize here

    ...
    using
    {
        var sysEventHandler = new HandlerRoutine(type =>
        {
            cancelled = true;

            while (!cleanCompleted)
            {
                // Watting for clean-up to be completed...
            }

            return true;
        });
		
        var sysEventSetResult = SetConsoleCtrlHandler(sysEventHandler, true);
        ...
    }
    ...

    // Cleanup here
}

在应用程序退出前, HandlerRoutine实例已经被回收掉了,在CTRL_SHUTDOWN_EVENT 被触发时就会引发NullReferenceException, 具体错误信息如下:

Managed Debugging Assistant 'CallbackOnCollectedDelegate':
A callback was made on a garbage collected delegate of type 'Program+HandlerRoutine::Invoke'. This may cause application crashes, corruption and data loss. When passing delegates to unmanaged code, they must be kept alive by the managed application until it is guaranteed that they will never be called.

类似场景: CallbackOnCollectedDelegate was detected

 

关于SetConsoleCtrlHandler的使用参考:

SetConsoleCtrlHandler function

HandlerRoutine callback function

 

最后, 如果要处理的应用程序类型不是Console App, 而是图形化的界面应用,则要处理的消息应该是WM_QUERYENDSESSION, 参见文档:

https://docs.microsoft.com/en-us/windows/console/setconsolectrlhandler#remarks

WM_QUERYENDSESSION message

Add File Extension to Windows IIS Container during image build

Let’s say: we need to add json file extension to the containerized IIS.

Dockerfile:

FROM {imageRegistry}/mcr.microsoft.com/dotnet/framework/aspnet:4.8-20200114-windowsservercore-ltsc2019
COPY . /inetpub/wwwroot
WORKDIR /inetpub/wwwroot

RUN C:\windows\system32\inetsrv\appcmd.exe set config "Default Web Site" -section:system.webServer/security/requestFiltering /+"fileExtensions.[fileExtension='json',allowed='True']"

ENV ASPNETCORE_URLS http://+:80
EXPOSE 80/tcp

An error occurs during build docker image:

Step 1/6 : FROM repo.q1lan.k8s:9999/mcr.microsoft.com/dotnet/framework/aspnet:4.8-20200114-windowsservercore-ltsc2019
 ---> a5bc996f06b3
Step 2/6 : COPY . /inetpub/wwwroot
 ---> bdb9536e506a
Step 3/6 : WORKDIR /inetpub/wwwroot
 ---> Running in f7666a9ffd0b
Removing intermediate container f7666a9ffd0b
 ---> c9fe76854f6c
Step 4/6 : RUN C:\windows\system32\inetsrv\appcmd.exe set config "Default Web Site" -section:system.webServer/security/requestFiltering /+"fileExtensions.[fileExtension='json',allowed='True']"
 ---> Running in 1c74d16420c2
Failed to process input: The parameter 'Web' must begin with a / or - (HRESULT=80070057).

Try to escape all double-quotes in Dockerfile:

RUN C:\windows\system32\inetsrv\appcmd.exe set config \"Default Web Site\" -section:system.webServer/security/requestFiltering /+\"fileExtensions.[fileExtension='json',allowed='True']\"

It works like a charm:

Step 1/6 : FROM repo.q1lan.k8s:9999/mcr.microsoft.com/dotnet/framework/aspnet:4.8-20200114-windowsservercore-ltsc2019
 ---> a5bc996f06b3
Step 2/6 : COPY . /inetpub/wwwroot
 ---> 646bbf3d5def
Step 3/6 : WORKDIR /inetpub/wwwroot
 ---> Running in 584471c0524a
Removing intermediate container 584471c0524a
 ---> 54f6a3ade821
Step 4/6 : RUN C:\windows\system32\inetsrv\appcmd.exe set config \"Default Web Site\" -section:system.webServer/security/requestFiltering /+\"fileExtensions.[fileExtension='json',allowed='True']\"
 ---> Running in f84c38da656a
Applied configuration changes to section "system.webServer/security/requestFiltering" for "MACHINE/WEBROOT/APPHOST/Default Web Site" at configuration commit path "MACHINE/WEBROOT/APPHOST/Default Web Site"
Removing intermediate container f84c38da656a
 ---> 7dfffe2d9813
Step 5/6 : ENV ASPNETCORE_URLS http://+:80
 ---> Running in dff81c8282f1
Removing intermediate container dff81c8282f1
 ---> cbd697556dd7
Step 6/6 : EXPOSE 80/tcp
 ---> Running in d10903bec188
Removing intermediate container d10903bec188
...

Describe Kubelet Service Parameters on Azure Windows node

Query Kubelet service

Managed by nssm

C:\k>sc qc kubelet
[SC] QueryServiceConfig SUCCESS

SERVICE_NAME: kubelet
        TYPE               : 10  WIN32_OWN_PROCESS
        START_TYPE         : 2   AUTO_START
        ERROR_CONTROL      : 1   NORMAL
        BINARY_PATH_NAME   : C:\k\nssm.exe
        LOAD_ORDER_GROUP   :
        TAG                : 0
        DISPLAY_NAME       : Kubelet
        DEPENDENCIES       : docker
        SERVICE_START_NAME : LocalSystem

Query kubelet AppParameters by nssm

C:\k>nssm get kubelet Application
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe

C:\k>nssm get kubelet AppParameters
c:\k\kubeletstart.ps1

Powershell scripts to start kubelet

$global:MasterIP = "q1game-q1game-6adca6-e3314a8c.hcp.westus2.azmk8s.io"
$global:KubeDnsSearchPath = "svc.cluster.local"
$global:KubeDnsServiceIp = "10.0.0.10"
$global:MasterSubnet = "10.240.0.0/16"
$global:KubeClusterCIDR = "10.240.0.0/16"
$global:KubeServiceCIDR = "10.0.0.0/16"
$global:KubeBinariesVersion = "1.17.3"
$global:CNIPath = "c:\k\cni"
$global:NetworkMode = "L2Bridge"
$global:ExternalNetwork = "ext"
$global:CNIConfig = "c:\k\cni\config\$global:NetworkMode.conf"
$global:HNSModule = "c:\k\hns.psm1"
$global:VolumePluginDir = "c:\k\volumeplugins"
$global:NetworkPlugin="azure"
$global:KubeletNodeLabels="kubernetes.azure.com/role=agent,agentpool=q1win,storageprofile=managed,storagetier=Premium_LRS,kubernetes.azure.com/cluster=MC_q1game_q1game_westus2"
Write-Host "NetworkPlugin azure, starting kubelet."

# Turn off Firewall to enable pods to talk to service endpoints. (Kubelet should eventually do this)
netsh advfirewall set allprofiles state off
# startup the service

# Find if network created by CNI exists, if yes, remove it
# This is required to keep the network non-persistent behavior
# Going forward, this would be done by HNS automatically during restart of the node

$hnsNetwork = Get-HnsNetwork | ? Name -EQ azure
if ($hnsNetwork)
{
    # Cleanup all containers
    docker ps -q | foreach {docker rm $_ -f}

    Write-Host "Cleaning up old HNS network found"
    Remove-HnsNetwork $hnsNetwork
    # Kill all cni instances & stale data left by cni
    # Cleanup all files related to cni
    taskkill /IM azure-vnet.exe /f
    taskkill /IM azure-vnet-ipam.exe /f
    $cnijson = [io.path]::Combine("c:\k", "azure-vnet-ipam.json")
    if ((Test-Path $cnijson))
    {
        Remove-Item $cnijson
    }
    $cnilock = [io.path]::Combine("c:\k", "azure-vnet-ipam.json.lock")
    if ((Test-Path $cnilock))
    {
        Remove-Item $cnilock
    }

    $cnijson = [io.path]::Combine("c:\k", "azure-vnet.json")
    if ((Test-Path $cnijson))
    {
        Remove-Item $cnijson
    }
    $cnilock = [io.path]::Combine("c:\k", "azure-vnet.json.lock")
    if ((Test-Path $cnilock))
    {
        Remove-Item $cnilock
    }
}

# Restart Kubeproxy, which would wait, until the network is created
# This was fixed in 1.15, workaround still needed for 1.14 https://github.com/kubernetes/kubernetes/pull/78612
Restart-Service Kubeproxy

$env:AZURE_ENVIRONMENT_FILEPATH="c:\k\azurestackcloud.json"

c:\k\kubelet.exe --address=0.0.0.0 --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --azure-container-registry-config=c:\k\azure.json --cgroups-per-qos=false --client-ca-file=c:\k\ca.crt --cloud-config=c:\k\azure.json --cloud-provider=azure --cluster-dns=10.0.0.10 --cluster-domain=cluster.local --dynamic-config-dir=/var/lib/kubelet --enforce-node-allocatable="" --event-qps=0 --eviction-hard="" --feature-gates=RotateKubeletServerCertificate=true --hairpin-mode=promiscuous-bridge --image-gc-high-threshold=85 --image-gc-low-threshold=80 --image-pull-progress-deadline=20m --keep-terminated-pod-volumes=false --kube-reserved=cpu=100m,memory=1843Mi --kubeconfig=c:\k\config --max-pods=30 --network-plugin=cni --node-status-update-frequency=10s --non-masquerade-cidr=0.0.0.0/0 --pod-infra-container-image=kubletwin/pause --pod-max-pids=-1 --protect-kernel-defaults=true --read-only-port=0 --resolv-conf="" --rotate-certificates=false --streaming-connection-idle-timeout=4h --system-reserved=memory=2Gi --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256 --node-labels=$global:KubeletNodeLabels --volume-plugin-dir=$global:VolumePluginDir --cni-bin-dir=c:\k\azurecni\bin --cni-conf-dir=c:\k\azurecni\netconf

Enable Hyper-V Isolation by modify kubelet parameters

1. Modify c:\k\kubeletstart.ps1 to add parameter to kubelet

--feature-gates="XXX=true,HyperVContainer=true"

2. Restart kubelet
C:\k>nssm restart kubelet
Kubelet: STOP: A stop control has been sent to a service that other running services are dependent on.

C:\k>sc queryex kubelet

SERVICE_NAME: kubelet
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 4  RUNNING
                                (STOPPABLE, PAUSABLE, ACCEPTS_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x0
        PID                : 4044
        FLAGS              :

C:\k>taskkill /PID 4044 /F

C:\k>sc start kubelet

Restart the Windows node if necessary

Run Windows container with Hyper-V isolation mode in Kubernetes

Windows Container有两种隔离运行模式Hyper-V和Process, 参见:Isolation Modes

两种模式下的host的OS版本与containter的OS版本存在兼容性又不相同,参见:Windows container version compatibility

很明显Hyper-V模式的兼容性要比Process模式要好,向下兼容,也就是高版本的host OS可以运行低版本的container OS, 反之不行;

而Process模式下Windows Server中则要求host OS与container OS的版本完全相同, Windows 10中则不支持Process模式.

 

某一天,我想在Kubernetes Windows 节点中以Hyper-V模式运行Container, 于是乎发现1.17的文档中写道:

Note: In this document, when we talk about Windows containers we mean Windows containers with process isolation. Windows containers with Hyper-V isolation is planned for a future release.

不甘心又google了一下,发现:

1. 有人提了bug, 已经被修复了: https://github.com/kubernetes/kubernetes/issues/58750
2. 代码也merge了: https://github.com/kubernetes/kubernetes/pull/58751
3. 有人在测试过程中遇到问题,也解决了: https://github.com/kubernetes/kubernetes/issues/62812

但我测试的过程中却提示:

Error response from daemon: CreateComputeSystem test: The container operating system does not match the host operating system.

我的环境:

Kubernetes Ver: 1.14.8

Kubernetes Node OS Ver: Windows Server Datacenter 10.0.17763.504, 属于1809的版本

Container Base Image: windowsservercore-1709

Deployment yaml:

apiVersion: apps/v1beta2
kind: Deployment
metadata:
  labels:
    app: test
  name: test
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: test
  template:
    metadata:
      annotations:
        experimental.windows.kubernetes.io/isolation-type: hyperv
      labels:
        app: test
...

 

然后对比了下github别人试成功的deployment yaml, 发现人家用的是apps/v1

apiVersion: apps/v1
kind: Deployment
metadata:
  name: whoami
  labels:
    app: whoami
spec:
  ...

 

目前在k8s中启用hyperv isolation的三个条件:

  1. kubelet 启用参数:  –feature-gates=HyperVContainer=true
  2. Pod/Deployment apiVersion: apps/v1
  3. spec.template.metadata.annotations[].experimental.windows.kubernetes.io/isolation-type:hyperv

参见: https://kubernetes.io/docs/setup/production-environment/windows/intro-windows-in-kubernetes/#hyper-v-isolation

 

目前我的云提供商给的kubernetes 1.14.8又不支持apps/v1 …

于是乎,我要么等提供商升级kubernetes,要么自己升级container OS跟kubernetes node OS一样…

Win10打开休眠模式

升级正式版win10以后,发现竟然没有休眠选项,从电源管理器里面也没有找到,有时候有些重要的工作,希望第二天打开直接在第一天的状态,不用重新打开各种文件,而电脑又没必要开一整夜,于是必需要打开休眠选项。

以管理员权限进入命令行
输入命令:

powercfg /a

查看电脑支持的睡眠模式,是否休眠未打开
如果未打开,然后输入

powercfg /h on

最后再用命令:powercfg /a 查看是否打开了休眠。
如果已经打开了,就可以去电源管理器中去设置了。

在开始菜单上右键,选择电源选项,
  选择“选择电源按钮的功能”,
  单击“更改当前不可用的设置,
  单击“更改当前不可用的设置”,
  选择需要的设置“休眠”,
  最后 “保存修改”
这样打开开始菜单,选择“电源”现在可以休眠了
———————
Refer:https://blog.csdn.net/saindy5828/article/details/72857332