{"id":4311,"date":"2018-12-31T14:13:14","date_gmt":"2018-12-31T14:13:14","guid":{"rendered":"http:\/\/howk.de\/w1\/blog-health-checking-grpc-servers-on-kubernetes\/"},"modified":"2018-12-31T14:13:14","modified_gmt":"2018-12-31T14:13:14","slug":"blog-health-checking-grpc-servers-on-kubernetes","status":"publish","type":"post","link":"https:\/\/howk.de\/?p=4311","title":{"rendered":"Blog: Health checking gRPC servers on Kubernetes"},"content":{"rendered":"<p><strong>Author<\/strong>: <a href=\"https:\/\/twitter.com\/ahmetb\" target=\"_blank\">Ahmet Alp Balkan<\/a> (Google)<\/p>\n<p><a href=\"https:\/\/grpc.io\" target=\"_blank\">gRPC<\/a> is on its way to becoming the lingua franca for<br \/>\ncommunication between cloud-native microservices. If you are deploying gRPC<br \/>\napplications to Kubernetes today, you may be wondering about the best way to<br \/>\nconfigure health checks. In this article, we will talk about<br \/>\n<a href=\"https:\/\/github.com\/grpc-ecosystem\/grpc-health-probe\/\" target=\"_blank\">grpc-health-probe<\/a>, a<br \/>\nKubernetes-native way to health check gRPC apps.<\/p>\n<p>If you&rsquo;re unfamiliar, Kubernetes <a href=\"https:\/\/kubernetes.io\/docs\/tasks\/configure-pod-container\/configure-liveness-readiness-probes\/\">health<br \/>\nchecks<\/a><br \/>\n(liveness and readiness probes) is what&rsquo;s keeping your applications available<br \/>\nwhile you&rsquo;re sleeping. They detect unresponsive pods, mark them unhealthy, and<br \/>\ncause these pods to be restarted or rescheduled.<\/p>\n<p>Kubernetes <a href=\"https:\/\/github.com\/kubernetes\/kubernetes\/issues\/21493\" target=\"_blank\">does not<br \/>\nsupport<\/a> gRPC health<br \/>\nchecks natively. This leaves the gRPC developers with the following three<br \/>\napproaches when they deploy to Kubernetes:<\/p>\n<p><a href=\"https:\/\/kubernetes.io\/images\/blog\/2019-09-30-health-checking-grpc\/options.png\"><img decoding=\"async\" src=\"https:\/\/kubernetes.io\/images\/blog\/2019-09-30-health-checking-grpc\/options.png\" alt=\"options for health checking grpc on kubernetes today\" \/><\/a><\/p>\n<ol>\n<li><strong>httpGet probe:<\/strong> Cannot be natively used with gRPC. You need to refactor<br \/>\nyour app to serve both gRPC and HTTP\/1.1 protocols (on different port<br \/>\nnumbers).<\/li>\n<li><strong>tcpSocket probe:<\/strong> Opening a socket to gRPC server is not meaningful,<br \/>\nsince it cannot read the response body.<\/li>\n<li><strong>exec probe:<\/strong> This invokes a program in a container&rsquo;s ecosystem<br \/>\nperiodically. In the case of gRPC, this means you implement a health RPC<br \/>\nyourself, then write and ship a client tool with your container.<\/li>\n<\/ol>\n<p>Can we do better? Absolutely.<\/p>\n<h2 id=\"introducing-grpc-health-probe\">Introducing \u201cgrpc-health-probe\u201d<\/h2>\n<p>To standardize the &ldquo;exec probe&rdquo; approach mentioned above, we need:<\/p>\n<ul>\n<li>a <strong>standard<\/strong> health check &ldquo;protocol&rdquo; that can be implemented in any gRPC<br \/>\nserver easily.<\/li>\n<li>a <strong>standard<\/strong> health check &ldquo;tool&rdquo; that can query the health protocol easily.<\/li>\n<\/ul>\n<p>Thankfully, gRPC has a <a href=\"https:\/\/github.com\/grpc\/grpc\/blob\/v1.15.0\/doc\/health-checking.md\" target=\"_blank\">standard health checking<br \/>\nprotocol<\/a>. It<br \/>\ncan be used easily from any language. Generated code and the utilities for<br \/>\nsetting the health status are shipped in nearly all language implementations of<br \/>\ngRPC.<\/p>\n<p>If you<br \/>\n<a href=\"https:\/\/github.com\/grpc\/grpc\/blob\/v1.15.0\/src\/proto\/grpc\/health\/v1\/health.proto\" target=\"_blank\">implement<\/a><br \/>\nthis health check protocol in your gRPC apps, you can then use a standard\/common<br \/>\ntool to invoke this <code>Check()<\/code> method to determine server status.<\/p>\n<p>The next thing you need is the &ldquo;standard tool&rdquo;, and it&rsquo;s the<br \/>\n<a href=\"https:\/\/github.com\/grpc-ecosystem\/grpc-health-probe\/\" target=\"_blank\"><strong>grpc-health-probe<\/strong><\/a>.<\/p>\n<p><a href='https:\/\/kubernetes.io\/images\/blog\/2019-09-30-health-checking-grpc\/grpc_health_probe.png'><br \/>\n    <img width=\"768\"  title='grpc-health-probe on kubernetes'\n        src='https:\/\/kubernetes.io\/images\/blog\/2019-09-30-health-checking-grpc\/grpc_health_probe.png'\/><br \/>\n<\/a><\/p>\n<p>With this tool, you can use the same health check configuration in all your gRPC<br \/>\napplications. This approach requires you to:<\/p>\n<ol>\n<li>Find the gRPC &ldquo;health&rdquo; module in your favorite language and start using it<br \/>\n(example <a href=\"https:\/\/godoc.org\/github.com\/grpc\/grpc-go\/health\" target=\"_blank\">Go library<\/a>).<\/li>\n<li>Ship the<br \/>\n<a href=\"https:\/\/github.com\/grpc-ecosystem\/grpc-health-probe\/\" target=\"_blank\">grpc_health_probe<\/a><br \/>\nbinary in your container.<\/li>\n<li><a href=\"https:\/\/github.com\/grpc-ecosystem\/grpc-health-probe\/tree\/1329d682b4232c102600b5e7886df8ffdcaf9e26#example-grpc-health-checking-on-kubernetes\" target=\"_blank\">Configure<\/a><br \/>\nKubernetes &ldquo;exec&rdquo; probe to invoke the &ldquo;grpc_health_probe&rdquo; tool in the<br \/>\ncontainer.<\/li>\n<\/ol>\n<p>In this case, executing &ldquo;grpc_health_probe&rdquo; will call your gRPC server over<br \/>\n<code>localhost<\/code>, since they are in the same pod.<\/p>\n<h2 id=\"what-s-next\">What&rsquo;s next<\/h2>\n<p><strong>grpc-health-probe<\/strong> project is still in its early days and it needs your<br \/>\nfeedback. It supports a variety of features like communicating with TLS servers<br \/>\nand configurable connection\/RPC timeouts.<\/p>\n<p>If you are running a gRPC server on Kubernetes today, try using the gRPC Health<br \/>\nProtocol and try the grpc-health-probe in your deployments, and <a href=\"https:\/\/github.com\/grpc-ecosystem\/grpc-health-probe\/\" target=\"_blank\">give<br \/>\nfeedback<\/a>.<\/p>\n<h2 id=\"further-reading\">Further reading<\/h2>\n<ul>\n<li>Protocol: <a href=\"https:\/\/github.com\/grpc\/grpc\/blob\/v1.15.0\/doc\/health-checking.md\" target=\"_blank\">GRPC Health Checking Protocol<\/a> (<a href=\"https:\/\/github.com\/grpc\/grpc\/blob\/v1.15.0\/src\/proto\/grpc\/health\/v1\/health.proto\" target=\"_blank\">health.proto<\/a>)<\/li>\n<li>Documentation: <a href=\"https:\/\/kubernetes.io\/docs\/tasks\/configure-pod-container\/configure-liveness-readiness-probes\/\" target=\"_blank\">Kubernetes liveness and readiness probes<\/a><\/li>\n<li>Article: <a href=\"https:\/\/ahmet.im\/blog\/advanced-kubernetes-health-checks\/\" target=\"_blank\">Advanced Kubernetes Health Check Patterns<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Author: Ahmet Alp Balkan (Google) gRPC is on its way to becoming the lingua franca for communication between cloud-native microservices. If you are deploying gRPC applications to Kubernetes today, you may be wondering about the best way to configure health checks. In this article, we will talk about grpc-health-probe, a Kubernetes-native way to health check [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[13],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v21.9.1 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Blog: Health checking gRPC servers on Kubernetes - Howk IT-Dienstleistungen<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/howk.de\/?p=4311\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Blog: Health checking gRPC servers on Kubernetes - Howk IT-Dienstleistungen\" \/>\n<meta property=\"og:description\" content=\"Author: Ahmet Alp Balkan (Google) gRPC is on its way to becoming the lingua franca for communication between cloud-native microservices. If you are deploying gRPC applications to Kubernetes today, you may be wondering about the best way to configure health checks. In this article, we will talk about grpc-health-probe, a Kubernetes-native way to health check [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/howk.de\/?p=4311\" \/>\n<meta property=\"og:site_name\" content=\"Howk IT-Dienstleistungen\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/howk.de\" \/>\n<meta property=\"article:published_time\" content=\"2018-12-31T14:13:14+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/kubernetes.io\/images\/blog\/2019-09-30-health-checking-grpc\/options.png\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/howk.de\/?p=4311#article\",\"isPartOf\":{\"@id\":\"https:\/\/howk.de\/?p=4311\"},\"author\":{\"name\":\"admin\",\"@id\":\"https:\/\/howk.de\/#\/schema\/person\/b029bd02d4f35dce869ef54c81a100c5\"},\"headline\":\"Blog: Health checking gRPC servers on Kubernetes\",\"datePublished\":\"2018-12-31T14:13:14+00:00\",\"dateModified\":\"2018-12-31T14:13:14+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/howk.de\/?p=4311\"},\"wordCount\":503,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/howk.de\/#organization\"},\"articleSection\":[\"Hi Tech\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/howk.de\/?p=4311#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/howk.de\/?p=4311\",\"url\":\"https:\/\/howk.de\/?p=4311\",\"name\":\"Blog: Health checking gRPC servers on Kubernetes - Howk IT-Dienstleistungen\",\"isPartOf\":{\"@id\":\"https:\/\/howk.de\/#website\"},\"datePublished\":\"2018-12-31T14:13:14+00:00\",\"dateModified\":\"2018-12-31T14:13:14+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/howk.de\/?p=4311#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/howk.de\/?p=4311\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/howk.de\/?p=4311#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/howk.de\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Blog: Health checking gRPC servers on Kubernetes\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/howk.de\/#website\",\"url\":\"https:\/\/howk.de\/\",\"name\":\"Howk IT-Dienstleistungen\",\"description\":\"Howk IT Services - Howk IT-Dienstleistungen\",\"publisher\":{\"@id\":\"https:\/\/howk.de\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/howk.de\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/howk.de\/#organization\",\"name\":\"HowK\",\"url\":\"https:\/\/howk.de\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/howk.de\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/howk.de\/w1\/wp-content\/uploads\/2013\/12\/howk-logo.png\",\"contentUrl\":\"https:\/\/howk.de\/w1\/wp-content\/uploads\/2013\/12\/howk-logo.png\",\"width\":170,\"height\":170,\"caption\":\"HowK\"},\"image\":{\"@id\":\"https:\/\/howk.de\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/howk.de\",\"http:\/\/de.linkedin.com\/in\/howkde\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/howk.de\/#\/schema\/person\/b029bd02d4f35dce869ef54c81a100c5\",\"name\":\"admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/howk.de\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/b5a20f4d07bca1b73f25cff58a1116c4?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/b5a20f4d07bca1b73f25cff58a1116c4?s=96&d=mm&r=g\",\"caption\":\"admin\"},\"url\":\"https:\/\/howk.de\/?author=1\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Blog: Health checking gRPC servers on Kubernetes - Howk IT-Dienstleistungen","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/howk.de\/?p=4311","og_locale":"en_US","og_type":"article","og_title":"Blog: Health checking gRPC servers on Kubernetes - Howk IT-Dienstleistungen","og_description":"Author: Ahmet Alp Balkan (Google) gRPC is on its way to becoming the lingua franca for communication between cloud-native microservices. If you are deploying gRPC applications to Kubernetes today, you may be wondering about the best way to configure health checks. In this article, we will talk about grpc-health-probe, a Kubernetes-native way to health check [&hellip;]","og_url":"https:\/\/howk.de\/?p=4311","og_site_name":"Howk IT-Dienstleistungen","article_publisher":"https:\/\/www.facebook.com\/howk.de","article_published_time":"2018-12-31T14:13:14+00:00","og_image":[{"url":"https:\/\/kubernetes.io\/images\/blog\/2019-09-30-health-checking-grpc\/options.png"}],"author":"admin","twitter_card":"summary_large_image","twitter_misc":{"Written by":"admin","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/howk.de\/?p=4311#article","isPartOf":{"@id":"https:\/\/howk.de\/?p=4311"},"author":{"name":"admin","@id":"https:\/\/howk.de\/#\/schema\/person\/b029bd02d4f35dce869ef54c81a100c5"},"headline":"Blog: Health checking gRPC servers on Kubernetes","datePublished":"2018-12-31T14:13:14+00:00","dateModified":"2018-12-31T14:13:14+00:00","mainEntityOfPage":{"@id":"https:\/\/howk.de\/?p=4311"},"wordCount":503,"commentCount":0,"publisher":{"@id":"https:\/\/howk.de\/#organization"},"articleSection":["Hi Tech"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/howk.de\/?p=4311#respond"]}]},{"@type":"WebPage","@id":"https:\/\/howk.de\/?p=4311","url":"https:\/\/howk.de\/?p=4311","name":"Blog: Health checking gRPC servers on Kubernetes - Howk IT-Dienstleistungen","isPartOf":{"@id":"https:\/\/howk.de\/#website"},"datePublished":"2018-12-31T14:13:14+00:00","dateModified":"2018-12-31T14:13:14+00:00","breadcrumb":{"@id":"https:\/\/howk.de\/?p=4311#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/howk.de\/?p=4311"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/howk.de\/?p=4311#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/howk.de\/"},{"@type":"ListItem","position":2,"name":"Blog: Health checking gRPC servers on Kubernetes"}]},{"@type":"WebSite","@id":"https:\/\/howk.de\/#website","url":"https:\/\/howk.de\/","name":"Howk IT-Dienstleistungen","description":"Howk IT Services - Howk IT-Dienstleistungen","publisher":{"@id":"https:\/\/howk.de\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/howk.de\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/howk.de\/#organization","name":"HowK","url":"https:\/\/howk.de\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/howk.de\/#\/schema\/logo\/image\/","url":"https:\/\/howk.de\/w1\/wp-content\/uploads\/2013\/12\/howk-logo.png","contentUrl":"https:\/\/howk.de\/w1\/wp-content\/uploads\/2013\/12\/howk-logo.png","width":170,"height":170,"caption":"HowK"},"image":{"@id":"https:\/\/howk.de\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/howk.de","http:\/\/de.linkedin.com\/in\/howkde"]},{"@type":"Person","@id":"https:\/\/howk.de\/#\/schema\/person\/b029bd02d4f35dce869ef54c81a100c5","name":"admin","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/howk.de\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/b5a20f4d07bca1b73f25cff58a1116c4?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/b5a20f4d07bca1b73f25cff58a1116c4?s=96&d=mm&r=g","caption":"admin"},"url":"https:\/\/howk.de\/?author=1"}]}},"_links":{"self":[{"href":"https:\/\/howk.de\/index.php?rest_route=\/wp\/v2\/posts\/4311"}],"collection":[{"href":"https:\/\/howk.de\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/howk.de\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/howk.de\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/howk.de\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4311"}],"version-history":[{"count":0,"href":"https:\/\/howk.de\/index.php?rest_route=\/wp\/v2\/posts\/4311\/revisions"}],"wp:attachment":[{"href":"https:\/\/howk.de\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4311"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/howk.de\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4311"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/howk.de\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4311"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}