OpenBao: Horizontally Scaling Secrets Management - OSSNA 2026

Slides and content from Alex's Open Source Summit NA 2026 talk, describing the horizontal scalability features of OpenBao. For a video, see the Linux Foundation's official YouTube channel . SVG rendering is not supported on your browser. Welcome everyone to my talk on OpenBao and how we added horizontal scalability to the project. I'm Alex Scheel, Head of OpenBao Development at ControlPlane, a long time member of the OpenBao TSC, and chair of the OpenBao Development Working Group. I've been fortunate to have a hand in the development of OpenBao since nearly the beginning of the project, and before that, at HashiCorp's Vault CryptoSec team. If, like me, you were wishing you could get out for a post-lunch walk, thank you for staying, but we'll have to settle for some photos of Minneapolis I've sprinkled through the presentation. And thank you all for visiting Minnesota, whether from near or far! SVG rendering is not supported on your browser. First, what is OpenBao? SVG rendering is not supported on your browser. OpenBao is an open-source secrets manager, featuring everything from static and dynamic secrets to PKI and key management services. We have integrations with everything from External Secrets Operator and Cert Manager to OpenTofu, SOPS, and cosign. We also have a growing ecosystem; check out our ecosystem page afterwards and if you're an adopter, integrator, or supporter, consider adding your logo! OpenBao is an OpenSSF Sandbox project and is licensed under the MPLv2. SVG rendering is not supported on your browser. If this sounds familiar, it should be! OpenBao is the open-source continuation of HashiCorp Vault, started in late 2023 after the relicensing to the non-OSI BUSL license. We aim to keep API compatibility for client applications, but to improve the operator and developer experience. Towards that goal, we've landed several Vault Enterprise features--such as horizontal scalability, which this talk focuses on--but also many original improvements like declarative self-initialization, storage-level improvements for better performance and snapshot consistency, and landing in our next release, things like an externally pluggable KMS interface, operator-defined workflows built on the existing profile engine, and per-namespace (per-tenant) barrier encryption keys with optional namespace sealing support. (This photo is taken from the Mill Ruins park, facing the Mill City Museum which is hosting tonight's drone show.) SVG rendering is not supported on your browser. OpenBao's top-level governance body, its technical steering committee (TSC), is currently made up of the following companies: my employer, ControlPlane, a seat jointly occupied by SAP and Liquid Reply who SAP has contracted with to contribute to our community, Adfinis, Wallix, IOTech, and GitLab. Our governance is openly documented in the project’s main repository. We have tracks for leadership (in the form of the TSC), for developers (in the form of the Dev Working Group and its many sub-working groups), and maintainership for those interested in contributing to reviews on the project. We’ve also started a marketing working group if talking about OpenBao or its usage is more your style. SVG rendering is not supported on your browser. In my early 2025 talk at FOSDEM (which you can see on the OpenBao blog archive), I mentioned that I had hoped to grow a community around OpenBao of companies that make money supporting it, just like Kubernetes. Not long after that talk, both Adfinis and SAP stepped up and started contributing to the community. This March, I joined ControlPlane to further that mission as well, leading our efforts to commercialize support and maintenance of the project. You can read more about that on ControlPlane's blog. ControlPlane employs maintainers for FluxCD, is a contributor to several CNCF and OpenSSF efforts, and focuses on a highly regulated customer base like banks and government entities. I'm happy to report for anyone following along, the OpenBao community is healthy, growing, and my hopes have been answered! Though of course, we always welcome more contributions of any sort! SVG rendering is not supported on your browser. This now brings us to what we're here to discuss. Horizontal Scalability, and the journey to support it. I'll focus on three parts mostly: What building blocks did the fork have? What were we starting with? How did we support Raft in OpenBao v2.5.0? What challenges did we face along the way? How will we support PostgreSQL and more going forward? How will this improve the operator experience? SVG rendering is not supported on your browser. We started by trying to understand what we inherited from HashiCorp Vault. SVG rendering is not supported on your browser. Most importantly, we started with a highly available (though, not horizontally scalable!) base. In OpenBao as it stood before this feature, we had data replication already taken care of. We supported two HA storage backends (Integrated Storage aka Raft and PostgreSQL), OpenBao and HashiCorp Vault have a single active node which can perform write operations; this is a limitation of the Raft protocol and used as a design decision elsewhere. In Vault Community Edition which we inherited, probably for open-core feature differentiation with Vault Enterprise, HashiCorp only supported cold standbys: these were "unsealed" and ready to take over if the active node went down, but could only forward requests to it and did not do any processing locally. In OpenBao v2.5.0, we wanted to land horizontal scalability, which meant we needed a way for these cold standby nodes to serve read requests. They still wouldn't handle write requests, but they could help alleviate some of the load from the single active node, depending on the user's workloads. And, of course, we had a constraint that we wanted to behave similarly to Vault Enterprise's Performance Standby node types. SVG rendering is not supported on your browser. How does Vault Enterprise behave? And what is a read request? Vault Enterprise roughly defines a read request as something that doesn't cause any storage writes. Confusingly, this has a very loose association with HTTP verbs. For instance, the Certificate Revocation List (CRL) rotation endpoint in the PKI engine incurs a storage write (the CRL itself), but uses the GET verb. More obviously, login, e.g., via the userpass authentication method, causes several storage writes. Both of these, if they come into a read-enabled standby, would be forwarded. For the types of requests that will be handled by a standby node, a KVv1 secret read operation would be handled locally; it is guaranteed to not have any write operations. Most KVv2 operations will not have any writes either. Additionally, some POST operations will be strictly handled locally: if a PKI engine's role is configured not to store leaf certificates, the issuance endpoint could be handled on the read-enabled standby node. SVG rendering is not supported on your browser. From an operational standpoint it is now clear what we want to achieve, though we have a few quirks we need to work out with our storage backends. How do we handle invalidation of caching that occurs naturally? For integrated storage, each OpenBao node becomes its own storage backend. It uses the Raft consensus protocol to handle replication of data between all nodes. This protocol is based on a leadership election process, which requires an odd number of voting nodes to be consistent. Each write sent by the leader is confirmed by another vote. This write confirmation forms the basis of the write-ahead-log (WAL) mechanism. This WAL mechanism contains enough information about every storage write, which gives us an easy way to invalidate cached storage entries. On PostgreSQL however, any number of nodes can be used as they all race to acquire a lock stored in the database. While PostgreSQL handles the data replication for us--giving us a M :N OpenBao service to…

添加评论
点赞收藏
点踩分享查看原文
评论
?
参与讨论