<p>When organizations start to scale, getting the correct infrastructure-as-code workflow right can be a daunting task. Thankfully, when teams think about the next step in their cloud-native journey after Terraform, Crossplane is there to help. In this blog post, we'll take a look at an example company going through some growing pains and how the Crossplane provider-terraform can help. For an in-depth how-to guide, check out the <a href="https://docs.upbound.io/providers/provider-terraform/migrate-provider-tf/" target="_blank" rel="noopener noreferrer">Upbound docs</a> migration tutorials.</p><h2>Introduction</h2><p>It's 2024, and your startup JNCOmatic isn't just hot; it's Unicorn-in-a-Ferrari hot! Your groundbreaking app, which can slap a pair of hyper-realistic, backwards JNCO jeans onto anyone (or any dog on its hind legs), is running on all cylinders.</p><h2>The Terraform Tango</h2><p>Back in the day, when Viv was the lone Op of the Ops team, things were simpler. Deployments were a breeze, the scale was low, and AWS bills were…manageable. Viv started out on the team of one by implementing a Terraform workflow to easily deploy all the necessary infrastructure to the cloud. She mapped out the resources, accounted for spot instance scaling, and ensured user data was encrypted from a few files. Here’s an example of a Terraform resource configuration she might have used:</p>

<h2>Scaling up</h2><p>But that was before, CourtJortster, a TikTok influencer with a flair for the '90s, happened. Livestreaming the entire 1993 Chicago Bulls roster in NBA2K – all decked out in JNCOmatic gear – turned your world upside down.</p><p>The Terraform approach worked well when the team was a single person responsible for serving a small number of users. The first step of the cloud-native journey was just laying a foundation, but as JNCOMatic scales by orders of magnitude, Viv knows the workflow needs an update.</p><p>Four people on an Ops team can’t collaborate as they would like on the Terraform configuration files. You’ve got a growing number of developers who need to spin up environments to test new methods of generating realistic jean texture with AI. And managing state files? "As if!"</p><h2>Shifting Gears: `provider-terraform` and Crossplane</h2><p>Viv decides the Ops team needs to shift to PlatformOps. PlatformOps is how companies ensure their software, infrastructure, and security experts can get the tools and environments they need to get their jobs done. The developer platform the company needs is <a href="https://www.crossplane.io/" target="_blank" rel="noopener noreferrer">Crossplane</a>.</p><p>Crossplane is an open source Kubernetes extension that transforms your Kubernetes cluster into a *universal control plane*. The control plane allows you to connect Kubernetes clusters to external resources and enables platform teams to build custom Kubernetes APIs to build and manage resources. The platform team already uses Kubernetes for application deployment so there is not a significant learning curve - in fact, Viv is removing tool sprawl from the organization by consolidating infrastructure deployment in the same workflow as application deployment.</p><p>By using control planes instead of Terraform, the team doesn't need to worry about configuration drift or use a custom language for their deployments. Painful rollbacks or errored apply operations that stop deployments in their tracks left the team with half-applied configurations with little choice but to start from scratch. In addition to complex module management, multi-cloud or hybrid deployments in Terraform require a lot of patchwork to make sure your resources work together.</p><p>The big problem: The team has amassed so many Terraform configurations that it’s difficult to easily start over on Crossplane.</p><p>The big solution: Crossplane has a <a href="https://marketplace.upbound.io/providers/upbound/provider-terraform/v0.14.1" target="_blank" rel="noopener noreferrer">`provider-terraform`</a> that executes HashiCorp Configuration Language (HCL) within Crossplane.</p><p>The same Terraform configuration above is easily slotted into a Crossplane spec:</p>

<p>The team can easily move from HCL to Crossplane with the `provider-terraform` option and reduces the learning curve a little more. After creating a few more Crossplane specs for their resources, the team gets more comfortable with the workflow.</p><p>The <a href="https://www.techtarget.com/searchitoperations/definition/GitOps" target="_blank" rel="noopener noreferrer">GitOps</a> philosophy baked into Crossplane makes it easy for the team to collaborate. The Crossplane configurations live in a team repository and there are no locks or state files to contend with. This workflow also minimizes resource drift. If a team member edits a resource in AWS through CloudFormation or in the GUI, Crossplane uses continuous reconciliation through Kubernetes to keep resources aligned with the source of truth in the Crossplane specs. In other words, your desired state in the spec is exactly what Crossplane will deploy and will prevent drift through Kubernetes' reconciliation loop.</p><h2>All-in on Crossplane</h2><p>Eventually, the team fully embraces Crossplane, opting to use its native Kubernetes spec for infrastructure management.</p>

<p>In this example, the team employs a consistent configuration language throughout the deployment spec. Crossplane's AWS provider supplants the Terraform provider, directly orchestrating the desired infrastructure.</p><p>This process takes time and it can’t be completed overnight. Meticulous planning is essential for critical infrastructure. Thankfully, you don't have to do it alone. As a Cloud Native Computing (<a href="https://www.cncf.io/" target="_blank" rel="noopener noreferrer">CNCF</a>) project, Crossplane's open-source, community-driven nature offers support through an open source <a href="https://github.com/crossplane" target="_blank" rel="noopener noreferrer">GitHub repository</a>, <a href="https://crossplane.slack.com/join/shared_invite/zt-2dimsong0-VwekA~pVM8OW~Mpg0JtawA#/shared-invite/email" target="_blank" rel="noopener noreferrer">Slack channel</a> and bi-weekly <a href="https://github.com/crossplane/crossplane#get-involved" target="_blank" rel="noopener noreferrer">community meetings</a>.</p><p>Once the team maps out their existing configuration in Terraform, the next step is to match the resources in Crossplane. This is where the <a href="https://marketplace.upbound.io/" target="_blank" rel="noopener noreferrer">Upbound Marketplace</a> comes in. The Marketplace is the central hub for finding Crossplane providers. Upbound, the company behind Crossplane, offers a selection of official, actively maintained, and tested providers. In the Marketplace, the team has access to Providers (external APIs you consume from cloud providers) and Configurations (out of the box solutions for deploying the building blocks of your architecture).</p><h2>The Next Level</h2><p>If your team is ready to level up your infrastructure, <a href="https://docs.upbound.io/providers/provider-terraform/migrate-provider-tf/" target="_blank" rel="noopener noreferrer">check out the Crossplane Get Started guide for a step-by-step tutorial to get familiar with Crossplane</a>. If you're looking for top-tier <a href="https://docs.upbound.io/concepts/mcp/" target="_blank" rel="noopener noreferrer">Managed Control Plane</a> environments for hosted, scalable infrastructure, check out <a href="/">Upbound</a> and sign up for a <a href="/register/a">free trial</a> now.</p><p>Looking for a real world example? <a href="webinars/kubernetes-called-and-it-wants-your-iac-back-using-control-planes-to-modernize-your-iac-tech-stack" target="_blank" rel="noopener noreferrer">Millennium BCP modernized its tech stack with Upbound</a> and has a few pointers for you.</p><p>For more information on migrating from Terraform or the `provider-terraform`, <a href="https://docs.upbound.io/providers/provider-terraform/migrate-provider-tf/" target="_blank" rel="noopener noreferrer">check out the Migration Guide and the Marketplace documentation</a>.</p>

Time to Cross Over: Migrating from Terraform to Crossplane

<p>Upbound announces a practical application of Crossplane's capabilities through a new reference implementation for <a href="https://cnoe.io/" target="_blank">Cloud Native Operational Excellence (CNOE)</a>. This example demonstrates how <a href="https://www.crossplane.io/" target="_blank">Crossplane</a>, a powerful open source control plane, serves as a valuable starting point for organizations navigating the complexities of cloud-native development.</p><h2>CNOE + Crossplane</h2><p>CNOE, supported by industry leaders such as Adobe, AWS, Autodesk, Salesforce, and Twilio, aims to simplify developer tooling and guide informed technology choices. However, current tools for building Internal Developer Platforms (IDPs) lack practicality, leaving organizations with the challenge of assembling components for effective cloud-native operational excellence.</p><p>Enter Crossplane, a robust open-source control plane created by Upbound and governed under the Cloud Native Computing Foundation (CNCF). This tailored example provides a seamless entry point for platform teams eager to initiate their <a href="https://www.upbound.io/solutions/internal-developer-platform" target="_blank">IDP</a> journey, showcasing Crossplane's potential in simplifying cloud-native operations.</p>

<p>Crossplane is a key component of CNOE, and with today’s announcement, engineers have an easy way to get started with best practices in building a control plane as the foundation of their IDPs.</p><h2>Benefits Simplified</h2><p>By highlighting this Crossplane configuration, organizations can explore the potential of Crossplane and the Upbound platform in a straightforward manner. Standardizing on Crossplane can help build a robust foundation for your cloud-native stack and accelerate the creation of IDPs. This is a practical illustration, inviting organizations to delve into the unique benefits and ease of use that Crossplane brings to the cloud-native operational landscape.</p>

<p>While this example is demonstrated with AWS, it's essential to note that it's designed as a starting point. Modifying basic configurations for Azure or Google Cloud is a small task with predefined configuration packages.</p><h2>Hands-On Overview in 3 Steps</h2><p>Ready to get started with Crossplane and CNOE? Follow these simple steps:</p><ol><li>Go to the <a href="https://marketplace.upbound.io/" target="_blank">Upbound Marketplace</a> and search for <a href="https://marketplace.upbound.io/configurations/upbound/platform-ref-aws-cnoe" target="_blank">platform-ref-aws-cnoe</a></li><li>Install this configuration to your cluster with "Install Manifest"</li><li>In just minutes, you'll have your first IDP up and running. This includes a Kubernetes Cluster with Network, ArgoCD, Backstage, and Keycloak as SSO.</li></ol><p>Now, you can start spinning up more resources within this powerful stack, customizing it to fit your specific needs.</p><p>If you want to contribute to the project you can head over to <a href="https://github.com/upbound/platform-ref-aws-cnoe/" target="_blank">platform-ref-aws-cnoe in GitHub</a>.</p><p>Unlock the potential of Crossplane and kickstart your cloud-native journey today with Upbound!</p>

Upbound Announces New Crossplane Reference Implementation for CNOE

<h2>Introduction</h2><p>Crossplane Compositions are a powerful abstraction layer that enables platform teams to create a custom API for their internal customers to simplify and standardize the infrastructure management process. The Composition authors wrestle with the complexity of cloud infrastructure and at the same time need to ensure a stable and user-friendly API surface for the platform consumers.</p><p>Historically, Compositions were intended to support only very simple resources’ manipulation to allow the API calls to patch and transform the information passed from a Claim or Composite Resource (XR) to the Composition engine. The lack of touring complete language supporting the required transformations resulted in a very verbose YAML files with lots of repetitions, increasing the toil of authoring Compositions.</p><p>With the addition of <a href="https://github.com/crossplane/crossplane/releases/tag/v1.11.0" target="_blank" rel="noopener noreferrer">Compositions Functions</a> in the Crossplane v.1.11 it is possible to use a programming language, like Go or Python (more to come) or in our case a capability of Go called <a href="https://pkg.go.dev/text/template" target="_blank" rel="noopener noreferrer">Go-templating</a> to enrich and transform the authoring of Compositions.</p><p>This blog will guide you through the process of rewriting a simple internal platform Composition in the Go templating style using the <a href="https://marketplace.upbound.io/functions/crossplane-contrib/function-go-templating" target="_blank" rel="noopener noreferrer">go-templating-function</a>.</p><h2>Use Case: Collapsing two Compositions into one</h2><p>Since traditional Composition's engine does not allow using conditionals, there is no way to render one Composition or another using a parameter from a Claim. To illustrate this example, let’s look at the two Compositions that render a GCP Bucket:</p><h3>Standard bucket</h3><p>This very simple composition renders a single GCP bucket</p>

<h3>Versioned Bucket</h3><p>Another Composition needs to be created to accommodate for a versioning setting.</p>

<p>In this case, we have to have two Compositions and need to use a Composition selector in a claim just for this one field.</p><h3>Using go-templating-function</h3><p>The two Compositions can be transformed into just one using the go-templating-function. Follow these steps</p><p>1. Install <a href="https://marketplace.upbound.io/functions/crossplane-contrib/function-go-templating" target="_blank" rel="noopener noreferrer">go-templating-function</a> and <a href="https://marketplace.upbound.io/functions/crossplane-contrib/function-auto-ready" target="_blank" rel="noopener noreferrer">function-auto-ready</a> from the marketplace.</p>

<p>2. Convert the Composition to use the go templating. The initial conversion gives us one to one translation between the standard “Patch and Transform” style Composition and the go-templating style. We are going to improve this design.</p>

<p>Using Composition Functions is very well documented in the <a href="https://docs.crossplane.io/v1.14/concepts/composition-functions/" target="_blank" rel="noopener noreferrer">Crossplane docs</a>. Read the details to learn more about it.</p><p>The current implementation of our Composition renders the <i>non-versioned</i> Bucket. Next steps are to change the XRD and template to accommodate for the versioning field.</p><p>Using the new Crossplane beta trace command is helpful in checking the latest events on the composed resources from the Claim/XR:</p>

<h3>Adding the versioning field</h3><p>1. Modify the XRD to set the versioning field with a boolean flag:</p>

<p>2. Now let’s modify the Composition template to add the conditional for the versioning field.</p>

<p>3. Now we are ready to apply the Claim of the versioning enabled Bucket to the preconfigured GCP project.</p>

<h3>Default values</h3><p>Not all the schema fields are required, and for those the Composition provides default values. location and storageClass are not required in the schema, and the Composition provides default values for those fields when omitted. Let’s add them to the template. Now supplying a Claim without the location and storageClass fields will result in the default values being rendered like so.</p>

<h3>Bringing back PatchSets</h3><p>This specific Composition is very simple, but there are multiple others where the same patch needs to be applied to multiple resources. In the original Composition we have the ownerAndServiceLabels that are applied to all resources that support labels.</p>

<p>With the go-templating-function, it’s easy to define the <i>patchSet-like</i> behavior</p><p>Here is the modified Composition template part with the labels:</p>

<h3>Template Variables</h3><p>One small quality of life improvement is to define variables in the template so there is no need to type .observed.composite.resource.spec.parameters all the time. Here is the final version of the Composition:</p>

<h2>Conclusion</h2><p>In conclusion, the use of the go-templating-function in Compositions authoring significantly simplifies the process of creating and managing resources with varying configurations and makes it easier to manage complex Compositions.</p><p>It allows for the creation of Compositions with conditional fields, reducing the need for multiple compositions for each variation of a resource. The re-introduction of PatchSets-like behavior makes the portability from “Patch and Transform” style Compositions easier. Adding variables scoped to Composite Resource paths makes it easier to reason about the template.</p><p>Additionally, using the new Crossplane CLI functionality to render the resources based on the Claim, Composition and functions and also trace the state of resources is a very helpful development tool. Here’s a tip: running the commands with watch makes the development loop even better.</p>

<p>If you are interested in technical details and design of the go-templating-function, check out the <a href="https://github.com/crossplane/crossplane/pull/4935" target="_blank" rel="noopener noreferrer">go-templating-function one-pager</a>. Big thanks to <a href="https://github.com/ezgidemirel" target="_blank" rel="noopener noreferrer">@ezgidemirel</a> for creating the function! It significantly improves the Compositions authoring process.</p><p>You can find various functions in our <a href="https://marketplace.upbound.io/functions" target="_blank" rel="noopener noreferrer">Marketplace</a> and <a href="https://docs.crossplane.io/knowledge-base/guides/write-a-composition-function-in-go/" target="_blank" rel="noopener noreferrer">create your own</a> to take the Crossplane Compositions to the next level.</p>

Improve Crossplane Compositions Authoring with go-templating-function

<h4>Bringing structure and security to the Kubernetes type system with Crossplane packages.</h4><p>Organizations running on Kubernetes have been steadily moving to a multi-cluster approach. The life-cycle of any individual cluster is relatively short and provisioning new clusters is a common occurrence. There are a variety of factors leading to this shift, including:</p><ol style="list-style-type:decimal;"><li><strong>Availability</strong>: more clusters means that losing any one cluster is not catastrophic.</li><li><strong>Isolation</strong>: while RBAC and Namespaces provide isolation within a single cluster, utilizing multiple clusters allows for cluster-scoped isolation, and physical isolation when necessary.</li><li><strong>Scalability</strong>: depending on the nature of your workloads, the various components that make up a Kubernetes cluster may not scale to accommodate your use case. One such example is the API server struggling to handle a large number of Custom Resource Definitions (CRDs), as detailed in a <a href="https://blog.upbound.io/__GHOST_URL__/scaling-kubernetes-to-thousands-of-crds/">recent post</a> by my colleague <a href="https://twitter.com/internegz" target="_blank" rel="noopener noreferrer">Nic Cope</a>.</li></ol><p>This is not a comprehensive list, but the reasons listed are broadly applicable to many organizations. Mitigating these issues has been compelling enough to drive them to adopt a more complex architecture.</p><p>As organizations have moved into this new world, there has simultaneously been an explosion in the number of CRDs and controllers that are installed in a given cluster. This greatly enhances the capabilities of Kubernetes, but also increases the management burden of replicating the same custom API across many clusters. As a result, interacting with a Kubernetes cluster has become more like writing a program, but without any of the safety or guarantees offered by a modern programming language. In this post, we’ll explore why a lack of API compatibility can lead to dangerous and unpredictable behavior, and how Crossplane packages make it possible to present a verifiably consistent programming interface across multiple clusters.</p><h2>The Perils of an Unknown API</h2><p>What can actually go wrong when API’s diverge across clusters? There are multiple possible outcomes, listed below in order of increasing severity.</p><ol style="list-style-type:decimal;"><li><strong>The “Early Conflict”</strong>: an object being created violates the type definition (CRD) in one cluster, but not the other. This is the best case scenario, but still presents a poor user experience to the developer.</li><li><strong>The “Hope I’m Not On-Call”</strong>: The object is successfully created in each cluster, but only reaches a healthy status in one due to divergence in reconciliation logic, which is decoupled from the definition of the type.</li><li><strong>The “Sleeper Agent”</strong>: The object is successfully created in each cluster, and both reach a healthy status, but reconciliation logic has performed different operations in response.</li></ol><p>In each of these scenarios, a stronger type system could surface these issues prior to creating the object. We are specifically looking for the following attributes:</p><ul><li>Verification that type definitions are equivalent across clusters.</li><li>Verification that reconciliation logic is equivalent across clusters.</li></ul><p><span style="background-color:rgb(255,255,255);color:rgb(3,7,36);font-family:Avenir-Roman, Arial, sans-serif;">Crossplane packages can offer these guarantees, but before exploring how, let’s examine how the Kubernetes type system compares to that of popular programming languages.</span></p><h2>What’s a type?</h2><p>Despite the ever-growing API surface area and the movement to a multi-cluster world, we have not seen a fundamental change in Kubernetes architecture. This is mostly a good thing; the stability and predictability of Kubernetes is a major factor in its widespread adoption. However, when we drastically change how we consume Kubernetes, we must also consider the impact that has on how we configure it. One of the most pronounced differences in configuring a Kubernetes cluster in a multi-cluster world is the scoping of types.</p><p>Kubernetes comes with some built-in types - <code>Pods</code>, <code>Deployments</code>, etc. - and allows you to define your own with CRDs. This is not dissimilar from most programming languages: there are built-in primitive types, such as <code>boolean</code>, <code>integer</code>, etc., but there are also user-defined types, which may be defined via a built-in type, such as a <code>struct</code>.</p><blockquote><p>I <a href="https://danielmangum.com/posts/the-missing-kubernetes-type-system/" target="_blank" rel="noopener noreferrer">have argued</a> that types defined by CRDs directly are, or should be, considered built-in types, but that’s a conversation for another day.</p></blockquote><p>User-defined types are typically scoped - they are identified relative to where they are defined. For example, in the Go programming language I can define a <code>MyType</code> in the <code>cooltypes</code> package. I can also define a <code>MyType</code> in the <code>okaytypes</code> package. The identifiers for these types internally are <code>cooltypes.MyType</code> and <code>okaytypes.MyType</code>, allowing them to be distinguished when consumed.</p>

<p>Zooming out even further, the packages are hosted somewhere on the internet, or at a location on my local machine. This adds further specificity to their scope, guarding against any collisions with another <code>cooltypes</code> package that may be hosted elsewhere. However, with these two types having identical definitions, do we really need to distinguish one from the other? As with many topics in computing: it depends.</p><h2>Defining Equivalence</h2><p>There are two popular mechanisms for defining equality in a type system: <strong>nominal equivalence</strong> and <strong>structural equivalence</strong>. Nominal equivalence defines two types as being the same if they have the same name. For example, in the following example, the types of variables <code>a</code> and <code>b</code> are considered equivalent, but the type of <code>c</code> is not considered equivalent to <code>a</code> or <code>b</code>.</p>

<p>This is because Go, like many languages, uses <strong>nominal equivalence</strong> when evaluating equality of types. Importantly, the <i>name</i> of the type in this context is referring to the fully qualified name. If we compile this program and look at the symbol table, we can see each of the two unique type names defined:</p>

<p>In a language that uses <strong>structural equivalence</strong>, such as TypeScript, OCaml, Haskell, and more, <code>cooltypes.MyType</code> and <code>okaytypes.MyType</code> would be considered equivalent because their members (i.e. <code>Message </code>and <code>ID</code>) are equivalent. In fact, Go uses structural equivalence when evaluating whether a given type implements an <code>interface</code> (i.e. the <i>structure</i> of the methods are equivalent).</p><p>In Go, dependencies are pinned at the <a href="https://pkg.go.dev/cmd/go#hdr-Modules__module_versions__and_more" target="_blank" rel="noopener noreferrer">module level</a> (we are going to skip over the new <a href="https://go.dev/blog/get-familiar-with-workspaces" target="_blank" rel="noopener noreferrer">workspaces</a> flow added in 1.18 for this post). When a new dependency is added, the go.mod file is updated with a reference to the location of the module, and the <a href="https://go.dev/ref/mod#go-sum-files" target="_blank" rel="noopener noreferrer">digest of the content of the package</a> (and any transitive dependencies) is added to the <code>go.sum</code> file. This makes it such that you, or anyone else who is adding code to the module, are able to verify that the type you are creating an instance of is exactly the same as it was last time, barring any change in the <code>go.mod</code> or <code>go.sum</code> files. It also means that we can easily verify whether two types in separate modules are equivalent by ensuring that they are nominally equivalent, then comparing the checksums of their source modules. This is not something that is typically required when writing programs because creating instances of types is typically scoped to a single code base (i.e. module).</p><p>Kubernetes also uses nominal equivalence in its type system, with each type having a name consisting of its Group, Version, and Kind (GVK). However, unlike a statically compiled binary, Kubernetes has types dynamically added and removed at runtime. This means when you are programming against a cluster’s API, the name of the type you are creating an instance of does not fully reflect the type’s identity—the API may have been modified since the last time you interacted with the cluster, and changing the source or content of the type is decoupled from creating new instances of the type. Furthermore, unlike writing a program, there are typically multiple personas involved when configuring and consuming a Kubernetes cluster.</p>

<p>This becomes an even larger problem when multiple clusters are involved, as the constraint of only one type existing for a single GVK is no longer upheld. When we are treating clusters as ephemeral, the inability to verify whether two types with the same name are equivalent becomes an issue, as the semantics of the “language” we are using is changing beneath our feet. A multi-cluster environment is akin to programming across multiple Go modules, where someone else may be changing types, and the only way to ensure consistency is by a literal byte-to-byte comparison.</p><p>In fact, the situation is even more complex than described thus far. While we have primarily been discussing their structure (i.e. definition), types also have associated behavior. In programming languages this is frequently represented as methods that may be invoked imperatively on an instance of the type. In a Kubernetes cluster, associated behavior takes the form of one or more controllers that are invoked based on the declarative life-cycle of the type. Despite these controllers being dependent upon the existence of the types they reconcile. Kubernetes does not natively create a strong association between types and their controllers. Because of this, even comparing the content of a CRD with another with the same GVK does not necessarily indicate equivalence in both structure and behavior.</p><p>In an ideal world, we could verify whether creating an instance of a type in one cluster would result in the same operations being performed as in another cluster.</p><h2>Crossplane Packages and OCI Images</h2><p><a href="https://crossplane.io/docs/v1.9/concepts/packages.html" target="_blank" rel="noopener noreferrer">Crossplane packages</a> (xpkgs) are how new types are added to Kubernetes clusters where Crossplane is installed. Their <a href="https://crossplane.io/docs/v1.9/reference/xpkg.html" target="_blank" rel="noopener noreferrer">specification</a> is a subset of the <a href="https://github.com/opencontainers/image-spec/blob/main/spec.md" target="_blank" rel="noopener noreferrer">OCI image specification</a> (i.e. xpkgs impose additional requirements), which allows for users to take advantage of the vast ecosystem of tooling and distribution.</p><p>OCI images are made up of content and manifests. Traditionally, the content includes filesystem changeset layers (i.e. <i>blobs</i>), which can be extracted to form a complete filesystem for a container at runtime. Manifests tie these layers together, specifying how to obtain the content, as well as the order in which the changesets should be applied. Crossplane takes advantage of this flexible model by assigning significance to content in the blobs, specifically a <code>package.yaml</code> file that exists in the root of the final filesystem. This file is a <a href="https://yaml.org/spec/1.2.2/#92-streams" target="_blank" rel="noopener noreferrer">YAML stream</a> containing package metadata, and all the types that Crossplane should install. Crossplane is indifferent as to the content of the rest of the image.</p>

<p>A Provider or Configuration package can be installed by creating an instance of the <code>Provider</code> or <code>Configuration</code> type in the cluster with a reference to the package image.</p>

<p>When installing, Crossplane first fetches the digest of the package image manifest, and creates either a <code>ProviderRevision</code> or <code>ConfigurationRevision</code>, depending on the package type, with a name constructed from the manifest digest. Next, the JSON content of the manifest is fetched. As mentioned previously, the manifest contains a list of content blobs in the form of descriptors. A descriptor contains the media type, size, and digest of the layer. It may also include annotations, among other fields, with additional information.</p><blockquote><p>Note that both the manifest, and descriptors in the manifest, may include annotations. In this case we are talking about annotations on descriptors in the manifest.</p></blockquote>

<p>If the one of the layer descriptors in the package image manifest has a <code>io.crossplane.xpkg</code>: base annotation, Crossplane only fetches that layer, then extracts the <code>package.yaml</code> file from it. If none of the layer descriptors in the manifest have the annotation, all layers in the manifest are fetched, the full filesystem is constructed, then the <code>package.yaml</code> is extracted from the root directory.</p>

<blockquote><p>Note that this manifest also includes an <code>io.crossplane.xpkg: upbound</code> annotation, which is an xpkg extension. We’ll cover extensions in a future post, but you can read about how to implement your own in the <a href="https://crossplane.io/docs/v1.9/reference/xpkg.html#manifests" target="_blank" rel="noopener noreferrer">specification</a>.</p></blockquote><p>Crossplane parses the contents of the <code>package.yaml</code> file, caches it, then begins installing the package. If the package is a Provider, the <code>package.yaml</code> contains metadata and a stream of CRDs, <a href="https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#validatingadmissionwebhook" target="_blank" rel="noopener noreferrer">Validating Webhook Configuration</a>, and <a href="https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#mutatingadmissionwebhook" target="_blank" rel="noopener noreferrer">Mutating Webhook Configurations</a>. If the package is a Configuration, it contains metadata and a stream of Composite Resource Definitions (XRDs) and Compositions. Before applying the objects to the cluster, Crossplane first adds the package revision to a singleton cluster-scoped object of kind <code>Lock</code>. <strong>This object plays a similar role to </strong><code><strong>go.mod</strong></code><strong> and </strong><code><strong>go.sum</strong></code><strong>, defining the packages that are installed in the cluster, the dependency relationships between them, and information about their content.</strong></p>

<p>For example, installing <code>xpkg.upbound.io/upbound/provider-aws:v0.11.0</code> would result in the following entry in the singleton <code>Lock</code>, where the package revision reference, type, and name are defined. The name includes the resolved short digest of the referenced image (<code>e1dde2d0d249</code>).</p>

<p>Installing objects from a package into the cluster consists of first verifying that the given revision can obtain sole control of each object, and, if so, applying them to the cluster with a <a href="https://kubernetes.io/docs/concepts/overview/working-with-objects/owners-dependents/#owner-references-in-object-specifications" target="_blank" rel="noopener noreferrer">controller owner reference</a> to the package revision. This ensures that in a cluster where the Kubernetes API surface is <i>only expanded via Crossplane packages</i>, we can ensure the <i>only one package</i> can ever act on a given type.</p><p>This is especially important with Providers, as installing a Provider not only adds new types to the cluster, but also implements each type’s corresponding functionality in the form of a <code>Deployment</code> that runs controllers to watch and take action based on the lifecycle of instances of the installed types. Unless overridden, Crossplane and the controller image from the same package are the only system components that are given permission on any installed type.</p><h2>(Content) Addressing the Issue</h2><p>A more subtle advantage of being built on top of OCI is the usage of a content-addressable API. Just like any OCI image, Crossplane packages can be installed by digest rather than by tag (e.g. <code>v0.11.0</code>) to ensure that the content that is installed matches what was expected.</p><blockquote><p>If you don’t think pulling by digest is important, I encourage you to <a href="https://www.youtube.com/watch?v=Ve_CFyVYo3M&amp;t=1066s" target="_blank" rel="noopener noreferrer">listen to Jon Johnson</a> describe all of the reasons why you can’t trust a registry when pulling by tag.</p></blockquote>

<p>As seen below, the package revision still uses the short digest in the name (e.g. <code>e1dde2d0d249</code>), but the version corresponds to the full digest of the package image.</p>

<p>Pulling by digest guards against a number of potential issues and vulnerabilities, including tags being updated to point at a different manifest, or a malicious party intercepting requests. However, it does make utilizing semantic versioning for upgrades and dependency resolution more difficult. It would be nice if we could have the convenience of using versions when referencing packages, but ensure that the content is still what we expect, similar to the guarantees we get with a programming language like Go. Fortunately, C<strong>rossplane does not force you to choose between convenience and security</strong>. When installing a Provider or Configuration, users can choose a <code>revisionActivationPolicy</code> of either <code>Automatic</code> or <code>Manual</code>.</p>

<p>If set to <code>Manual</code>, Crossplane resolves the image tag to its digest, but waits until the revision is manually updated from <code>Inactive</code> to <code>Active</code> before continuing with installation. This allows the human to interact with human-readable identifiers (tags), while still relying on the machine to verify the content.</p>

<p>Using this strategy is particularly useful when installing or updating packages across clusters where a consistent API is desired. It also means that if package contents are lost from the cache and the package source has updated the tag to point at a different manifest, Crossplane will wait for approval before rolling to a new revision of the package.</p><blockquote><p>Combining <code>Manual</code> revision activation with <code>packagePullPolicy</code>: Always can be used in interesting ways, such as pinning to a channel tag (e.g. <code>latest</code>) and letting Crossplane create new <code>Inactive</code> revisions any time the tag is updated to point at a new manifest. It is highly recommended that users consult the <code>packagePullPolicy</code> <a href="https://crossplane.io/docs/v1.9/concepts/packages.html#specpackagepullpolicy" target="_blank" rel="noopener noreferrer">documentation</a> before enabling this functionality.</p></blockquote><p><strong>Most importantly, using a content-addressable API and a unit of installation (a package) that contains both type definitions and their corresponding reconciliation logic enables the verifiable equivalence of types across clusters we are seeking.</strong> However, though we do want to couple definition and behavior, sometimes it is advantageous to be able to verify them independently. Let’s take a look at a scenario where this would be useful.</p><h2>Case Study: Multi-Arch Bundled Provider Packages</h2><p>In the <a href="https://github.com/crossplane/crossplane/releases/tag/v1.6.0" target="_blank" rel="noopener noreferrer">v1.6.0 release</a>, Crossplane added support for <a href="https://github.com/crossplane/crossplane/pull/2758" target="_blank" rel="noopener noreferrer">bundled Provider packages</a>. Prior to this change, Provider package metadata was required to include a reference to the controller image for the types included in the package image. Bundled packages allow for omitting this reference, and instead building the package content and controller machinery into a single image. The previous system of decoupling the images had a number of advantages:</p><ol style="list-style-type:decimal;"><li>Crossplane only needed to download and cache the package contents from the package image, while the often larger controller image was only pulled by the node that the <code>Deployment</code> was scheduled to.</li><li>If <i>only</i> the content of the package changed, or only the content of the controllers changed, the digest of the other would remain the same, making it simple to determine if changes occurred to the API surface, the controller logic, or both. This is especially useful when evaluating whether an upgrade includes breaking changes.</li></ol><blockquote><p>Actually performing the verification in (2) proved to be cumbersome, as the digest of the package image would change if its controller image reference changed. The ability to c<a href="https://doc.crds.dev/github.com/crossplane/crossplane/pkg.crossplane.io/ControllerConfig/v1alpha1@v1.9.0#spec-image" target="_blank" rel="noopener noreferrer">hange the controller image reference at runtime</a> via a <code>ControllerConfig</code> somewhat mitigated this issue, but is not a recommended pattern in production settings.</p></blockquote><p>However, the flexibility of the OCI image specification, as well as the formalization of the xpkg specification in the subsequent <a href="https://github.com/crossplane/crossplane/releases/tag/v1.7.0" target="_blank" rel="noopener noreferrer">v1.7.0 release</a>, made it possible to achieve the above benefits while also reducing the complexity of managing multiple images. As previously described, Crossplane packages may annotate a descriptor with the <code>io.crossplane.xpkg</code>: base annotation to indicate that the package manager should only consider that single layer when extracting package contents. In doing so, Crossplane does not increase the amount of data it maintains in its package cache, and, importantly, can easily verify if two packages install an equivalent set of type definitions.</p><p>This functionality is useful in a number of scenarios, such as verifying whether a patch release of a Provider presents the same API as previous patches for the same minor version. One area where this is especially useful is multi-architecture Provider images. Prior to supporting bundled Provider images, there was no need to publish variants of package images for multiple platforms as they only contained YAML content. However, when the controller and package images are bundled into a single image, ensuring that the package content layer is consistent across variants for each platform is critical. It would be highly confusing if installing a package on a <code>linux/arm64</code> host resulted in different types being installed than when installed on a <code>linux/amd64</code> host.</p><p>Multi-architecture OCI images are supported by <a href="https://github.com/opencontainers/image-spec/blob/main/image-index.md" target="_blank" rel="noopener noreferrer">image indexes</a> (sometimes referred to as manifest lists). When you pull a multi-architecture image, your client is first hitting an image index, then resolving to the manifest for your host platform, before finally downloading the layers specified in that platform-specific manifest. The image index for the <code>xpkg.upbound.io/upbound/provider-aws:v0.11.0</code> image we have been using throughout this post looks like this:</p>

<p>As expected, the <code>digest</code> for the <code>linux/amd64</code> image manifest does not match the <code>linux/arm64</code> image manifest <code>digest</code>. However, if we were to examine the layers for each of these manifests, we would see the final two layers are identical:</p>

<p>The ability to both couple the definition and behavior of types (image digest), as well as decouple the verification of the definitions (<code>base</code> layer digest) is critical in ensuring the integrity of multi-architecture images. When pushing multi-architecture packages to Upbound’s registry (<code>xpkg.upbound.io</code>), different digests between the <code>base</code> layers in each variant cause packages to be rejected during verification.</p><h2>Where We’re Going</h2><p>This post outlines ways in which Crossplane enables a more robust Kubernetes type system, but the current functionality only serves as a foundation for future tooling and machinery, similar to how OCI images have helped enable more secure distribution of containerized applications, and as we’ve seen today, any content. A small sample of improvements that can be made to this system includes:</p><ul><li>Supporting installing by type, rather than by package.</li><li>Verifying controller logic on a per-type basis.</li><li>Verifying which types are not equivalent across packages, rather than just that some types are not.</li><li>Supplementing the <code>apiVersion</code> and <code>kind</code> of objects with the digest of the type and its corresponding controller logic.</li><li>Supporting <a href="https://github.com/crossplane/crossplane/issues/3048" target="_blank" rel="noopener noreferrer">package image signature verification</a>.</li></ul><p>If you are interested in joining <a href="https://www.upbound.io/">Upbound</a> in doing our part to enable this future, check out our <a href="https://upbound-www.vercel.app/blog">open roles</a>.</p><p>—</p><p>Upbound is the company that invented the open source Crossplane project. Our mission is to help customers build Internal Cloud Platforms using control planes. Every Upbound customer gets access to Upbound Universal Crossplane (UXP) and Official Providers so they can build their Internal Cloud Platforms with Crossplane confidently. UXP is Upbound’s downstream distribution of Crossplane, and Official Providers are production ready versions of Providers available exclusively to Upbound customers. Both are maintained, tested, and supported by Upbound on behalf of our customers, and are included with an Upbound subscription.</p>

Verifiable Kubernetes API Compatibility

<p>What better way to kick off the new year than to reflect on the last one? Last year, we had some huge milestones that propelled Upbound and Crossplane forward in the cloud native space. From major product releases to important community contributions, there is much to recap. Looking forward, 2024 brings a new light on innovation as we build upon the foundation of our past achievements. It's in this retrospective moment that we find the essence of inspiration, knowledge, and strength to chart our course ahead.</p><h2>2023’s Big Wins</h2><h3>Upbound Becomes Generally Available, Helping Platform Teams Scale And Simplify Crossplane</h3>

<p>Though Crossplane can help any company streamline their cloud infrastructure, customers have told us how they don’t want to spend their bandwidth managing Crossplane, especially at scale.</p><p>In April of last year, <a href="https://blog.upbound.io/upbound-general-availability">we launched Upbound</a> to let customers offload the management of their control planes to Upbound. Our Managed Control Planes (MCPs) were built from the ground up for performance, scalability, and full lifecycle management. You can depend on MCPs to scale to thousands of CRDs, allowing you to use control planes to manage resources across as many cloud services as they desire. We even released a 30-day trial with the product, which you can <a href="https://www.upbound.io/register/a">sign up for at no cost here</a>.</p><h3>New Provider Families are Released, Offering a Better Cloud Native Integration Experience for Crossplane</h3>

<p>Since we released the official providers in 2022, our coverage of the most used resources for cloud-native deployments has quickly grown to 75%. As the number of resources in each official provider has increased, it is clear that no organization needs ALL those resources in one single provider.</p><p>With that in mind, <a href="https://blog.upbound.io/new-provider-families">we split up and organized the official providers into Provider Families or groups of smaller providers</a> that more closely align to how platform teams actually use them in June. This reduced bloat, improved performance, and provided a path for future growth as the number of cloud resources grows on a seemingly daily basis. Check out the <a href="http://docs.upbound.io/knowledge-base/migrate-to-provider-families/">migration documentation</a> and read <a href="https://blog.upbound.io/new-provider-families">the full update here</a> to leverage the streamlined providers.</p><h3>Upbound Achieves SOC 2 Type II Compliance, Strengthening Trust and Security</h3>

<p>In July, we successfully <a href="https://blog.upbound.io/soc2-type2-and-trust-center">achieved SOC 2 Type II compliance</a> to showcase our unwavering commitment to ensuring the highest level of security, privacy, and data protection for our valued customers.</p><p>We also launched the <a href="https://www.upbound.io/trust-center">Upbound Trust Center</a> where companies can find a consolidated view of everything we are doing to validate our adherence to industry-recognized standards and best practices, providing peace of mind to businesses and organizations that entrust us with their critical data.</p><h3>Leading the Partner Ecosystem for Crossplane, Upbound Establishes a New Partner Program</h3>

<p>Also in July, we launched our <a href="https://www.upbound.io/why-upbound/partners">Global Upbound Partner Program</a> to support the rapidly growing ecosystem around <a href="https://www.crossplane.io/" target="_blank" rel="noopener noreferrer">Crossplane</a>. The new program enabled partners to access Upbound’s products and services along with resources, technical expertise and go-to-market opportunities.</p><p>Some notable partners are Accenture, Akuity, Aviatrix, AWS, Azure, Datadog, Google Cloud, and more. <a href="https://blog.upbound.io/upbound-partner-program">Learn more about the partner program here</a>.</p><h3>Upbound Releases Spaces, Allowing You to Bring Managed Control Planes into Your Own Environment</h3>

<p><a href="https://blog.upbound.io/announcing-spaces">We released a new feature of Upbound in September: Spaces</a>. Upbound Spaces allows teams to run their own Upbound managed control planes in any hyperscale cloud provider or data center where compliance or data sovereignty is a concern.</p><p>With Spaces, users can self-host many managed control planes in a single Kubernetes cluster running in their own environment in addition to the Upbound managed environment that we offer today. <a href="https://www.upbound.io/request-a-demo">You can get a demo of how it works here. Just say want to try out Spaces in the comment box</a>!</p><h3>Upbound Contributes Control Plane Provider Technology to Crossplane</h3>

<p>At Upbound, we are committed to supporting the open-source landscape, particularly through the contributions we make to Crossplane. As such, <a href="https://blog.upbound.io/donate-upjet-provider-project-to-cncf">we donated Upbound’s control plane provider technology</a>, <a href="https://github.com/upbound/upjet" target="_blank" rel="noopener noreferrer">Upjet</a>, to the open source Crossplane project so that it can be officially governed under the Cloud Native Computing Foundation (CNCF).</p><p>With this, we are moving the providers for AWS, Google Cloud, and Microsoft Azure to the Crossplane project. Our goal is to accelerate the growth of the ecosystem around Crossplane and remove any doubt about the viability of the project especially in light of recent licensing changes in the industry.</p><h3>Crossplane’s Biggest Release to Date, v1.14</h3>

<p>With over <a href="https://github.com/crossplane/crossplane/compare/release-1.13...release-1.14">700 commits</a>, <a href="https://github.com/crossplane/crossplane/releases/tag/v1.14.0" target="_blank" rel="noopener noreferrer">Crossplane v1.14</a> is packed full of big features that will make Platform Engineers more productive and effective in building control planes to power their infrastructure. In particular, the beta release of <a href="https://blog.crossplane.io/composition-functions-in-production/" target="_blank" rel="noopener noreferrer">Composition Functions</a> is a game changer that allows teams to programmatically configure their infrastructure with familiar languages.</p><p>The velocity has only continued to increase as the project matures. Additionally, the community has recently reached an exciting milestone by surpassing over <a href="https://twitter.com/crossplane_io/status/1717194303846641755" target="_blank" rel="noopener noreferrer">10,000 members on Slack</a>, so everything about the project continues to grow. <a href="https://blog.crossplane.io/crossplane-v1-14/" target="_blank" rel="noopener noreferrer">See all the exciting progress in v1.14 here</a>!</p><h2>Other Highlights from 2023</h2><p>As you might have seen in our numerous product update blogs, we’ve come out with a number of features to improve the Upbound and Crossplane experience. You’ll find that in only a year, we have released about 30 new product features in addition to what you’ve already read above. Here are some of the highlights:</p><h4><a href="https://blog.upbound.io/2023-11-07-product-updates">November</a>:</h4><ul><li>Improved User Flow in the <a href="https://blog.upbound.io/2023-11-07-product-updates">Control Plane Explorer</a></li><li>Filterable Control Plane list in the dashboard</li><li>New “Referenced By” section in the Resource Details drawer</li></ul><h4><a href="https://blog.upbound.io/2023-10-24-product-updates">October</a>:</h4><ul><li><a href="https://blog.upbound.io/2023-10-24-product-updates">Navigating the Upbound Marketplace faster</a></li><li>Managed control plane connector now supported in an Upbound Spaces</li><li>New capabilities in up CLI</li><li>Release of <a href="https://blog.upbound.io/2023-10-10-product-updates">Upbound Spaces v1.1</a><ul><li>Auto-scaling control planes</li><li><i>up</i> CLI compatibility with Spaces</li><li>External Secret Store support comes to Spaces</li></ul></li><li>Improved resource details drawer and events stream</li></ul><h4><a href="https://blog.upbound.io/2023-09-26-product-updates">September</a>:</h4><ul><li><a href="https://blog.upbound.io/2023-09-26-product-updates">A control plane explorer for fleet scale use</a></li><li>Use Argo CD to target managed control planes in Upbound</li><li>Offer <a href="https://blog.upbound.io/2023-09-12-product-updates">Cluster-as-a-Service to your organization with configuration-caas</a></li></ul><h4><a href="https://blog.upbound.io/2023-08-25-product-updates">August</a><span style="background-color:rgb(255,255,255);color:rgb(3,7,36);font-family:Avenir-Roman, Arial, sans-serif;">:</span></h4><ul><li>New <a href="https://blog.upbound.io/2023-08-25-product-updates">provider management experience</a></li><li><span style="background-color:rgb(255,255,255);color:rgb(3,7,36);font-family:Avenir-Roman, Arial, sans-serif;">Streamlined configuration creation flow</span></li><li>Declaratively create managed control planes</li><li><a href="https://blog.upbound.io/2023-08-11-product-updates">Control Plane Dashboard Expanded View</a> (up to 100 Control Planes)</li><li>Single Table to View and Install Configurations</li><li>Events Tab enhancements to View Crossplane Events</li></ul><h4><a href="https://blog.upbound.io/2023-07-28-product-updates">July</a>:</h4><ul><li>Viewing <a href="https://blog.upbound.io/2023-07-28-product-updates">Crossplane Events in the Upbound Console</a></li><li>Surfacing Configuration Installation Errors in Context</li><li><a href="https://blog.upbound.io/crossplane-architecture-framework">Upbound Crossplane Architecture Framework Released</a></li></ul><h4><a href="https://blog.upbound.io/2023-06-30-product-updates">June</a>:</h4><ul><li>Upbound as an <a href="https://blog.upbound.io/2023-06-30-product-updates">identity source with Azure</a></li><li>ProviderConfig CRUD (Azure)</li><li>New Azure-based configuration starters</li><li>Skippable First Run Experience</li><li>Global CRD searching in the Upbound Marketplace</li><li><a href="https://blog.upbound.io/2023-06-09-product-updates">CRUDable ProviderConfigs</a></li><li>Improved Control Plane Explorer</li></ul><p>If you’d like to read more about any of them, you can <a href="https://resources.upbound.io/product-news">see them all here</a>.</p><h3>Looking Forward to 2024</h3><p>2023 was a busy year for us and we are sure 2024 will see more huge improvements led by feedback from our Upbound customers and the Crossplane community. It is increasingly clear that teams don’t want infrastructure to stand in the way of innovation while at the same time they need to meet security, compliance, and policy requirements. We believe that Upbound and Crossplane are the best way to meet these needs and the features that will come out in 2024 will help make companies' lives easier when deploying applications in cloud-native environments.</p><h2>See It In Action</h2><p>To see all of these wins in action, try out the product for yourself! We have a free 30-day trial you can <a href="https://www.upbound.io/register/a">sign up for here</a>.</p>

Elevating Platforms: Celebrating Upbound and Crossplane Milestones of 2023

<p>Welcome to the latest Upbound Resource Roundup! Last month, we delved into a spectrum of tailored solutions. This month, we’re going to dive into a few more that will help you streamline platform team operations with Upbound's versatile capabilities powered by Crossplane. Here are some resources that can help you stay ahead in the ever-changing cloud-native landscape and achieve cloud native excellence.</p><h2>New Resources From Upbound</h2><p><a href="https://blog.upbound.io/upbound-announces-new-crossplane-reference-implementation-for-cnoe">Upbound Announces New Crossplane Reference Implementation for CNOE</a> – Upbound announced a practical application of Crossplane's capabilities through a new reference implementation for Cloud Native Operational Excellence (CNOE)! Now, you can start spinning up more resources within this powerful stack, customizing it to fit your specific needs.</p><p><a href="https://blog.upbound.io/k8s-is-not-the-platform-or-is-it-and-we-all-misunderstood">K8s Is Not the Platform – Or Is It and We All Misunderstood?</a> – Stefan Schimanski, a long-time Kubernetes expert, takes a look at Kubernetes and Crossplane as the core of a developer platform beyond container orchestration.</p><p><a href="https://blog.upbound.io/ai-ml?utm_medium=email&amp;utm_source=email-newsletter&amp;utm_term=blog">AI Made Easy: Your Guide to Fast and Reliable AI/ML Workload Deployment</a> – The AI and ML landscape of software, tools, and techniques is growing larger by the hour. Let’s take a look at how you can build a platform to get ahead in the AI/ML race.</p><p><a href="https://blog.upbound.io/2023-11-07-product-updates?utm_medium=email&amp;utm_source=email-newsletter&amp;utm_term=blog">New from Upbound: Dashboard Filtering &amp; Control Plane Explorer Flow Enhancements</a> – See new enhancements in the Upbound Console to share – from improvements to the user flow in the control plane explorer, to new filtering functionality in the control plane dashboard, to linked references in the resource details view to support debugging.</p><h2>Crossplane Highlights</h2><p><a href="https://blog.crossplane.io/crossplane-v1-14/" target="_blank" rel="noopener noreferrer">Crossplane 1.14 is here</a> – This month, the latest version of the Crossplane open source control plane framework was released. It is packed full of big features that will make Platform Engineers more productive and effective in building control planes to power their infrastructure.</p><p><a href="https://blog.crossplane.io/design-compositions-for-your-organizational-needs-with-crossplane/" target="_blank" rel="noopener noreferrer">Design Compositions For Your Organizational Needs With Crossplane</a> – This article is for curious individuals with specific requirements for building and operating cloud infrastructure efficiently along corporate guidelines, governance, security, and the breadth of systems that can be configured and resource lifecycle managed with Crossplane.</p><p><a href="https://blog.crossplane.io/platform-building-with-crossplane-your-questions-answered-at-control-plane-day-with-crossplane/" target="_blank" rel="noopener noreferrer">Platform Building with Crossplane: Your Questions Answered at Control Plane Day with Crossplane</a> – A few months ago Crossplane hosted Control Plane Day with Crossplane event. The event was a smashing success with hundreds of attendees and some outstanding talks from both maintainers and users. Here are some answers to all of the most-asked questions from the event.</p><h2>What’s Next?</h2><p>Interested in seeing these updates in action? Our team is on standby to walk you through it! <a href="https://www.upbound.io/request-a-demo">Request to have a live demo with them now</a>.</p>

Upbound Resource Roundup: November 2023

<p>Since the release of <a href="https://aws.amazon.com/lambda/" target="_blank">AWS Lambda</a> in 2014, serverless applications have gained wide adoption. But it can be a challenge to configure and manage the mix of events, code, backends, permissions, and other dependencies that make up a serverless application.</p><p>With powerful <a href="https://www.crossplane.io/" target="_blank">Crossplane</a> primitives like <a href="https://docs.crossplane.io/v1.12/concepts/#managed-resources" target="_blank">Managed Resources and Composition</a> we can manage these serverless deployments and provide internal development teams with custom Kubernetes-based API.</p><p>This blog will cover two topics: Using Crossplane to provision AWS Lambda functions, and invoking those functions in order to return data to Crossplane so that we can use it to provision other Resources.</p><h2>The Challenge</h2><p>The platform engineering team supports developers in many global regions. Each team has a primary and a backup region for their cloud infrastructure.</p><p>The proposed solution is to have an <a href="https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html" target="_blank">AWS Parameter Store</a> (SSM) to store common parameters and have Crossplane provision infrastructure based on each team’s Primary and Backup regions.</p>

<h2>Our Lambda Architecture</h2><p>To start, we’ll review the infrastructure architecture. Our application is made up of several Crossplane <a href="https://docs.crossplane.io/v1.12/concepts/composition/" target="_blank">Composite Resources</a>. To make Composite Resource available and functional in your custom API need to configure two components:</p><ul><li>An API defined by the platform team called the CompositeResourceDefinition or XRD.</li><li>One or more Compositions that implements the API defined by the XRD by provisioning Managed Resources like S3 Buckets or GKE Clusters. For the platform we are building, we have defined a number of XRDs and Compositions.<br><br>One of the ways to deploy AWS Lambda functions is by storing them in an S3 Bucket as a .zip file, so the <strong>XBucket</strong> Composition creates an S3 bucket and associated configuration settings like Encryption and BucketVersioning. This bucket will be used to store the packaged Lambda functions.<br><br>The <strong>XParameter</strong> Composition provides an API for storing values into AWS SSM, and the <strong>XFunction</strong> composition takes care of provisioning the Lambda function and associated IAM roles (for example, we have to allow our Lambda function to fetch parameters from the SSM Parameter store.</li></ul>

<h2>Storing Parameters in AWS SSM</h2><p>First, let’s look at the <strong>XParameter</strong> Composition. As part of the Composition, the platform team can define a namespace-scoped <a href="https://docs.crossplane.io/v1.12/getting-started/introduction/#claims" target="_blank">Claim</a> called <strong>Parameter</strong>, which allows teams to request Crossplane infrastructure into a Kubernetes namespace.</p><p>Our Lambda function will use the value stored in these Parameters to tell Crossplane where to install the user’s infrastructure.</p><p>In our example team-1 and team-2 use parameters to store their Primary and Backup cloud regions.</p><p>To create a new SSM Parameter, a team submits a standard Kubernetes YAML manifest to the Cluster to deploy using our <strong>Parameter</strong> Kind. In this example, team-1 configures their backupRegion to be us-west-1</p>

<p>On the cluster each team has their Parameters scoped to their namespace:</p>

<h2>Creating our Lambda Function</h2><p>We’ll use Crossplane to provision and manage our Lambda function, using a Composition called <strong>XFunctionDeploy</strong>. This Composition creates an S3 bucket and uploads our Lambda function’s python code into an Object in the created bucket. BucketVersioning is set so that any new code uploaded to the S3 bucket triggers an update of the Lambda function.</p><p>Since compositions can be nested, we use our existing <strong>XBucke</strong>t and <strong>XFunction</strong> Compositions. In this way we’re creating a reusable library of components.</p>

<h2>Giving Our Lambda Function the Right Permissions</h2><p>Our Lambda function needs to be able to connect to the SSM parameter store in order to query it. In our <strong>XFunction</strong> Composition create a Role with the following Policy and also attach the AWSLambdaBasicExecutionRole managed policy to this Role.</p>

<h2>The Lambda Function</h2><p>To make things easy to understand, we’ve created a simple Lambda function in Python that uses the <a href="https://aws.amazon.com/sdk-for-python/" target="_blank">boto library</a> to fetch SSM parameters. This function is packaged into a Zip archive and uploaded to the S3 bucket using a <a href="https://marketplace.upbound.io/providers/upbound/provider-aws/v0.33.0/resources/s3.aws.upbound.io/BucketObject/v1beta1" target="_blank">BucketObject</a>.</p>

<h2>Invoking Our Lambda Function</h2><p>To trigger our Lambda function, we need to send it a JSON payload with the parameter we are interested in fetching:</p>

<p>In Crossplane we can use an&nbsp;<a href="https://marketplace.upbound.io/providers/upbound/provider-aws/v0.33.0/resources/lambda.aws.upbound.io/Invocation/v1beta1"><u>Invocation</u></a>&nbsp;to trigger the function to run and collect the output. How does the Invocation know which Lambda function to call? One method is to use a Selector to match the Lambda function’s Kubernetes Object on the cluster:</p>

<p>When provisioned by Crossplane, the Invocation will return the output of the Lambda in its status:</p>

<p>Now that we have results, we can use the outputs in other Compositions.</p><h2>Putting It All Together</h2><p>Finally we can define a sample Application that will be created in a Composition called <strong>XApplication</strong>. The Composition consists of two Invocations that call the lambda function and store the output in the compositions status field. Those values are then passed to a VPC Managed resource to create VPCs in both the primary and backup regions.</p>

<p>The results of each function invocation is applied to the XApplication’s status field:</p>

<p>Now that Crossplane has created our VPCs, we can use kubectl’s support of&nbsp;<a href="https://kubernetes.io/docs/reference/kubectl/jsonpath/"><u>JSONpath</u></a>&nbsp;to extract the region for each VPC:</p>

<h2>Deploying the Platform</h2><p>This is the first in a series of blog posts highlighting how Crossplane and AWS Lambda work together. This next section will describe how you can install and deploy this configuration into your Kubernetes and AWS environments.</p><p>The code and examples can be cloned from <a href="https://github.com/upbound/platform-ref-lambda" target="_blank">https://github.com/upbound/platform-ref-lambda</a>, with installation instructions located in the <a href="https://github.com/upbound/platform-ref-lambda/blob/main/README.md" target="_blank">README</a> file. You’ll need a Kubernetes cluster and AWS credentials to deploy the platform</p><h2>Final Thoughts</h2><p>We hope this is a starting point for integrating AWS Lambda with Crossplane. If you’re interested in learning more or have questions, <a href="http://slack.crossplane.io/" target="_blank">join the Crossplane Slack</a> and ask us there or open an issue on the <a href="https://github.com/upbound/platform-ref-lambda" target="_blank">https://github.com/upbound/platform-ref-lambda</a> repository.</p>

Using Crossplane to Provision and Invoke AWS Lambda functions

<p>At <a href="https://akuity.io/" target="_blank">Akuity</a>, we are <a href="https://www.upbound.io/why-upbound/partners" target="_blank">collaborating with Upbound</a> to streamline our infrastructure using <a href="https://akuity.io/what-is-argo/" target="_blank">Argo CD</a> and <a href="https://www.crossplane.io/" target="_blank">Crossplane</a>. This blog post will explore how Argo CD users can implement Crossplane to manage infrastructure resources beyond Kubernetes to simplify infrastructure management and create a clear sense of the dependencies for an application.</p><h2>Introduction</h2><p>Argo CD is a prevalent tool for implementing GitOps, the practice of declaratively storing the desired state with immutable versions and using tooling to reconcile the live state, in Kubernetes. Argo CD enables GitOps by implementing the Application resource, which provides a declarative approach to managing config management tooling for Kubernetes resources (e.g., Helm, Kustomize) and a controller for handling the reconciliation loop for syncing the resources into the cluster (i.e., updating the live state to match the desired state).</p><p>A basic example of an Application looks like this:</p>

<p>This 'guestbook' Application will create a Deployment and a Service in Kubernetes based on the manifests in the `guestbook` folder of the `akuity/argo-cd-crossplane` repository.</p>

<p>What is not represented by the manifests in the repository is that the fictitious guestbook application relies on a bucket for storing images uploaded by guests. But since the bucket isn’t a Kubernetes resource, Argo CD can’t manage it without the help of another tool.</p><h2>Before a Universal Control Plane</h2><p>The existing process to create the storage bucket is to log in to the GCP console and create a new bucket, specifying its name, location, storage class, access controls, configure versioning, and the lifecycle rules. Once the bucket is created, you need to configure the application to use it.</p>

<p>This whole process is done once, with none of it represented declaratively. So the next time the application is deployed to a new environment, another bucket must be created manually. Hopefully, the same person is creating it, or the documentation is good. Otherwise, you will waste time figuring out what settings are needed for the bucket (probably by comparing it to the existing one).</p><p>The downside of this approach is that it requires the application developers to have direct access to GCP or rely on an operator to create it, and it’s a manual imperative process prone to human error. For example, the bucket access could be easily misconfigured, compromising the bucket's contents. Additionally, if the bucket configuration needs to be updated, the change must be made manually, potentially leading to inconsistencies between the desired and actual state.</p><h2>Introducing Crossplane</h2><p>Crossplane is an open-source Kubernetes-native framework for building cloud-native control planes using a consistent API without writing code. Crossplane facilitates creating resources on various providers, including GCP, AWS, and Azure, using configurations defined as Kubernetes resources, making it easier to manage infrastructure using familiar Kubernetes tooling.</p><p><a href="https://www.upbound.io/" target="_blank">Upbound</a> created Crossplane to democratize control plane technology for all and continues to drive it forward as maintainers and contributors. Upbound also offers <a href="https://www.upbound.io/product/upbound" target="_blank">commercial products</a> and <a href="https://www.upbound.io/product/universal-crossplane" target="_blank">support</a> for those organizations that want to take Crossplane even further.</p><h3>Using Crossplane to Manage the Storage Bucket</h3><p>To use Crossplane to manage the storage bucket, you must install <a href="https://marketplace.upbound.io/providers/upbound/provider-gcp/latest" target="_blank">the GCP provider</a> (`provider-gcp`), which requires authentication to a service account with permission to manage the resources on the provider. This can be done with a JSON key from a service account in GCP or using Workload Identity to authenticate based on a service account in Kubernetes.</p><p>Creating the Crossplane GCP provider in Kubernetes will also introduce all of the Custom Resource Definitions (CRDs) representing resources on GCP. Including one for managing storage buckets, `Bucket`. <a href="https://marketplace.upbound.io/providers/upbound/provider-gcp/latest/resources/storage.gcp.upbound.io/Bucket" target="_blank">The CRD</a> defines the structure of the resource and the parameters that can be configured, such as the bucket name, location, and access credentials.</p><p>A basic example looks like this:</p>

<h3>Adding the Storage Bucket to the Application</h3><p>To add the storage bucket to the Kubernetes resources managed by the Application, you need to add the `Bucket` manifest to the `guestbook` folder of the `akuity/argo-cd-crossplane` repository. Argo CD will then deploy the new `Bucket` custom resource (CR) alongside the existing Kubernetes resources, ensuring that the live state of the cluster matches the state described in the Git repository. Once Argo CD creates the CR in the cluster, Crossplane will use the provider to create the corresponding resource on GCP using all of the configurations from the `Bucket` CR.</p>

<p>This approach provides several benefits, including:</p><ul><li>creating visibility into how the bucket is configured,</li><li>reducing the likelihood of human error, and</li><li>making it easier to manage infrastructure alongside application resources using Kubernetes tooling.</li></ul><h2>Conclusion</h2><p>Using Argo CD and Crossplane together provides a powerful toolset for managing infrastructure and application dependencies in Kubernetes environments. Defining infrastructure as Kubernetes resources makes it easier to manage the entire stack using GitOps practices and Kubernetes tooling.</p><p>Resources that were once a distant and implied dependency of the application can now be managed declaratively alongside the Kubernetes-native resources. The complete configuration for the Bucket is transparent and easily reproducible. Creating a simple mechanism for deploying new instances of the application in any Kubernetes environment with Crossplane and Argo CD set up.</p>

Argo CD and Crossplane for Managing the Whole Application Stack

<h2>What if onboarding a new software vendor was as easy as adding a dependency?</h2><p>The process of onboarding a new vendor in a large enterprise is long and arduous. While organizations frequently have a uniform set of checks that any software must undergo before it can be adopted, the process inevitably leads to special cases being identified as unique to each vendor. This creates an inefficient iteration cycle that can last months — time in which the vendor could have been providing value to the customer.</p><p>If vendors were able to package their product in a way that allowed them to easily integrate into existing customer platforms, adoption could be drastically more efficient.</p><h2>Crossplane Packages and API Dependencies</h2><p><a href="https://www.crossplane.io/" target="_blank">Crossplane</a> is a framework for building control planes. Large cloud providers offer a suite of services that provide varying levels of abstraction over physical compute. To build the platform that can offer these services to thousands of customers, cloud providers have an underlying control plane that ensures the desired state of infrastructure that a customer has requested is aligned with the actual state.</p><p>If they are not already, the control plane will work to drive the two states to alignment. In simplest terms, the control plane is what translates from the interface exposed to users to actions required to satisfy their requests. A platform is the API that the control plane exposes.</p>

<p>In Crossplane, the interface, or APIs, as well as the translation logic is grouped into&nbsp;<a href="https://docs.crossplane.io/v1.10/concepts/packages/"><u>packages</u></a>. Packages that include both APIs and logic are referred to as&nbsp;<a href="https://marketplace.upbound.io/providers"><u>Providers</u></a>, while those that strictly translate from an abstract interface to a more granular one are referred to as&nbsp;<a href="https://marketplace.upbound.io/configurations"><u>Configurations</u></a>.</p><p>Much like libraries in a programming language, packages may&nbsp;<a href="https://docs.crossplane.io/v1.10/concepts/packages/#configuration-packages"><u>declare dependencies</u></a>&nbsp;on other packages and build upon their APIs. When a package is installed in a Kubernetes cluster where Crossplane is running, the package manager will evaluate its dependencies and install compatible versions of any that are not already present.</p>

<p>For example, if a user wanted to build a Configuration package that presented a developer-friendly interface to requesting MySQL databases, they may define a translation from that interface to the <a href="https://marketplace.upbound.io/providers/upbound/provider-aws/latest/resources/rds.aws.upbound.io/Instance/v1beta1" target="_blank">RDS API</a> offered by <a href="https://marketplace.upbound.io/providers/upbound/provider-aws/latest" target="_blank">provider-aws</a>, then declare a dependency on provider-aws, with optional version constraints. When their Configuration package is installed, Crossplane will ensure that provider-aws is also present.</p><p>If another user then wanted to include the MySQL API alongside other APIs in their control plane, they could either install the MySQL Configuration package directly or declare a dependency on it from another package that is installed. Either way, Crossplane will traverse the dependency tree, ensuring that a control plane with healthy packages will respond to the creation of the instance of any API with the desired behavior.</p><h2><strong>Defining a Platform with a Root Package</strong></h2><p>While Crossplane allows users to install many packages alongside one another, a common pattern has emerged of defining a single “root” package, which is an ancestor node of every other package that gets installed in a control plane. Doing so makes the entire API surface area reproducible by installing that one package. As an organization grows and evolves over time, that package can be expanded by defining new APIs or establishing new dependencies.</p>

<p>Furthermore, because using a root package is a convention rather than a technical constraint, the property of composability is not violated. Taking the previously mentioned MySQL Configuration package for example, a user may install it as the root package of their MySQL database control plane, while another user may depend on it as only one component of a much larger API surface for use cases like internal cloud platforms.</p><h2>Declaring a Vendor Dependency</h2><p>While the attributes of Crossplane packages enable platform builders to quickly add functionality to their control planes, they also present a unique distribution mechanism for vendors. When your customers all build on a common substrate, you need only target that substrate. Furthermore, the APIs you offer to your customers do not have to be “one size fits all” — there is room for customization both in front and behind the interface.</p>

<p>As an example, a developer tools vendor may offer a product to enterprise customers that manages hosted development environments. The service needs to be distributed in a manner that it can be run on the customer’s cloud provider of choice, and the interface to deploy a development environment needs to support all configuration options that a customer could need. Typically, this would require a combination of scripts, deployment manifests and documentation.</p><p>However, when bundling as a Crossplane package, a vendor can specify dependencies on external infrastructure, such as a relational database or in-memory cache and rely on the customer’s control plane to provide them.</p>

<p>To make it concrete, a Configuration package to deploy the aforementioned hosted development environment product on AWS could establish a dependency on provider-aws, and include a Composition that created an RDS instance. When the customer adds the Configuration package as a dependency in their root package, Crossplane will verify the presence of a compatible version of provider-aws in their control plane.</p><p>When the customer interacts with the API offered by the vendor package, the creation of an RDS instance object in the cluster will result in the provisioning of an RDS instance in the customer’s AWS account, subject to all of the policies and constraints they already have in place. Because the vendor’s deployment ultimately depends upon the APIs that the customer’s control plane already offers, a successful deployment implies compliance.</p><p>Similarly, the customer is not required to interface with the vendor’s product via the interface that the vendor offers. A common pattern in the Crossplane community is to take an API and wrap it in a higher-level abstraction, optionally including additional resources that are to be provisioned alongside the original.</p><p>For example, if a customer wanted to create monitoring and alerting policies that are custom to their organization every time they deployed a developer environment, they could wrap the vendor’s developer environment API and their monitoring APIs in a higher-level abstraction, and only give developers access to interact with the abstract API.</p><h2>Discovering Your Next Platform Component</h2><p>To build platforms like the ones described in this post, customers need a place to discover packages and vendors need a means of distribution. The Crossplane community takes advantage of the automatic API documentation and robust discoverability features of the <a href="https://marketplace.upbound.io/" target="_blank">Upbound Marketplace</a>. Search for your next platform component or publish your own, and feel free to reach out in <a href="https://slack.crossplane.io/" target="_blank">Crossplane Slack</a> if you have any questions!</p><p>This article was originally posted on <a href="https://thenewstack.io/crossplane-a-package-based-approach-to-platform-building/" target="_blank">The New Stack here.</a></p>

Crossplane: A Package-Based Approach to Platform Building

<p><a href="https://www.crossplane.io/" target="_blank">Crossplane</a> has fundamentally changed the game in allowing organizations to converge on a single platform for all of their needs – apps, infrastructure and everything in between. If you have not heard about Crossplane before and want to learn more about its core concepts, I recommend Pete Lumbis’ blog post “<a href="https://blog.upbound.io/__GHOST_URL__/crossplane-operators-new-bff/">Crossplane, the Operator's New Best Friend</a>.”</p><p>Those who have at least dabbled in Kubernetes and have heard about Crossplane may still find there’s a learning curve between the two. It’s common to wonder how to get started on the journey to building your own platform by designing your own APIs. <a href="https://marketplace.upbound.io/" target="_blank">Upbound Marketplace</a> is an essential tool for users embarking on their Crossplane journey, and I will demonstrate why.</p><h2>A Guided Tour of the Upbound Marketplace</h2><p>The Marketplace is a critical tool for Day-Zero Crossplane users. Before a user sets out to build their API, it’s important to first outline the shape of their API, based on inbound requirements. Let’s suppose the user has these requirements:</p><ul><li>Their organization runs predominantly on Google Cloud Platform (GCP).</li><li>Their goal is to deliver a platform that other teams can use to create VM instances that have already been configured to comply with their company policy. Other app teams don’t want to hassle with knowing which properties to set or what sizes are outside of company policy. They just need a VM to host their app</li><li>They’ve decided they’ll only ask the app team to specify whether they want small, medium or large-sized VMs. They’ll also give app teams the option of selecting a region: us-east or us-west.</li></ul><p>From these requirements, the user knows they’ll need to build API abstractions on top of GCP’s Compute Engine service (VMs). So, their first objective is to check whether there is a Crossplane provider for the cloud service they want to build on. This is where Upbound Marketplace already comes into the picture.</p><h2>Provider Discovery</h2>

<p>The Marketplace promotes a set of popular providers on the frontpage, of which the GCP Crossplane provider happens to be one. Just the same, users can also use the search bar to browse a catalog of nearly 30 Crossplane providers. If the user were to search “Google,” it should return two results: one implementation that is maintained by Upbound &nbsp;(the benefits explained in this <a href="https://blog.upbound.io/__GHOST_URL__/first-official-providers/">blog post</a>) and one maintained by the Crossplane community.</p><p>The Upbound Marketplace already has popular providers like AWS or Azure, and the Crossplane community continues building more Crossplane providers with each passing week, making it that much easier to get started with Crossplane.</p><h2>API Discovery</h2><p>If the user were to click into the Upbound official provider-gcp, they’ll be taken to a page where users can browse through what the provider has to offer. This page gives users a consolidated view to peruse through and explore the API types available for users to build on top of.</p>

<p>Thinking back to the requirements mentioned earlier, if a user is building a platform on top of Compute Engine VMs, they can use the search bar in the CRDs pane to look for “instance”.&nbsp;<i>Note: you may have to try a couple search terms. The marketplace searches based on a few fields exposed by the CRD. For example, if you search for “VM,” &nbsp;you will get no results, but “instance” gets accurate results.</i></p>

<p>If the user clicks the CRD that has an “Instance” kind under the “compute.gcp.upbound.io” API group, they’ll be able to read the description provided by the CRD and see that, “Instance is the schema for the Instances API. Manages a VM instance resource within GCE.” With confidence that they’ve found the correct base CRD to build on top of, they can now familiarize themselves with the API documentation of the CRD, especially the “spec” field.</p><p>Upbound Marketplace makes it easy to learn which fields are required objects under the “forProvider” field. This tells users these fields will need to be set when they instantiate them as part of their API abstraction. For example, scanning the fields of the Instance CRD, users can see that “boot disk,” “machineType,” “networkInterface” and “zone” are required.</p>

<p>If the required fields are an “object” type, users can recursively expand the object’s fields and see the required fields for that nested object.</p><h2>API Examples</h2><p>The Marketplace is also able to provide examples for each API. These examples are functioning .yaml configs that users could submit directly to their control plane to create instances of this CRD as a managed resource. This is a useful tool that gives users a launch pad for writing .yaml config.</p>

<h2>Discovering Configurations</h2><p>A third way the Marketplace is useful for Day-Zero Crossplane users is it can be used to discover Crossplane configurations. Many of the configurations in the Marketplace are reference implementations with permissive licenses. Configurations can either be discovered by scrolling down on the main page or using the search bar. If you use the search bar, make sure to change the filter type to “Configurations.”</p>

<p>Users may want to build an API for which there is already an implementation (like a Kubernetes cluster or PostgreSQL instance). It is completely fair practice to use these as launch pads, since they already have a number of fully completed compositions. If you click into the “platform-ref-gcp” configuration, you can explore the XRDs and compositions that make up the configuration.</p>

<p>All of these examples are but a few ways that Upbound Marketplace bootstraps platform builders’ knowledge and accelerates them down the path of getting started building with Crossplane. There’s plenty more for Crossplane users to explore with the Marketplace, like publishing your own configurations and providers. If this is one of your first times hearing about Crossplane and you want to learn more, check out the&nbsp;<a href="https://docs.crossplane.io/v1.9/"><u>getting started</u></a>&nbsp;guide or join the community on the&nbsp;<a href="http://crossplane.slack.com/"><u>Crossplane slack</u></a>.</p><p>Best,<br>Craig D Wilhite</p><p>This post was originally published on&nbsp;<a href="https://thenewstack.io/crossplane-curious-marketplace-makes-onboarding-easy/"><u>The New Stack here.</u></a></p>

Crossplane Curious? Marketplace Makes Onboarding Easy

<p>For many Crossplane users just beginning their journey, users must eventually venture off the beaten path of reference docs and Crossplane tutorials and begin crafting APIs of their own. Users may find themselves in situations where they try to submit a claim for a new, custom API they just defined to their control plane and find, “...wait, is the control plane doing <i>anything</i>?” Knowing how to debug issues with Crossplane is a crucial tool to have on your Crossplane journey. Here are three tips for Crossplane users who want to understand how to begin debugging their control planes.</p><h2>Anatomy of Crossplane resources</h2><p>It’s helpful to familiarize yourself with the moving parts that are involved in making your APIs work. If we assume users are creating claim objects as desired end state for consuming your control plane’s API, there are several objects that get created in response to the claim:</p><ul><li>Claims generate a <strong>Composite Resource (XR)</strong></li><li>The Composite Resource generates <i>at a minimum</i> either <strong>Managed Resources</strong> (one or more), <strong>nested Composite Resources</strong>, or a combination of <strong>both </strong>(both Managed Resources <i>and </i>nested Composite Resources</li><li>The Managed Resources are ultimately backed by <strong>Controllers </strong>provided by the Crossplane <strong>Provider </strong>they originate from, which must be correctly configured with a <strong>ProviderConfig </strong>object to talk to whichever external service’s API that you’re building on top of</li></ul><p>It’s worth highlighting that Composite Resources can themselves be composed of nested Composite Resources, which recurses the diagram until there are no more nested Composite Resources to unpack. Here is a diagram to illustrate:</p>

<p>Walking the tree of the diagram, we see that a Crossplane Claim generates a Composite Resource (XR). That Composite Resource will generate Managed Resources (minimum of one to many) OR it will generate nested Composite Resources OR it could do both. The faded boxes are meant to illustrate the optionality here–it ultimately depends on how a given Composite Resource is defined. When you finally unwind a Composite Resource to its foundations–that is, unwind it plus its nested XRs (if any) and you will eventually end up with only Managed Resources–each of the resulting Managed Resources are defined and implemented by a Crossplane Provider.</p><p>Having a grasp of this anatomy will pay dividends when it comes to debugging complex Crossplane resources (which Composite Resources are). For now, let’s stick to surface level troubleshooting and look at some tips for debugging simpler Crossplane resources–ProviderConfigs and Managed Resources.</p><h3>Tip #1: Verify you have a valid ProviderConfig</h3><p>The first problem area users might encounter when submitting a claim and finding that they do not deploy: make sure the ProviderConfig is correctly configured. If the ProviderConfig is misconfigured, the control plane will never be able to successfully call the external cloud service. Here are some ways ProviderConfigs can end up in a faulty state:</p><ul><li>If you are using a Service Account, it may have insufficient permissions.</li><li>The secret associated with the ProviderConfig has the wrong name or is in the wrong namespace</li><li>The secret may be misconfigured our formatted incorrectly</li></ul><p>In this blog post, I demonstrate debugging a ProviderConfig on a control plane that has <a href="https://marketplace.upbound.io/configurations/upbound/platform-ref-gcp/v0.3.0" target="_blank">platform-ref-gcp</a> installed on it. The first thing to note: Configurations and Providers can report a “healthy” and “installed” status of “true” even if the ProviderConfig is absent or misconfigured.</p>

<p>For now, the best approach to knowing whether or not you have your Provider configured correctly is to try creating a simple Managed Resource (MR) from that Provider and see if it succeeds. We’ll use Tips #2 and #3 to help us accomplish Tip #1.</p><h3>Tip #2: Use Kubernetes events to debug</h3><p>To demonstrate the first troubleshooting tip, I purposefully misconfigured my ProviderConfig and tried creating a GCP Bucket below. Notice in the screenshot below, my bucket does not show a “ready” status. Now I can demonstrate the second tip: when in doubt, grab the Kubernetes events for the Crossplane object you need to debug! I can use the following kubectl command to grab events associated with that bucket:</p><p><i>kubectl get events --field-selector involvedObject.name=upbound-bucket-9269b6d08</i></p>

<p>This command returns a helpful hint: it is reporting that my secret (which is referenced by my ProviderConfig) is not found. I am using UXP, Upbound’s distro of Crossplane, so I look for which secrets exist in the&nbsp;<i>upbound-system&nbsp;</i>namespace because Upbound’s documentation references this namespace in Get Started docs:</p>

<p>We have confirmed the secret exists.</p><h3>Tip #3: Use kubectl describe to see rendered configs</h3><p>The next thing to do is describe my ProviderConfig and see if that reveals anything. Here is another powerful troubleshooting tip: you can use “kubectl describe” on Crossplane objects to get events <i>and </i>a rendering of the submitted .yaml config. In the screenshot below, we use this command (and trim out the middle of the output for brevity):</p><p><i>kubectl describe ProviderConfig default</i></p>

<p>The culprit has been found: the ProviderConfig thinks the Secret exists in&nbsp;<i>crossplane-system</i>, but we know this is not the case (UXP is installed into&nbsp;<i>upbound-system</i>). We can fix this by updating our ProviderConfig. After doing that and getting the events of the Bucket now, we see this:</p>

<p>I purposefully misconfigured the private key for the Secret associated with my ProviderConfig. This error complains about an issue with the private key, which matches our expectation. If I fix the Secret associated with the ProviderConfig and check the status of the bucket afterwards, I will now see this:</p>

<p>Success! The resource is now showing a ready status and the event log shows the resource creation was requested successfully. If we wait a few seconds, the Bucket will be provisioned and will show up in the GCP console. This test using a simple MR revealed two separate errors, we successfully troubleshot them, and have now confirmed the Provider is correctly configured.</p><h2>Closing</h2><p>These three tips will set you down the path to troubleshooting Crossplane and squashing those pesky bugs. In a future post, we’ll explore how to use these same tips to debug more complex objects like Composite Resources. For &nbsp;now, check out the Crossplane troubleshooting resources on the <a href="https://docs.crossplane.io/v1.10/reference/troubleshoot/" target="_blank">Crossplane docs</a>. Stay tuned!</p><p>Cheers,<br>Craig D Wilhite</p>

Learning to Troubleshoot Crossplane

<p>With <a href="https://marketplace.upbound.io/providers/upbound/provider-terraform/v0.3.0" target="_blank">provider-terraform</a> we have a new Crossplane provider that runs Terraform configurations and enables platform teams to include custom Terraform code within a Crossplane powered architecture. This provider enables many integration scenarios and allows teams who have invested in Terraform to leverage a powerful control plane architecture. Please read <a href="https://blog.upbound.io/official-provider-terraform-launch/">the announcement</a> for more information. Furthermore, if you haven’t consider watching the webinar: <a href="https://resources.upbound.io/webinars/kubernetes-called-and-it-wants-your-iac-back-using-control-planes-to-modernize-your-iac-tech-stack" target="_blank">Kubernetes Called and it Wants Your IaC Back: Using Control Planes to Modernize Your IaC Tech Stack</a>.</p><p>In this blog post, we would like to explore a phased approach on how to include existing Terraform code and start migrating towards a Crossplane native solution.</p><p><strong>Example: Let developers create their own subnet</strong></p><p>Let’s say we are a platform team and are tasked to create AWS Subnets for developers. We have already spent a significant amount of time into a Terraform module, and we want to re-use that code, so developers can self-serve subnets for their application. Let’s see on how can use Crossplane to do so and port our custom code to Crossplane and possibly migrate fully to Crossplane if we'd like to.</p><h2>Overview</h2><p>We will do this in four steps:</p><ol><li>Create a common interface for developers to use</li><li>Run the full module in provider-terraform</li><li>Extract one resource (subnet) into the native AWS provider</li><li>Move the second resource into the native AWS provider</li></ol><p>The beauty of this approach is that we have a running platform after step one and can test our platform and do the subsequent steps 3 and 4 when the time is right.</p>

<p>The following steps will contain short abbreviated code examples. See the <a href="https://github.com/upbound/provider-terraform/tree/main/examples/transition" target="_blank">upbound/provider-terraform repository</a> for the complete example.</p><h3>1. Create a common interface for developers to use</h3><p>With Crossplane, a platform team offers their users a way to create a subnet by simply creating a specific Kubernetes resource. e.g. something like:</p>

<p>In order for this to materialize, we first need to create a Composite Resource Definition (or short XRD). This is Kubernetes resources that Crossplane understands. See&nbsp;<a href="https://docs.crossplane.io/v1.10/concepts/composition/"><u>Composite Resources · Docs</u></a>&nbsp;for more information. The XRD contains a spec where we can describe the interface for the developer. In this case, we want to make it easy for the developer and make him only specify the vpcName.</p>

<p>(Abbreviated. See <a href="https://github.com/upbound/provider-terraform/blob/main/examples/transition/definition.yaml" target="_blank">definition.yaml</a> for the complete version.)</p><h3>2. Run the full module in provider-terraform</h3><p>After we have created the interface, we need to create an implementation, which is called a Composition in Crossplane. In this step, we want to run the full Terraform module in this composition by using provider-terraform. As you might guess, we are creating another Kubernetes resource with the type Composition. We start by just specifying one Crossplane resource by the kind Workspace which includes the Terraform code and a patches to convert the necessary information to Kubernetes. In this example, we have in-lined all the Terraform code, but we can also specify remote resources (see <a href="https://github.com/upbound/provider-terraform/blob/main/examples/workspace-remote.yaml#L19" target="_blank">examples/workspace-remote.yaml</a> for an example.)</p>

<p><i>(Abbreviated. See&nbsp;</i><a href="https://github.com/upbound/provider-terraform/blob/main/examples/transition/01-composition-tf-only/composition.yaml"><i><u>01-composition-tf-only/composition.yaml</u></i></a><i>&nbsp;for the complete version.)</i></p><p>🎉 Success! We now have a fully running control plane with our custom Terraform logic. You can start testing it with your team.</p><h3>3. Extract one resource into the native provider</h3><p>To fully transition to Crossplane, we now want to refactor the Subnet resource to use <a href="https://marketplace.upbound.io/providers/upbound/provider-aws/v0.26.0" target="_blank">provider-aws</a>. For this, we adapt our composition to now have two resources of kind Workspace and Subnet and use patching to combine these two.</p>

<p><i>(Abbreviated. See </i><a href="https://github.com/upbound/provider-terraform/blob/main/examples/transition/02-composition-tf-and-native/composition.yaml" target="_blank"><i>02-tf-and-native/composition.yaml</i></a><i> for the complete version.)</i></p><p>You can see on how this enables us to transition to Crossplane native providers gradually. And depending on the complexity of the resources you have been developing, you can repeat this step.</p><h3>4. Move the second resource into the native provider</h3><p>In our last step, we want to remove provider-terraform from our composition and fully use the native aws provider. The outcome is for folks familiar with Crossplane straightforward. One composition that contains the two resource VPC and Subnet:</p>

<p><i>(Abbreviated. See </i><a href="https://github.com/upbound/provider-terraform/blob/main/examples/transition/03-composition-native-only/composition.yaml" target="_blank"><i>03-native-only/composition.yaml</i></a><i> for the complete version.)</i></p><h2><strong>Summary</strong></h2><p>Crossplane is a great way to build a developer platform powered by Kubernetes. Terraform has a great ecosystem and existing investments from platform teams. With provider-terraform we can combine both ecosystems and enable integration scenarios. This blogpost showed a quick get started guide on how to take a step-by-step guide behind a stable developer facing API. If you want to learn more and see the above in action, please check out the <a href="https://docs.upbound.io/providers/provider-terraform/migrate-provider-tf/" target="_blank">migration guides.</a></p>

Migrating to Crossplane With Provider-Terraform

<p>Crossplane was one of the most discussed topics of <a href="https://www.cncf.io/kubecon-cloudnativecon-events/?utm_content=inline-mention" target="_blank">KubeCon + CloudNativeCon North America</a> 2022. It was mentioned in keynotes and talks, and the surrounding chatter was all over the show floor. &nbsp;Crossplane is being deployed in production across Global 2000 organizations, and at KubeCon companies like Grupo Boticário, Adobe, and Autodesk shared how they are adopting Crossplane.</p><p>For those who might not be familiar, <a href="https://crossplane.io/" target="_blank">Crossplane</a> is a framework to build internal cloud platforms using <a href="https://thenewstack.io/gitlab-deploys-the-crossplane-control-plane-to-offer-multicloud-deployments/" target="_blank">control planes</a>.</p><h2>Control Plane-Powered Platforms</h2><p>You may be asking, “Why should I use control planes?” Let’s start with the big picture. Every bigger company has some kind of internal “cloud, DevOps, platform” organization. Let’s call them the “platform team.” They manage the operations and consumption of public clouds and internal services. After the first wave of automation with Infrastructure as Code (IaC) tools, these platform teams have realized that a pure IaC approach is not sufficient, as they often fail to address two fundamental problems:</p><ul><li>How do we govern infrastructure and services in a central place to ensure they are used efficiently and in compliance with our security and other standards?</li><li>How do we unblock development teams and enable self-service that doesn’t require deep Infrastructure as Code knowledge?</li></ul><p>Various approaches have attempted to tackle these objectives and promised self-service with a single pane of glass at reduced costs. But all the offerings have trade-offs: They lock platform teams into a particular technology or infrastructure vendor, and they are too opinionated about the layer of abstraction provided.</p>

<p>The missing ingredient here is a control plane. A control-plane-based architecture gives you a declarative approach to defining and managing resources, a continuous reconciliation to eliminate configuration drift and offer a single point of control for concerns like policy enforcement and permission management.</p><p>Which brings us back to Crossplane, an <a href="https://thenewstack.io/crossplane-a-kubernetes-control-plane-to-roll-your-own-paas/" target="_blank">open source framework</a> that runs on top of Kubernetes to build platforms using a control-plane-based architecture. Let’s see how that is actually done.</p><h2>Building a Platform with Crossplane</h2><p>Building and using a Crossplane-based platform involves three major steps: <strong>Connect</strong> to the resources you want to manage, <strong>Compose</strong> the different resources into a custom API for your team, and finally <strong>Consume</strong> those resources by developers:</p>

<h3><strong>Connect to the Cloud with Providers</strong></h3><p>The first step for a platform team is to connect to the Cloud APIs they would like to offer internally. With Crossplane, this involves creating a custom resource definition (CRD) and running a controller that makes the respective API calls. These specific CRDs are called <a href="https://crossplane.io/docs/v1.9/concepts/managed-resources.html" target="_blank">managed resources</a> and are bundled into a specific <a href="https://crossplane.io/docs/v1.9/concepts/providers.html" target="_blank">Crossplane provider</a>.</p><p>It’s been a consistent message in the community that the quality and coverage of providers is important to their adoption of Crossplane. There are hundreds of contributions and open source listings available, but it’s often unclear which ones are the highest quality, and it’s becoming increasingly more difficult to discover the latest or more relevant contributions.</p><p>It’s because of this that <a href="https://www.upbound.io/?utm_campaign=2022_Q3_EVER_GBL_The-NewStack-GENERAL&amp;utm_medium=The-New-Stack&amp;utm_source=referral&amp;utm_content=inline-mention" target="_blank">Upbound</a> created <a href="https://marketplace.upbound.io/" target="_blank">The Upbound Marketplace</a> as the one-stop shop for all things Crossplane, including official providers and other listings. It’s a great place to see what’s happening with the project. The marketplace offers users:</p><ul><li>A catered place to host Crossplane packages.</li><li>Auto-generated API documentation for packages published to it.</li><li>An easy way to find and learn about different Crossplane extensions and how to use them.</li><li><a href="https://blog.upbound.io/__GHOST_URL__/first-official-providers/">Upbound Official providers</a>: a set of Crossplane providers that are maintained, tested and supported by Upbound.</li></ul><p>As of today, the marketplace offers 1,182 cloud resources to choose from. This goes from the usual suspects of Amazon Web Services (AWS), Google Cloud Provider (GCP) and Microsoft Azure over to smaller cloud providers such as <a href="https://marketplace.upbound.io/providers/civo/provider-civo/v0.1" target="_blank">Civo</a>, <a href="https://marketplace.upbound.io/providers/digitalocean/provider-digitalocean/v0.2.0" target="_blank">DigitalOcean</a> and <a href="https://metal.equinix.com/?utm_content=inline-mention" target="_blank">Equinix,</a> but also other “special” providers such as Kubernetes, Helm or even <a href="https://marketplace.upbound.io/providers/upbound/provider-terraform/v0.6.0" target="_blank">Terraform</a> and <a href="https://marketplace.upbound.io/providers/crossplane-contrib/provider-ansible" target="_blank">Ansible</a>. Upbound has released a first set of <a href="https://blog.upbound.io/__GHOST_URL__/first-official-providers/">official providers</a> and made them available to the whole Crossplane community.</p><p>Additionally <a href="https://www.upbound.io/learn/resources/upbound-marketplace-launch-partners" target="_blank">the community</a> keeps contributing many providers. We will continue to support this effort and if you want to generate a provider for your API, please see <a href="https://github.com/upbound/upjet" target="_blank">Upjet</a> as a great tool to generate a provider.</p><h3><strong>Compose by Patching Resources</strong></h3><p>With so many services at our disposal, it’s now time to move to our next step: compose. With Crossplane, you don’t expose the raw API to developers. Instead, you provide a tailored API in what we call a <a href="https://crossplane.io/docs/v1.9/concepts/composition.html" target="_blank">Composite Resource Definition</a> (XRD). So instead of giving a developer a raw <a href="https://marketplace.upbound.io/providers/upbound/provider-aws/v0.18.0/resources/rds.aws.upbound.io/Instance/v1beta1" target="_blank">PostgresSQLInstance</a>, you set guardrails by providing a tailor-made <a href="https://marketplace.upbound.io/configurations/upbound/platform-ref-aws/v0.4.1/resources/aws.platformref.upbound.io/XPostgreSQLInstance/v1alpha1" target="_blank">XPostgreSQLInstance</a>. Now the developer can only change the options you’ve enabled and the platform team has permitted; the rest is pre-defined and ready to use.</p><p>For this new API to come into reality, you need to build a composition by configuring a set of resources and combining them. This is what we call patching, and you can see the <a href="https://marketplace.upbound.io/configurations/upbound/platform-ref-aws/v0.4.1/compositions/xpostgresqlinstances.aws.platformref.upbound.io/aws.platformref.upbound.io/XPostgreSQLInstance" target="_blank">XPostgreSQLInstance</a> as an example. Combining a set of resources is very powerful and fun (as fun as YAML goes ;-). So, the next big step for the Crossplane community is to come up with new and exciting APIs by creating some patches.</p><p>&nbsp;</p>

<p>The last patch I’ve created was a&nbsp;<a href="https://github.com/luebken/platform-example-logistics/blob/main/demo.md"><u>digital twin for a ship</u></a>. So, I’d like to challenge you to create something more fun and usable. If you are looking for a starting point, I’d like to point to our reference platforms:&nbsp;<a href="https://github.com/upbound/platform-ref-aws"><u>upbound/platform-ref-aws</u></a>,&nbsp;<a href="https://github.com/upbound/platform-ref-gcp"><u>upbound/platform-ref-gcp</u></a>&nbsp;or&nbsp;<a href="https://github.com/upbound/platform-ref-azure"><u>upbound/platform-ref-azure</u></a>. You can also start from scratch by following our quick starts from&nbsp;<a href="https://marketplace.upbound.io/providers/upbound/provider-aws/latest/docs/quickstart"><u>AWS</u></a>,&nbsp;<a href="https://marketplace.upbound.io/providers/upbound/provider-gcp/v0.30.0/docs/quickstart"><u>GCP</u></a>, or&nbsp;<a href="https://marketplace.upbound.io/providers/upbound/provider-azure/latest/docs/quickstart"><u>Azure</u></a>. And be sure to join&nbsp;<a href="https://slack.crossplane.io/"><u>https://slack.crossplane.io/</u></a>&nbsp;and ping me&nbsp;<a href="https://twitter.com/luebken"><u>@luebken.</u></a></p><p>This article was&nbsp;<a href="https://thenewstack.io/composing-a-platform-by-patching-crossplane-resources/"><u>originally posted on The New Stack.</u></a></p>

Composing a Platform by Patching Crossplane Resources

<h2>A Flexible Framework</h2><p><a href="http://crossplane.io/" target="_blank">Crossplane</a> is a popular open source Cloud Native Computing Foundation project that provides a framework to help you build your own opinionated cloud native control plane – complete with all of the policy, best practices and configuration that’s important to your organization. A vibrant and healthy community has rallied around the project as a neutral place for vendors and individuals to come together and collaborate on enabling a new approach to managing all of your resources.</p><p>What do we mean by Crossplane being a framework? Crossplane contains the logic, machinery and reusable components that can be assembled to achieve your organization’s unique vision of efficiently and reliably delivering infrastructure and applications to production. You can think of this framework as the “building blocks” for creating specialized and customized platforms.</p><p>While this framework approach provides extreme flexibility to cover a wide variety of unique use cases, it means that there is no one-size “solution” that fits everyone. &nbsp;The units of functionality offered by the Crossplane framework must be constructed into a bigger picture, and this can often be challenging to navigate without some assistance.</p><p>Let’s take some first steps together today by exploring a bit of introductory material that will jump-start your productivity with Crossplane right away.</p><h2>Critical Crossplane Concepts</h2><p>As folks new to the Crossplane community begin to learn about its key concepts, they can be confused about the responsibilities and purpose of the various architectural components. &nbsp;A really helpful mental model to make sense of the functional areas of Crossplane is a series of “layers” that build upon one another as shown in the diagram below.</p>

<p>Starting at the <strong>bottom</strong> layer in the diagram, we have:</p><ul><li><strong>Cloud provider infrastructure and services</strong> are all of the resources you think of when you go to your cloud provider of choice and browse the available services they offer, such as Amazon S3 buckets, Google CloudSQL database, Azure AKS clusters, etc. These exist in the real world outside of Crossplane.</li><li><a href="https://docs.crossplane.io/v1.10/concepts/terminology/#managed-resource" target="_blank"><strong>Managed resources</strong></a> are the next layer up and are how Crossplane brings this real world infrastructure into the control plane to expose them as extensions to the Kubernetes API. Much like Kubernetes has built-in objects for Pod, ConfigMap and Deployment, Crossplane exposes objects for S3, CloudSQL, AKS, etc.</li><li><a href="https://docs.crossplane.io/v1.10/concepts/terminology/#composite-resource" target="_blank"><strong>Composite resources</strong></a> are built on top of managed resources by the platform engineer. They compose together multiple resources along with configuration and policy into their own custom abstraction or API that is then exposed to their developers for self-service access to infrastructure.</li><li><a href="https://docs.crossplane.io/v1.10/concepts/terminology/#composite-resource-claim" target="_blank"><strong>Claims</strong></a> are the highest layer of functionality and are what the end user developers work with directly. When they need infrastructure alongside their applications, they simply create a claim and the underlying layers will ensure a real-world instance is provisioned and ready for consumption.</li></ul><p>Now that we have cleared up the relationships between the key concepts of Crossplane, we’ll leave you with another tip for successfully adopting this resource model. Crossplane is all about extending the control plane with new types that represent real-world infrastructure and services. Similarly, the composite resources that you define for your platform also extend the surface area of the control plane’s API, so it’s helpful to also think of these abstractions as first-class citizens of the API too, &nbsp;just like Pods and ConfigMaps.</p><h2>Getting Our Hands Dirty</h2><p>One of the most effective ways to start internalizing these concepts is to see them in action by trying out Crossplane for yourself. The Crossplane documentation has a <a href="https://docs.crossplane.io/v1.10/getting-started/" target="_blank">getting started guide</a>, which walks you through a practical exercise that uses all the layers of functionality we just explored together.</p><p>Let’s break down what you will have accomplished by the end of this guide:</p><ul><li><strong>Install </strong>Crossplane into your cluster to extend the control plane with infrastructure management capabilities</li><li><strong>Connect</strong> the control plane to your cloud provider of choice so it can provision and manage resources in the cloud on your behalf</li><li><strong>Define</strong> a new custom abstraction for Postgres that composes together infrastructure resources and exposes them as a simple declarative API for your developers</li><li><strong>Provision </strong>a new Postgres instance using the new API you defined, as well as an application that consumes this new database<br>&nbsp;</li></ul><p>One important yet often overlooked nuance of this simple flow is that it blends together a couple of distinct personas. Crossplane supports a strong separation of concerns <a href="https://blog.upbound.io/__GHOST_URL__/developers-and-operators-complicated-relationship/">between the platform builder and the platform consumer</a>. The builder has full permissions to configure and compose the granular resources into abstractions as part of an API, while the consumer has limited access to just these APIs, allowing them to provision new infrastructure for their applications according to the guardrails that their platform builder specified. &nbsp;It is helpful to be mindful of this separation and multiple personas when first getting your hands dirty with Crospslane within this streamlined flow.</p><h2>When it Doesn’t Work out as Planned</h2><p>Configuring cloud provider infrastructure and services can be a complicated task that doesn’t always result in success on the first attempt. &nbsp;We’ve seen new users sometimes struggle with identifying the root cause of runtime issues they encounter while building their platforms, so we wanted to share a few tips to help you get to the bottom of any problems, should they arise.</p><ol><li>Status and conditions — Crossplane has a very consistent resource model for all objects within the control plane, so you can expect every resource to have a status section that will give you up-to-date details about the most recent observed state from the real world. &nbsp;This status section should be your first place to check for errors, especially the conditions list, which can give you insight into why a resource isn’t provisioning or getting to a “ready” state.</li><li>Events tell a story — Like many well-behaved Kubernetes objects, all Crossplane objects will emit events that detail noteworthy milestones within its life cycle. &nbsp;These events will tell a story of the history of the object and often help you understand the most recent action taken and the result.</li><li>Follow the chain of references — We saw earlier that Crossplane has multiple “layers.” One of the most important troubleshooting concepts is that you sometimes need to follow the references between the layers to get to the root cause. &nbsp;For instance, starting with the developer- facing claim, you often need to check the composite resource that it references, and subsequently the managed resources that it is composed of. &nbsp;You should always try to <a href="https://docs.crossplane.io/v1.10/reference/composition/#troubleshooting-claims-and-xrs" target="_blank">follow this chain of references</a>, like peeling back the layers of an onion, to get deeper insight into the objects within lower layers of Crossplane’s layered functionality.</li></ol>

<h2>Walk, Crawl…Run?</h2><p>We’ve covered some basics about Crossplane architecture and building a very simple platform, so let’s talk about the next steps for continuing to progress on your Crossplane journey. At this point, you may have a grand vision forming in your head of how your control plane will automate the life cycle of all the infrastructure in your entire organization. That’s exciting, but it’s wise to narrow the scope down to a more practical and quickly attainable goal.</p><p>We recommend that you start with a tangible, but reasonably scoped scenario as you transition from docs and guides to your own custom platform. &nbsp;For example, you could start with a single infrastructure resource that you want to expose to your development team and get that working first before moving on to further resource needs. Or you could start with a small “spike” of functionality, such as provisioning an application cluster and a single database for those apps to store their state.</p><p>This app cluster and database “spike” is exactly what is implemented in Upbound’s <a href="https://github.com/upbound/platform-ref-aws/" target="_blank">reference platform for AWS</a>. &nbsp;In this detailed example, you can see how resources are abstracted and exposed to developers, how limited configuration details from the developer are patched down to the managed resources and how this entire platform is packaged up into a <a href="https://marketplace.upbound.io/configurations/upbound/platform-ref-aws/v0.5.0" target="_blank">Crossplane configuration</a> for reuse and distribution across multiple teams.</p><p>We encourage you to dive deeper into this AWS reference platform to get a better understanding of how to scope your first practical platform scenario, as well as see a few advanced examples of composition authoring.</p><h2>Charting a Course with the Experts</h2><p>In this article, we’ve demystified the key concepts and layers of Crossplane, learned techniques for deeper observability into the control plane’s runtime and explored the first concrete steps to starting building your own platform. As you continue defining your platform and consider how to deploy and run it in critical production environments, you’ll likely also consider how you can accelerate this effort.</p><p><a href="https://www.upbound.io/" target="_blank">Upbound</a> created the Crossplane project in 2018 with a vision to take the cloud native space to a new level. &nbsp;As the inventors of this technology, the folks at Upbound are uniquely positioned with a wealth of knowledge and understanding that enables them to effectively harness Crossplane’s capabilities into a complete solution that fits your specific needs. They are ready and more than happy to be <a href="https://www.upbound.io/contact" target="_blank">your expert guide</a> and ensure your platform is built according to best practices and fit for production demands.</p><p>If you are interested in leveraging the power of control planes without needing to worry about operational concerns, or managing the underlying infrastructure, try out Upbound for yourself. Enable SRE teams with a more efficient workflow and the building blocks of platform engineering by setting up your first cloud platform powered by Upbound’s managed control planes. &nbsp;<a href="https://upbound.io/register" target="_blank">Sign up for your 30 day free trial here</a>!</p>

<p>This was originally posted on the <a href="https://thenewstack.io/crossplane-what-most-people-get-wrong-and-how-to-get-it-right/" target="_blank" rel="noopener noreferrer">New Stack here</a>.</p>

Crossplane First Look: What Most People Get Wrong (and How to Get it Right)

<p>We’re at the cusp of a special moment in Cloud Computing. Cloud services that cover the spectrum of “-as-a-service”—ranging from the foundational “infrastructure-as-a-service” and building on up to “5G networks-as-a-service” and beyond—have cropped up and promised customers faster time-to-market and lower overhead to operate their businesses on the global stage. Now, more than ever, organizations have a wealth of options to choose from.</p><p>But with these options comes inevitable sprawl and the challenges associated with stitching services from many vendors together. While it might feel easier to go all-in on one cloud vendor, customers risk closing themselves off from innovation happening across the rest of the industry. And even within the scope of a single cloud vendor—golmany of which offer ten’s of their own services—sprawl&nbsp;<i>still&nbsp;</i>must be contended with.</p><p>Crossplane is a technology that will unlock a “Golden Age” for organizations: it is a framework for building control planes; it gives organizations the tools and knowledge required to create solutions that can consolidate teams onto a single platform (bye bye sprawl) while enabling them to stitch across whichever cloud services are desired. However, Crossplane is not a silver bullet and there is work involved. After all, the Golden Ages of antiquity did not just “happen” upon them. It took work to achieve that prosperity, and more importantly it was the discovery of certain learnings that set them down the path towards prosperity.</p>

<h2>Building in the Cloud</h2><p>The Cloud is massive and growing: In addition to large public cloud providers such as AWS, Azure, and GCP, other providers such as Cloudflare, Snowflake, Okta, Datadog, DigitalOcean, and more offer cloud services that help organizations accomplish their business goals.</p><p>The goal of the Cloud is to allow an organization to focus on building what matters most for their business. Whether that means offloading management of underlying hardware (because they’re running in virtual machines), or managing the underlying virtual machines (because their solution is built atop of “serverless” containers), organizations should move up the stack and focus on building the unique value that makes their business tick.</p><p>However, the promises of the Cloud carry challenges in complexity as usage scales upward. Organizations may find:</p><ul><li>Teams build “their own way of doing things”. The more teams in an organization, the more “bespoke” platforms crop up, which impedes knowledge sharing from one team to the next and limits engineers’ lateral mobility within an organization.</li><li>It’s hard to know whether their infrastructure is <i>really </i>up and running or whether it’s actually on fire. How can they be certain their infrastructure hasn’t drifted from their desired state?</li><li>There may be a temptation to burn it all down and start over – building a platform from scratch <i>again</i>, until the cycle repeats itself down the road.</li></ul><p>To be a bit tongue in cheek, whereas organizations were promised the rich benefits of The Cloud, they may find it takes an overwhelming amount of work to keep their bet on the Cloud from descending into the state depicted below:</p>

<h2>You need a control plane</h2><p>Control planes are a key ingredient to combatting what is described above. You need something that will:</p><ul><li>Configure</li><li>Control</li><li>Monitor</li><li>Reconcile changes</li><li>Be resilient</li><li>Offers a stable interface</li></ul><p>A control plane-based architecture gives you a declarative approach to defining and managing resources, continuous reconciliation of resources to eliminate configuration drift, and a pattern for achieving self-service.</p><p>The requirements list above may sound familiar because the industry recently similarly solved these challenges for container orchestration by consolidating around <a href="https://kubernetes.io/" target="_blank">Kubernetes</a>. But set aside containers for a moment. Kubernetes fundamentally offers some capabilities that make it ripe for being a control plane <i>in general</i>: it offers a declarative model, is self healing, offers role-based access control, has a strong ecosystem, is extensible, and offers capabilities around secrets &amp; config management.</p><p><a href="https://crossplane.io/" target="_blank">Crossplane</a> is the open-source framework for building control planes, built on the foundations of Kubernetes and extending it to allow for orchestration of <i>anything</i>. Upbound built Crossplane and contributed back to the CNCF for this purpose: to help organizations build their platforms like the cloud vendors build theirs—with control planes. It encapsulates policies, permissions, and other guardrails behind a user-defined API line that enables users to self-service without needing to become an infrastructure expert, all while eliminating configuration drift.</p><h2>Getting started with Crossplane</h2><p>Crossplane extends a Kubernetes cluster, so it must be installed into one; it can be a local cluster (like <a href="https://kind.sigs.k8s.io/" target="_blank">kind</a>) or pick your favorite managed K8s offering. Once you have Crossplane installed, <i>Crossplane Providers </i>are the first building blocks you will come across; they are bundles for a cloud service and they are the glue that allows your Crossplane control plane to talk to its associated cloud service’s API. To find what providers are available in the community today, go to the <a href="https://marketplace.upbound.io/providers" target="_blank">Upbound Marketplace</a>, which is the discovery hub for Crossplane.</p>

<p>Suppose a user wanted their control plane to have the ability to provision a pre-configured EC2 instance along with a composite monitor in Datadog for that EC2 instance. To do this, their control plane needs to talk to AWS and Datadog, so it needs the respective Crossplane providers installed into their control plane.</p>

<p>If one were to peek under the covers of a Crossplane provider, they will find pairs of Kubernetes controllers and Crossplane “Managed Resources”—think of these as Crossplane representations of resources in an external system:</p>

<p>Going back to the AWS EC2 instance + Datadog monitor scenario above, the last key piece is that Crossplane would allow this user to expose this ability as its own API–callable, repeatable, stable, and well-defined. A single API call that:</p><ol><li>abstracts away the inner-working to go set up an appropriate EC2 instance (setting its size, storage backing, whether it has an enclave, etc) and Datadog monitor (trigger window, monitor thresholds, etc)</li><li>Stitches together these two services running in different cloud environments</li></ol><p>This powerful capability of Crossplane is called <a href="https://crossplane.io/docs/v1.9/concepts/terminology.html#composition" target="_blank">Compositions</a>, and it’s what is going to really unlock the Cloud Golden Age. Compositions are your custom API. Compositions sit on top of those Managed Resources and “compose” them into higher-level abstractions. Not only are Compositions functionally powerful, they naturally enforce a great software development practice of “separation of concerns”. That is, “Platform teams” can be responsible for the API (its definition and implementation) and consumer/app teams can self-service consume the API without needing to understand its inner workings.</p>

<p>In this model with Crossplane, organizations can get started building their own internal cloud platforms on top of the growing public cloud. This is Kubernetes, after all, so standardizing on Crossplane means organizations can converge on a single platform that is flexible enough to meet the varied needs of the teams that compose their organization.</p><p><i>Note: If these topics interest you &amp; you would like to help build upon them, </i><a href="https://grnh.se/54d95d844us" target="_blank">we are hiring for various roles regarding Kubernetes and more</a><i>.</i></p><h2>Welcome to the Golden Age (Thanks Crossplane!)</h2><p>Exciting times are ahead for the Crossplane community. If you are wondering where to get started, check out Upbound’s quickstart documentation. These quickstarts use <a href="https://www.upbound.io/products/universal-crossplane" target="_blank">Upbound’s Universal Crossplane</a> (<a href="https://docs.upbound.io/uxp/" target="_blank">UXP</a>), an open-source, downstream distro of Crossplane that is commercially supported by Upbound, and is focused on Crossplane <a href="https://marketplace.upbound.io/providers?tier=official" target="_blank">Official Providers</a>, also open-source and fully supported by Upbound:</p><ul><li><a href="https://marketplace.upbound.io/providers/upbound/provider-aws/latest/docs/quickstart" target="_blank"><u>AWS quickstart</u></a></li><li><a href="https://marketplace.upbound.io/providers/upbound/provider-azure/latest/docs/quickstart" target="_blank"><u>Azure quickstart</u></a></li><li><a href="https://marketplace.upbound.io/providers/upbound/provider-gcp/latest/docs/quickstart" target="_blank"><u>GCP quickstart</u></a></li></ul><p>If you want to spend more time learning about Crossplane, I have explained the rise to the Golden Age even further in my webinar,<a href="https://resources.upbound.io/webinars/kubernetes-powered-control-planes-with-universal-crossplane" target="_blank"> “Kubernetes Powered Control Planes With Universal Crossplane”</a>. It’s a great watch if you are looking to learn more about everything you’ve read today. You can also visit the Crossplane <a href="http://crossplane.slack.com/" target="_blank">slack community</a>.</p><p>If you are interested in leveraging the power of control planes and interested to learn more about what Upbound is doing in this space, try out Upbound for yourself. Enable SRE teams with a more efficient workflow and the building blocks of platform engineering by setting up your first cloud platform powered by Upbound’s managed control planes. &nbsp;<a href="https://upbound.io/register?ref=blog.upbound.io" target="_blank">Sign up for your 30 day free trial here</a>!</p>

Unlocking the Golden Age of Cloud Computing

<p>Today we are very excited to announce a new open source project, provider-ansible, a Crossplane provider designed to extend the Crossplane scope by enabling its integration with Ansible.</p><h2>Why Ansible Crossplane Provider</h2><p>Ansible is a popular automation technology. It is widely adopted by many IT organizations nowadays to automate management for their IT systems, ranging from non cloud native to cloud native. It has a very mature community and an ecosystem with many existing automation assets that are publicly available at <a href="https://galaxy.ansible.com/" target="_blank">https://galaxy.ansible.com/</a>. In the meantime, many IT organizations have their own private Ansible assets accumulated for years to address their specific IT management requirements.</p><p>On the other hand, Crossplane, as a cloud native control plane framework, has evolved rapidly in recent years and now has become a very active CNCF project. It allows people to drive IT automation in a cloud native manner and orchestrate management automation for different varieties of IT resources using its powerful composition engine. All these can happen inside a Kubernetes cluster.</p><p>To combine the two powerful tools together and unleash the synergy, we have built a Crossplane provider for Ansible. It works as a bridge that can connect the two communities.</p><p>If you are familiar with Ansible and want to get benefits from what Crossplane provides, the Crossplane provider for Ansible is definitely a good option for you! For example, you can use this provider to drive Ansible automation inside a Kubernetes cluster rather than from the command line. By doing so, your Ansible automation can work with some modern IT operation methodology such as GitOps seamlessly. You can also use this provider to orchestrate multiple resource management automation approaches including Ansible using Crossplane’s composition mechanism. This will allow you to mix the use of many great automation assets from both two communities consistently.</p><p>This provider also opens a door to IT organizations who may have already invested lots of money on their IT systems management automation by using Ansible, including those traditional systems that do not behave cloud-natively, and want to adopt Crossplane without losing their investments. They can now reuse their existing automation assets without reinventing the wheels, specifically in some case when there’s no easy replacement for their traditional automation assets in Crossplane community in cloud native world. For example, to provision a VM in a LAN and install something on that is easy to be done by Ansible, but in Crossplane, it may need to introduce a new provider which is not necessary especially when people have some Ansible assets in place already.</p><p>In the next few sections, you will learn how the Crossplane provider for Ansible works at an architectural level in order to enable the above use cases. You will also learn how to drive Ansible execution using this provider by going through a couple of concrete examples step by step. Keep reading!</p><h2>Understanding How It Works</h2><p>The Ansible provider execution is driven by the Crossplane ecosystem.</p>

<p>In Fig.1, we see the high level architecture for the Ansible provider: it uses Kubernetes resource <i>AnsibleRun</i> to leverage Ansible content, e.g.: Playbook, Role, etc., to do the actual work. The <i>AnsibleRun</i> resource refers to the <i>ProviderConfig</i> resource which includes the required information for the proper functioning of our Ansible provider, e.g.: credentials, requirements, behavior variables, etc.</p><p>Within the Crossplane reconciliation process, the Ansible provider relies on standard Ansible binaries to execute the job: it uses the <i>ansible-galaxy </i>cli to download Ansible contents specified in the <i>AnsibleRun</i> resource using credentials from <i>ProviderConfig</i> resource, then depending on different run policy specified, it will invoke <i>ansible-runner</i> cli to execute them at different time.</p><h2>The Hello World Example</h2><p>In this section, let’s start with a very simple example to explore the basic use of Ansible provider. Our first example is the hello world Ansible playbook which will print a “Hello world!” message to the console.</p><p>To demonstrate the use of Ansible provider, let’s start a KIND cluster at first. KIND is a local Kubernetes cluster using Docker container “nodes”. To start the cluster, you need KIND CLI. For more information on KIND, please refer to <a href="https://kind.sigs.k8s.io/" target="_blank">https://kind.sigs.k8s.io/</a>.</p><p>Run below command to start the cluster:</p>

<p>Then, install Crossplane into the cluster:</p>

<p>And the Crossplane CLI, which is essentially a kubectl plugin that extends kubectl with functionality to build, push, and install Crossplane packages:</p>

<p>Then, run below command to install the Ansible provider:</p>

<p>After the provider is installed, you can run below command to verify if the installation is successful:</p>

<p>If everything works fine, you should see that both the column INSTALLED and HEALTHY for the&nbsp;<i>Provider</i>&nbsp;resource are True. For example:</p>

<div class="raw-html-embed"><table style="border:none;border-collapse:collapse;"><colgroup><col width="1578"></colgroup><tbody><tr style="height:0pt"><td style="border-left:solid #000000 1pt;border-right:solid #000000 1pt;border-bottom:solid #000000 1pt;border-top:solid #000000 1pt;vertical-align:top;padding:5pt 5pt 5pt 5pt;overflow:hidden;overflow-wrap:break-word;"><p dir="ltr" style="line-height:1.2;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:'Courier New';color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">NAME &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; HEALTHY &nbsp; REVISION &nbsp; IMAGE &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; STATE&nbsp; &nbsp; DEP-FOUND &nbsp; DEP-INSTALLED &nbsp; AGE</span></p><p dir="ltr" style="line-height:1.2;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:'Courier New';color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">providerrevision.pkg.crossplane.io/crossplanecontrib-provider-ansible-6fb5112f17a6 &nbsp; True&nbsp; &nbsp; &nbsp; 1&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; crossplanecontrib/provider-ansible:v0.3.0 &nbsp; Active &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 33m</span></p><br><p dir="ltr" style="line-height:1.2;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:'Courier New';color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">NAME&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; INSTALLED &nbsp; HEALTHY &nbsp; PACKAGE &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; AGE</span></p><p dir="ltr" style="line-height:1.2;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:'Courier New';color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">provider.pkg.crossplane.io/crossplanecontrib-provider-ansible &nbsp; True&nbsp; &nbsp; &nbsp; &nbsp; True&nbsp; &nbsp; &nbsp; crossplanecontrib/provider-ansible:v0.3.0 &nbsp; 33m</span></p><br><p dir="ltr" style="line-height:1.2;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:'Courier New';color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">NAME&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; AGE &nbsp; TYPE &nbsp; &nbsp; &nbsp; &nbsp; DEFAULT-SCOPE</span></p><p dir="ltr" style="line-height:1.2;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:'Courier New';color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">storeconfig.secrets.crossplane.io/default &nbsp; 48d &nbsp; Kubernetes &nbsp; crossplane-system</span></p></td></tr></tbody></table></div>

<p>After then, apply the following YAML manifests which include a&nbsp;<i>ProviderConfig</i>&nbsp;and an&nbsp;<i>AnsibleRun</i>:</p>

<p>In this example, we do not really need any real credential to access a remote repository that hosts Ansible roles or playbooks, so the&nbsp;<i>spec.credentials</i>&nbsp;field in&nbsp;<i>ProviderConfig</i>&nbsp;is empty. For&nbsp;<i>AnsibleRun</i>, we define the hello world playbook as an inline playbook by&nbsp;<i>spec.forProvider.playbookInline</i>. After applying these YAML manifests, you will see something similar to the following in the Ansible provider pod logs:</p>

<p>As you can see, the hello world message was printed out to the console. This is exactly the same as you see when running Ansible playbook from command line, the only difference is that you can now drive the Ansible run in a cloud native way by applying a Kubernetes custom resource and customizing the run by modifying the custom resource at runtime.</p><h2>A More Powerful Example</h2><p>In this section, we are introducing a more complex custom resource. Our example target is to create and maintain a GCP bucket.</p><p>First of all we need to create a Kubernetes secret that handles the provider credentials, GCP in this case.</p>

<p>Then we should introduce this secret in the&nbsp;<i>ProviderConfig</i>&nbsp;resource which is used to authenticate against upstream API and repositories:</p>

<p>Once we create&nbsp;<i>ProviderConfig</i>&nbsp;ressource, the last step is always to create the&nbsp;<i>AnsibleRun</i>&nbsp;resource:</p>

<p>In the above <i>AnsibleRun</i> resource, we are using a remote Ansible role hosted by a remote git repository at: <a href="https://github.com/multicloudlab/crossplane-ansible-provider-sample.git" target="_blank">https://github.com/multicloudlab/crossplane-ansible-provider-sample.git</a>, and relying on the gcpconfig as <i>ProviderConfig</i>.</p><p>By checking the above <i>AnsibleRun</i> manifest, you may also notice that it allows us to customize the run by passing variables using <i>spec.forProvider.vars</i>. As you know, Ansible uses variables to manage differences among systems on which Ansible operates, so it can run roles or playbooks on multiple systems by using a single command. Ansible provider allows you to pass those differences into Ansible run when you define the AnsibleRun resource, in this case, to customize the gcp bucket provisioning, e.g.: specify the disk size, key, and zone, etc.</p><h2>Find the Ansible Provider in the Upbound Marketplace</h2><p>Previously, when Crossplane contributors or third-party organizations wanted to introduce new providers to the community, it took a lot of effort to ensure that users were aware of its existence and knew how to find it. With the recent <a href="https://www.upbound.io/learn/resources/upbound-marketplace-resources" target="_blank">launch of the Upbound Marketplace</a>, new providers like provider-ansible can be easily discovered by anyone searching for keywords, such as “<a href="https://marketplace.upbound.io/providers?query=ansible" target="_blank">ansible</a>”. Furthermore, the Marketplace allows package authors the ability to include documentation and examples for any resource, such as <a href="https://marketplace.upbound.io/providers/crossplane-contrib/provider-ansible/v0.3.0/resources/ansible.crossplane.io/AnsibleRun/v1alpha1" target="_blank"><i>AnsibleRun</i></a>, with minimal effort.</p><p>This ease of use empowers platform teams not only to start using the Ansible provider directly, but also to build and publish their own abstractions in the form of <a href="https://marketplace.upbound.io/configurations" target="_blank">configuration packages</a> built on its primitives.</p><h2>Conclusion</h2><p>Congratulations! You have worked through all the steps needed to run an Ansible playbook or role, either inline or hosted remotely, and customize the run by defining variables. It all happens inside a Kubernetes cluster, and driven by the Crossplane Ansible provider.</p><p>If you are interested in this project and want to join with us, feel free to open issues or submit pull requests to: <a href="https://github.com/crossplane-contrib/provider-ansible" target="_blank">https://github.com/crossplane-contrib/provider-ansible.</a></p>

Crossplane Ansible Provider: Getting Started

<p>This article was <a href="https://itnext.io/gitopsify-cloud-infrastructure-with-crossplane-and-flux-d605d3043452" target="_blank" rel="noopener noreferrer">originally posted on medium</a>. In this article, we are going to learn how to automate the provisioning of cloud resources via <a href="http://crossplane.io/" target="_blank" rel="noopener noreferrer">Crossplane</a> and combine it with GitOps practices.</p><p>You will most benefit from this blog if you are a Platform or DevOps Engineer, Infrastructure Architect or Operations Specialist.</p><p><i>Note: If you are one of the above, we are hiring for various engineering roles. </i><a href="https://grnh.se/54d95d844us" target="_blank" rel="noopener noreferrer"><i>View our open roles here</i></a><i>.</i></p><blockquote><p>If you are new to GitOps, read more about it in my blog <a href="https://itnext.io/gitops-with-kubernetes-740f37ea015b" target="_blank" rel="noopener noreferrer">GitOps with Kubernetes</a></p></blockquote><p>Let’s set the stage by imagining the following context. We are working as a part of a Platform</p><p>Team in a large organization. Our goal is to help Development Teams to onboard get up to speed with using our Cloud Infrastructure. Here are a few base requirements:</p><ul><li>Platform Team doesn’t have resources to deal with every request individually, so there must be a very high degree of <strong>automation</strong></li><li>Company policy is to adopt the <strong>principle of least privilege</strong>. We should expose the cloud resources only when needed with the lowest permissions necessary.</li><li>Developers are not interested in managing the cloud, they should only <strong>consume cloud resources</strong> without even needing to login to a cloud console.</li><li>New Teams should get <strong>their own</strong> set of cloud resources for <strong>self-managing</strong> when on-boarding to the Platform.</li><li>It should be easy to provision new cloud resources on demand.</li></ul><h2>Initial Architecture</h2><p>The requirements lead us to an initial architecture proposal with the following high-level solution strategy.</p><ul><li>create template repositories for various types of workloads (using <a href="https://backstage.io/docs/features/software-templates/software-templates-index" target="_blank" rel="noopener noreferrer">Backstage Software Templates</a> would be helpful)</li><li>once a new Team is onboarded and creates the first repository from a template, it will trigger a CI pipeline and deploy common infrastructure components by adding the repository as Source to Flux infrastructure repo</li><li>once a Team wants to create more cloud infrastructure, they can place the Crossplane claim YAMLs in the designated folder in their repository</li><li>adjustments to this process are easily implemented using Crossplane Compositions</li></ul><blockquote><p>In a real world scenario we would manage Crossplane also using Flux, but for demo purposes we are focusing only on the application level.</p></blockquote><p>The developer experience should be similar to this:</p>

<h2>Tools and Implementation</h2><p>Knowing the requirements and initial architecture, we can start selecting the tools. For our example, the tools we will use are <a href="https://fluxcd.io/" target="_blank" rel="noopener noreferrer">Flux</a> and <a href="https://crossplane.io/" target="_blank" rel="noopener noreferrer">Crossplane</a>.</p><blockquote><p>We are going to use Flux as a GitOps engine, but the same could be achieved with ArgoCD or Rancher Fleet.</p></blockquote><p>Let’s look at the architecture and use cases that both tools support.</p><h3>Flux Architecture Overview</h3><p>Flux exposes several components in the form of Kubernetes CRDs and controllers that help with expressing a workflow with the GitOps model. Short description of 3 major components. All those components have their corresponding CRDs.</p>

<p><a href="https://fluxcd.io/docs/components/source/" target="_blank" rel="noopener noreferrer">Source Controller</a> main role is to provide a standardized API to manage sources of the Kubernetes deployments; Git and Helm repositories.</p>

<p><a href="https://fluxcd.io/docs/components/kustomize/" target="_blank" rel="noopener noreferrer">Kustomize Controller</a> this is a CD part of the workflow. Where source controllers specify sources for data, this controller specifies what artifacts to run from a repository.</p>

<p><i>This controller can work with kustomization files, but also plain Kubernetes manifests</i></p><p><a href="https://fluxcd.io/docs/components/helm/" target="_blank" rel="noopener noreferrer">Helm Controller</a> this operator helps manage Helm chart releases containing Kubernetes manifests and deploy them onto a cluster.</p>

<h2>Crossplane Architecture Overview</h2><p>Let’s look at what the <a href="https://blog.upbound.io/__GHOST_URL__/scaling-kubernetes-to-thousands-of-crds/" target="_blank" rel="noopener noreferrer">Crossplane</a> component model looks like. A word of warning, if you are new to Kubernetes this might be overwhelming, but there is value in making an effort to understand it. The below diagram shows the Crossplane component model and its basic interactions.</p>

<p>Learn more about Crossplane in my blog “Infrastructure as Code: the next big shift is here”</p><h3>Demo</h3><p>If you want to follow along with the demo, clone <a href="https://github.com/Piotr1215/crossplane-gitops-model" target="_blank" rel="noopener noreferrer">this repository</a>, it contains all the scripts to run the demo code.</p><h3>Prerequisites</h3><p>In this demo, we are going to show how to use Flux and Crossplane to provision an EC2 instance directly from a new GitHub repository. This simulates a new team onboarding to our Platform.</p><p>To follow along, you will need AWS CLI configured on your local machine.</p><blockquote><p>Once you obtain credentials, configure the default profile for AWS CLI following <a href="https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html#cli-configure-quickstart-config" target="_blank" rel="noopener noreferrer">this tutorial</a>.</p></blockquote><p>Locally installed you will need:</p><ul><li>Docker Desktop or other container run time</li><li>WSL2 if using Windows</li><li>kubectl</li></ul><p>Run <code>make</code> in the root folder of the project, this will:</p><p>If you are running on a Mac, use <code>make setup_mac</code> instead of <code>make</code></p><ul><li>Install <a href="https://kind.sigs.k8s.io/" target="_blank" rel="noopener noreferrer">kind</a> (Kubernetes IN Docker) if not already installed</li><li>Create KIND cluster called crossplane-cluster and swap context to it</li><li>Install crossplane using helm</li><li>Install crossplane CLI if not already installed</li><li>Install flux CLI if not already installed</li><li>Install AWS provider on the cluster</li><li>Create a temporary file with AWS credentials based on the default CLI profile</li><li>Create a secret with AWS credentials in the crossplane-system namespace</li><li>Configure AWS provider to use the secret for provisioning the infrastructure</li><li>Remove the temporary file with credentials so it’s not accidentally checked in the repository</li></ul><p>Following tools need to be installed manually</p><ul><li><a href="https://github.com/cli/cli#installation" target="_blank" rel="noopener noreferrer">GitHub CLI gh</a></li></ul><blockquote><p><strong>IMPORTANT: </strong>The demo code will create a small EC2 Instance in eu-central-1 region. The instance and underlying infrastructure will be removed as part of the demo, but please make sure all the resources were successfully removed and in case of any disruptions in the demo flow, be ready to remove the resources manually.</p></blockquote><h2>Setup Flux Repository</h2><ul><li>Create a new KIND cluster with <code>make</code>, this will install Crossplane with AWS provider and configure secret to access selected AWS account</li><li>Flux CLI was installed as part of the Makefile script, but optionally you can configure shell completion for the CLI . <code>&lt;(flux completion zsh)</code></li><li>Refer to the <a href="https://fluxcd.io/docs/installation/#install-the-flux-cli" target="_blank" rel="noopener noreferrer">Flux documentation</a> page for more installation options</li><li>Create an <a href="https://github.com/settings/tokens" target="_blank" rel="noopener noreferrer">access token</a> in GitHub with full repo permissions.</li></ul>

<ul><li>export variables for your GitHub user and the newly created token</li></ul><p><code>export GITHUB_TOKEN=&lt;token copied form GitHub&gt;</code></p><p><code>export GITHUB_USER=&lt;your user name&gt;</code></p><ul><li>use flux to bootstrap a new GitHub repository so flux can manage itself and underlying infrastructure</li></ul><blockquote><p>Flux will look for GITHUB_USER and GITHUB_TOKEN variables and once found will create a private repository on GitHub where Flux infrastructure will be tracked.</p></blockquote>

<h2>Setup Crossplane EC2 Composition</h2><p>Now we will install a <a href="https://crossplane.io/docs/v1.6/concepts/composition.html#configuring-composition" target="_blank" rel="noopener noreferrer">Crossplane Composition</a> that defines what cloud resources to crate when someone asks for an EC2 claim.</p><ul><li>setup Crossplane composition and definition for creating EC2 instances</li><li><code>kubectl crossplane install configuration piotrzan/crossplane-ec2-instance:v1</code></li><li>fork repository with the EC2 claims <code>gh repo fork https://github.com/Piotr1215/crossplane-ec2</code> and answer YES when prompted whether to clone the repository</li></ul><h2>Clone Flux Infra Repository</h2><ul><li>clone the flux infra repository created in your personal repos</li><li><code>git clone git@github.com:${GITHUB_USER}/flux-infra.git</code></li><li><code>cd flux-infra</code></li></ul><h2>Add Source</h2><ul><li>add source repository to tell Flux what to observe and synchronize</li><li>Flux will register this repository and every 30 seconds check for changes.</li><li>execute below command in the flux-infra repository, it will add a Git Source</li></ul>

<ul><li>the previous command created a file in clusters/crossplane-cluster subfolder, commit the file</li></ul><p><code>git add .</code></p><p><code>git commit -m "Adding Source Repository"</code></p><p><code>git push</code></p><ul><li>execute <code>kubectl get gitrepositories.source.toolkit.fluxcd.io -A</code> to see active Git Repositories sources in Flux</li></ul><h2>Create Flux Kustomization</h2><ul><li>setup watch on the AWS managed resources, for now, there should be none</li><li><code>watch kubectl get managed</code></li><li>create Flux Kustomization to watch for a specific folder in the repository with the Crossplane EC2 claim</li></ul>

<p><code>git add .</code></p><p><code>git commit -m "Adding EC2 Instance"</code></p><p><code>git push</code></p><ul><li>after a minute or so you should see a new EC2 Instance being synchronized with Crossplane and resources in AWS</li></ul>

<h2>Recap</h2><p>Let’s take a step back and make sure we understand all the resources and repositories used.</p>

<p>The first repository we have created is what Flux uses to manage itself on the cluster as well as other repositories. To tell Flux about a repository with Crossplane EC2 claims, we have created a <code>GitSource</code> YAML file that points to the HTTPS address of the repository with the EC2 claims.</p><p>The EC2 claims repository contains a folder where plain Kubernetes manifest files are located. To tell Flux what files to observe, we have created a <code>Kustomization</code> and linked it with <code>GitSource</code> via its name. <code>Kustomization</code> points to the folder containing K8s manifests.</p><h2>Cleanup</h2><ul><li>to clean up the EC2 Instance and underlying infrastructure, remove the claim-aws.yaml demo from the crossplane-ec2 repository</li></ul><p><code>rm ec2-claim/claim-aws.yaml</code></p><p><code>git add .</code></p><p><code>git commit -m "EC2 instance removed"</code></p><ul><li>after a commit or timer lapse Flux will synchronize and Crossplane will pick up the removed artifact and delete cloud resources</li></ul><blockquote><p>the ec2-claim folder must be present in the repo after the claim yaml is removed, otherwise Flux cannot reconcile</p></blockquote><h2>Manual Cleanup</h2><p>In case you cannot use the repository, it's possible to cleanup the resources by deleting them from flux.</p><ul><li>deleting Flux kustomization <code>flux delete kustomization crossplane-demo</code> will remove all the resources from the cluster and AWS</li><li>to clean up the EC2 Instance and underlying infrastructure, remove the ec2 claim from the cluster <code>kubectl delete VirtualMachineInstance sample-ec2</code></li></ul><h2>Cluster Cleanup</h2><ul><li>wait until <code>watch kubectl get managed</code> the output doesn't contain any AWS resources</li><li>delete the cluster with <code>make cleanup</code></li><li>optionally remove the <code>flux-infra</code> repository</li></ul><h2>Summary</h2><p><a href="https://www.youtube.com/watch?v=Y7x7Nka_d_0&amp;list=PLHhKcdBlprMdIMzUZX6ho0OPTikTamLwa&amp;index=97&amp;ab_channel=ContainerDays" target="_blank" rel="noopener noreferrer">GitOps with Flux or Argo CD and Crossplane</a> offers a very powerful and flexible model for Platform builders. In this demo, we have focused on the applications side with Kubernetes clusters deployed some other way, either with Crossplane or Fleet or Cluster API etc.</p><p>What we achieved on top of using Crossplane’s Resource Model is the fact that we do not interact with kubectl directly any longer to manage resources, but rather delegate this activity to Flux. Crossplane still runs on the cluster and reconciles all resources. In other words, we’ve moved the API surface from kubectl to Git.</p>

GitOpsify Cloud Infrastructure with Crossplane and Flux

<p>At <a href="https://www.upbound.io/"><u>Upbound</u></a> we’re building your Universal Cloud Platform. Platform teams use Upbound to build bespoke cloud APIs and cloud consoles for the application development teams they support.</p>

<h2>The Upbound Cloud Console</h2><p>At the heart of Upbound is <a href="https://crossplane.io/" target="_blank">Crossplane</a>, the open source cloud control plane we founded and <a href="https://blog.crossplane.io/crossplane-cncf-incubation/" target="_blank">donated to the CNCF</a>. Crossplane can take anything with an API (like an AWS service, an SQL server, or a Kubernetes cluster) and let you build your own declarative API for it. Perhaps as a platform engineer you want to offer your application developers access to deploy and manage their own AWS RDS instances for example, but you want a very simple, opinionated API where only storage size is configurable.</p>

<h2>Crossplane enables a simpler database provisioning API</h2><p>Upbound helps you design and deploy Crossplane-powered cloud APIs. It also offers a cloud console to help manage them. Think AWS management console, but all the services and concepts are designed by you to suit your organisation’s unique needs. We can do this in part thanks to the Crossplane Resource Model, or XRM. While Crossplane encourages you to be opinionated about the <i>content</i> of your APIs (e.g. what knobs they expose) the XRM ensures the <i>shape</i> of your APIs is always consistent. Metadata is always distinct from desired and observed state, for example, while external identity, and secret properties like credentials are always presented one way. Crossplane is built on <a href="https://kubernetes.io/" target="_blank">Kubernetes</a>, and the XRM is an extension of the <a href="https://blog.upbound.io/__GHOST_URL__/how-the-kubernetes-resource-model-enables-configuration-as-data/">Kubernetes Resource Model</a> (KRM).</p><p>A typical production API powered by Crossplane will consist of bespoke types designed by your platform team, as well as types installed by a <a href="https://crossplane.io/docs/v1.5/concepts/providers.html" target="_blank">Provider</a>. The latter are called “<a href="https://crossplane.io/docs/v1.5/concepts/managed-resources.html" target="_blank">Managed Resources</a>” (MRs) and represent external systems like an AWS service. They are the building blocks used to create the former, which we call “<a href="https://crossplane.io/docs/v1.5/concepts/composition.html" target="_blank">Composite Resources</a>” (XRs). Crossplane’s API is powered by the Kubernetes API server, which serves a <a href="https://kubernetes.io/docs/concepts/overview/kubernetes-api/" target="_blank">JSON based HTTP REST API</a>.</p>

<h2>The API Server serves a JSON REST API</h2><p>The Upbound Console’s API server queries can be expensive. Take for example a platform engineer who has defined a <code>PostgreSQLInstance</code> XR. Each <code>PostgreSQLInstance</code> is &nbsp;composed of various MRs — say an <code>RDSInstance</code>, a <code>DBSubnetGroup</code>, and a <code>SecurityGroup</code>. If there were 10 <code>PostgreSQLInstances</code> the Upbound Console would have to make 31 REST API calls to show an application developer everything there was to know about these <code>PostgreSQLInstances</code> - one call to list the 10 XRs, then three calls per XR to lookup the MRs of which they are composed. To make matters worse, the responses to these requests always include every detail about the queried resource. Even if the developer just wanted to know whether or not they were all healthy, they would be forced to retrieve the entire JSON documents. These large queries get even slower when network latency between the Upbound Console and the API Server its querying increases.</p><p>Enter <a href="https://graphql.org/" target="_blank">GraphQL</a>. GraphQL is a strongly typed query language that is frequently used to build APIs. Compared to a typical JSON REST API a GraphQL API allows the caller to define the structure of the data it requires, allowing the server to omit any irrelevant data from the response. As the name suggests, GraphQL can query a graph of related types — so in our example above it would be possible to retrieve <i>only</i> the health of our 10 <code>PostgreSQLInstances</code> and all the Managed Resources of which they are composed in a single GraphQL query. GraphQL <a href="https://graphql.org/users/" target="_blank">enjoys broad adoption</a>. Invented at Facebook a decade ago, it’s now an open standard and powers high profile APIs like those of <a href="https://docs.github.com/en/graphql" target="_blank">GitHub</a> and <a href="https://shopify.dev/api/admin-graphql#top" target="_blank">Shopify</a>. Often large organisations will build a GraphQL aggregation layer to tie their many microservices together.</p>

<h2>An example GraphQL query with xgql</h2><p>The Upbound Console team identified the benefits of GraphQL early on, but despite its broad adoption GraphQL has seen little use within the Kubernetes ecosystem. Perhaps this is due to a technology stack disconnect. While language agnostic as a standard, much of the GraphQL ecosystem favors TypeScript whereas the Kubernetes ecosystem favors Go. Nascent projects such as <a href="https://github.com/qlkube/qlkube" target="_blank">qlkube</a> and <a href="https://github.com/onelittlenightmusic/kubernetes-graphql" target="_blank">kubernetes-graphql</a> weren’t a good fit as they are primarily auto-generated wrappers around “built in” Kubernetes API types like <code>Pod</code> and <code>Deployment</code>. Upbound needed a solution to support the XRM’s more advanced type system, which has subtypes (e.g. <code>RDSInstance</code> is a subtype of Managed Resource) and dynamic types that are added and removed at runtime (e.g. by defining a new Composite Resource, or installing a Provider). We decided to build our own GraphQL representation of the XRM — a Crossplane Resource Graph.</p>

<h2>The GraphQL schema of a Crossplane Composite Resource</h2><p><a href="https://github.com/upbound/xgql/" target="_blank">xgql</a> is Upbound’s implementation of the Crossplane Resource Graph. It’s open source, and part of our <a href="https://github.com/upbound/universal-crossplane" target="_blank">Universal Crossplane</a> distribution. xgql is written in Go using <a href="https://gqlgen.com/" target="_blank">gqlgen</a>. This approach has two key benefits:</p><ol><li>gqlgen is “schema first” — it generates much of its server code from a schema.</li><li>xgql can leverage the extensive caching support of the Kubernetes Go libraries.</li></ol><p>GraphQL schemas are all about types and relationships. Designing the schema up-front was an excellent test of the XRM, forcing us to think deeply about an idiomatic GraphQL representation of concepts like Providers, Configurations, Composite Resources, Managed Resources, and the relationships between them. In fact doing so highlighted <a href="https://github.com/crossplane/crossplane/issues/2255" target="_blank">one</a> or <a href="https://github.com/crossplane/crossplane/issues/2262" target="_blank">two</a> small gaps in the XRM that we intend to address in upstream Crossplane. The resulting schema enables GraphQL queries like:</p><ul><li>“Show me the ‘Ready’ status condition for the tree of resources created by this Claim.”</li><li>“Show me all events pertaining to provider-aws’s Managed Resources”.</li><li>“Tell me whether this is a Managed Resource or a Composite Resource”.</li><li>“Fetch the definition of this Composite Resource”.</li></ul><p>Building xgql in Go allows us to use best-in-class Kubernetes client machinery. xgql maintains an in-memory “watch cache” for each unique caller, meaning it subscribes to API server updates rather than making tens or hundreds of REST API calls to resolve each GraphQL query. This often results in xgql answering queries 10x faster with a warm cache.</p><p>There’s a lot more to the Upbound Console, including another GraphQL layer that uses a novel approach to route queries to thousands of xgql instances. <a href="https://twitter.com/upbound_io" target="_blank">Let us know on Twitter</a> if you’d be interested in another post about how we leverage GraphQL at Upbound. In the meantime if you’re building on the Crossplane API do try Universal Crossplane and xgql.</p><p>Finally, we’re hiring! We’re a fully remote company, deeply invested in open source, and recently <a href="https://blog.upbound.io/__GHOST_URL__/upbound-raises-60m-in-funding-to-advance-its-universal-cloud-platform/">closed a strong Series B</a> to help us build a universal cloud platform. We have open engineering roles from frontend to distributed systems to management — <a href="https://boards.greenhouse.io/upbound" target="_blank">apply today</a>.</p>

The Crossplane Resource Graph

<p>Continuous Integration and Continuous Deployment, better known by the acronym CI/CD is the foundation of modern DevOps.</p><p>At the Crossplane Community Day Europe virtual event, Jan Willies, Platform Architect at Accenture outlined <a href="https://www.youtube.com/watch?v=KqRolAqKV_8&amp;list=PLj6h78yzYM2OFWBatWHbWgyoLNmCyAqJ0&amp;index=10" target="_blank">how to connect all the different platform pieces together for automated CI/CD</a>. Accenture is one of the world's largest consulting firms and has been actively engaged in many different cloud native efforts including Kubernetes and <a href="https://blog.upbound.io/crossplane.io">Crossplane</a>.</p><p>The first part of the CI/CD process is onboarding a new application, which can be an involved and complicated process in many organizations. Willies explained that onboarding a new application often involves lots of manual steps including finding the right settings for a Git code repository, connecting code integration systems and figuring out deployment manifests. From a platform engineering perspective, Willies said that there is a clear need and desire to automate the application onboarding process.</p><blockquote><p>"Obviously the application needs to be deployed and for the GitHub software we also need to connect the Git repo and we need to connect the execution environment," Willies said. “We can do that manually, but we can also use API's for that.”</p></blockquote><h2>Using Crossplane for Application Onboarding</h2><p>Willies explained that among Crossplane's core features is the Crossplane Provider capability that implements arbitrary API's and mirrors them into Kubernetes.&nbsp;</p><blockquote><p>"This is really helpful because we now have drivers for each of those API's and are able to configure the systems in a standardized way via Kubernetes," Willies said.<br>&nbsp;</p></blockquote><p>So for example, a YAML configuration file can be created for a given Git repository. That said, Willies noted that there is still a need to know what to put in the YAML files, since they can potentially expose everything the upstream provider based API enables. That's where Crossplane Compositions come into play.</p><blockquote><p>"A composition is a templated resource, which acts as a controller and creates a single YAML file," Willies said.</p></blockquote><p>A Crossplane Composition from a technical perspective is an XRC, a Composite Resource Claim (<a href="https://crossplane.io/docs/v0.14/getting-started/compose-infrastructure.html" target="_blank"><strong>XRC)</strong></a> which enables the composition to abstract the underlying implementation details from the assembled provider resources.</p><p>In a <a href="https://www.youtube.com/watch?v=KqRolAqKV_8&amp;list=PLj6h78yzYM2OFWBatWHbWgyoLNmCyAqJ0&amp;index=10" target="_blank">demo</a>, Willies showed how it was possible to easily onboard a new application using Crossplane. The demo involved creating a GitLab project, an Argo CD repository and an Argo CD application as well. The providers for<a href="https://github.com/crossplane-contrib/provider-gitlab" target="_blank"> <strong>GitLab</strong></a> and ArgoCD are all available in the Crossplane GitHub repository for anyone to use and create their own onboarding approach.</p><blockquote><p><span style="background-color:rgb(255,255,255);color:rgb(3,7,36);font-family:Avenir-Roman, Arial, sans-serif;">"All of this is open source," Willies said.</span></p></blockquote>

Connecting the Platform Pieces for Automated CI/CD

<p>Organizations and users find <a href="https://blog.upbound.io/__GHOST_URL__/composing-cloud-services-for-fintech-apps/www.crossplane.io">Crossplane</a> in different ways. For Kelly Ferrone, DevOps and software engineer at <a href="https://blog.upbound.io/__GHOST_URL__/composing-cloud-services-for-fintech-apps/www.akirix.com">Akirix</a>, the challenge he had was that another team relied on a Kubernetes service that had been deprecated and he needed a solution - fast.</p><p>Ferrone outlined how he and his company found Crossplane during a session at the Crossplane Community Day event, in December 2020. <a href="https://blog.upbound.io/__GHOST_URL__/composing-cloud-services-for-fintech-apps/www.akirix.com">Akirix</a> is a global payment platform that allows buyers and sellers from around the world to conduct secure transactions. The development team was using a Kubernetes project called the service broker that had become deprecated. While that project went away, there was a need for another tool for Ferrone and his team to help manage the company's Amazon Web Services (AWS) infrastructure and synchronize it with all of the services in Kubernetes that Akirix uses.</p><p>"Throughout the community a number of people suggested Crossplane as an alternative," Ferrone said. "I tried it out and I haven't looked back since and it has been great."</p><h2>The Akirix Use Case</h2><p>Ferrone explained that his team is replatforming a monolithic app with Kubernetes. He noted that overall Kubernetes has made it alot easier to maintain, as did other AWS services that helped to enable the deployment.</p><p>"As we are a smaller shop, I had to maintain alot and I wanted to have a consistent interface for managing Kubernetes and AWS simultaneously and have it all in one project which I was able to do with Crossplane," Ferrone said. "Now all from Kubernetes, AWS is maintained, as is our app and it greatly simplified everything."</p><h2>Attractive Features in Crossplane</h2><p>Ferrone primarily uses Crossplane to enable setting up a type of account in Akirix he called an IRSA account—which is basically an Identity and Access Management (IAM) role for service accounts.</p><p>"We heavily use the IAM section of Crossplane and then map that out to service accounts and other common Kubernetes components like external DNS, cluster auto-scaler and various load balancers that all want permission to access AWS," Ferrone explained. "Crossplane has made that quite easy and very stateful and I can keep it all backed up as YAML files in a git repository."</p><p>Ferrone is also making use of Crossplane Compositions. He noted that his team has been using Compositions to roll out new services including Redis and MySQL data services.</p><p>As part of his session Ferrone gave a brief demo showing his use case for IAM roles using Crossplane. Among the steps he demonstrated was using a customized plug-in to inject various variables into the YAML, based on different stacks and different environments, as well as the provider configuration. The end result is that the account has YAML backed up in a git repository and it then simply takes one command to launch all of the YAML into the cluster. "Thanks to Crossplane it makes it very easy to manage, look at and view. And it makes it pretty fun too," Ferrone said.</p>

Composing Cloud Services for Fintech Apps

<h4><span style="background-color:rgb(255,255,255);color:rgb(3,7,36);font-family:Avenir-Roman, Arial, sans-serif;">Upbound Founder and CEO, Bassam Tabbara, announces Crossplane 1.0 and details the promise and vision of Crossplane</span></h4>

<p>Bassam Tabbara, Founder and CEO of Upbound, opened <a href="https://events.crossplane.io/crossplane-community-day-modernizing-with-api-centric-control-plane-post" target="_blank">Crossplane Community Day</a> with the history and vision for <a href="https://crossplane.io/" target="_blank" rel="noopener noreferrer">Crossplane</a><a href="https://crossplane.io/" target="_blank"> </a>and how control planes are leading the transformation of IT modernization.</p><p>2020 clearly brought into focus the importance of IT modernization and digital transformation. Through the challenges of year, with resources constrained and teams working remotely, organizations of all sizes felt the urgent need to increase productivity and accelerate the pace of innovation. A number of common challenges were cited by Tabbara, including the need to support multiple environments like hybrid and multi-cloud deployment, while still having strong controls for policy and compliance.</p><p>While cloud is a cornerstone of IT modernization, managing multi-cloud resources can be complex and there has been a shift in the way organizations use the cloud.</p><p>"What we're seeing is essentially a shift from a DevOps culture where people are accessing the cloud directly," Tabbara said. "We're seeing the rise of platform teams."</p><p>The platform teams essentially offer a set of shared services across cloud infrastructure that abstract the underlying complexity of the cloud provider's back end. Cloud providers themselves have pioneered an approach for enabling easy access to service management with the use of control planes. Tabbarra explained that when a user is provisioning a virtual machine instance or a storage bucket or anything else in the cloud, it all goes through the control plane. It acts as a single point of entry to the cloud and is responsible for storing configuration, lifecycle management and policy for security auditing as well as compliance.</p><p>"Essentially what motivated us to create Crossplane is we wanted to bring that same approach that has caused the cloud to become the largest platform in the world, bring that to your organization, to the enterprise, to the open source community," Tabbara said. "We wanted platform teams to actually use control planes to unlock the next level of efficiency and productivity."</p>

<h2>The Core Vision of Crossplane</h2><p>Tabbara explained that Crossplane has two primary functionalities. The first is to expose a single universal API for the cloud, which is a Kubernetes-style API, that enables users to address infrastructure and services across multiple vendors in a consistent approach.</p><p>The second is to enable developers to create their own platform APIs and do that without having to write code.</p><p>"You can expose your own catalog of services, you can capture your guardrails configuration automation policy, all of that lives behind the API, &nbsp;ready for your application teams to consume," Tabbara said.</p><h3>The True Power of Kubernetes</h3><p>Crossplane is based on the Kubernetes control plane, which is often just associated with controlling container resources.</p><p>"I think when we look back five years from now at Kubernetes, I think we will see that its true power is not just container orchestration," Tabbara observed. “It's the fact that it's a framework for control planes.”</p><p>Kubernetes pioneered the model where you essentially use an API approach to manipulate desired state and a document store. Then a set of asynchronous controllers are able to take that state and essentially reconcile it with the real world. Tabbara noted that the control plane approach has worked really well for Kubernetes in the container space but it's actually widely applicable.</p><h2>How Crossplane Expands on the Kubernetes Control Plane</h2><p>One of the first challenges that Crossplane had to address was how to bring various resources exposed by different cloud vendors, into the same control plane. What the Crossplane project has done is invested in creating a set of providers that bring the different resources under the same Kubernetes style API.</p><p>"We created the Crossplane Resource Model which is an extension of the Kubernetes Resource Model, " Tabbara said. “It gives us consistency across the different vendors when you're provisioning resources.”</p><p>With the Crossplane Resource Model (XRM) users will get the same style of resources across different cloud providers. Tabbara noted that the project has worked with multiple vendors to create an entire ecosystem of Crossplane Providers. Among the cloud platforms that have enabled Crossplane Providers are AWS, Microsoft Azure, Google Cloud and Alibaba Cloud. At the Community Day Event Tabbara was also pleased to announce that IBM Cloud and Equinix have recently made their Crossplane Providers available as well.</p><h2>Crossplane Composition, Configurations and Registry</h2><p>Beyond the XRM there are multiple layers that build out the control plane vision that Crossplane enables.</p><p>Tabbara explained that with Composition, Crossplane users can create their own APIs. Crossplane has a concept known as Composite Resource Definition(XRD) which is similar to a Kubernetes Custom Resource Definition (CRD). An XRD is an abstracted API that developers use while Compositions are implementations of these APIs using templates or code.</p><p>Another layer that Crossplane provides is with Configuration which provides blueprints for comprehensive platform configurations.</p><p>"This enables you to create fully reproducible environments, made up of XRDs, policies of configurations of your managed resources, all packaged in essentially, a container that represents your platform," Tabbara said.</p><p>Finally there is also a Registry that contains a set of reference platforms and configurations for them. Over the next six months, Tabbara expects that there will be 20 to 30 reference platforms in the Registry that cover a wide range of scenarios including serverless, virtual machines and AI/ML workloads.</p><h2>Two Years of Progress</h2><p>The Crossplane project is only two years old and has made tremendous progress in a short period of time.</p><p>"We actually just released Crossplane 1.0 which is a massive milestone," Tabbara said. "The project is being used in production by a number of companies today and we believe it's ready to be deployed in production in every organization out there."</p><p>The second Crossplane Community Day was held December 15, 2020 and you can watch the complete replay on <a href="https://www.youtube.com/playlist?list=PL510POnNVaaZx1QDIn_-77igM1COuHaHV" target="_blank">Youtube</a>, as well as read the full wrap-up report. Ready to dive deeper? Join the Crossplane community on <a href="https://slack.crossplane.io/" target="_blank">Slack</a>, follow on <a href="https://twitter.com/crossplane_io" target="_blank">Twitter</a> or get started on <a href="https://github.com/crossplaneio/crossplane" target="_blank">GitHub</a>.</p>

Crossplane Community Day Keynote: Modernizing IT with Control Planes

<p>In this webinar we show you how easy it is to lay the foundations of your own Kubernetes PaaS in just 10 minutes with Crossplane and Upbound Cloud.</p>

<p>Here are some highlights:</p><p><a href="https://youtu.be/5pNAEvkxTfU?t=425" target="_blank">07:05</a> - Why enterprises are adopting cloud native, and so many of them are trying to build their own Kubernetes PaaS</p><p><a href="https://youtu.be/5pNAEvkxTfU?t=705" target="_blank">11:45</a>- How you can declare, instantiate, and run your own PaaS with Crossplane and Upbound Cloud</p><p><a href="https://youtu.be/5pNAEvkxTfU?t=803" target="_blank">13:23</a> - What your Platform looks like using Crossplane YAML</p><p><a href="https://youtu.be/5pNAEvkxTfU?t=906" target="_blank">15:06</a> - Defining “EKS with batteries included” using Crossplane’s Composition feature</p><p><a href="https://youtu.be/5pNAEvkxTfU?t=1213" target="_blank">20:13</a> &nbsp;Packaging up your Platform Configuration to Upbound Registry</p><p><a href="https://youtu.be/5pNAEvkxTfU?t=1602" target="_blank">26:42</a>- Operating your Platform from Upbound Cloud</p><p><a href="https://youtu.be/5pNAEvkxTfU?t=1683" target="_blank">28:03</a> - Upbound Cloud team management</p><h2>Why is this a big deal?</h2><p>In order to adopt cloud native technologies and practices, many enterprises have chosen to build their own Kubernetes PaaS. Their motivated by reducing the costs of running applications in the cloud and increasing developer productivity.</p><p>This is because <strong>containers are cheaper and faster to run than VMs</strong> and microservices lend themselves well to <strong>modern Heroku-like development practices</strong>.</p><p>The fastest way to start your cloud native journey is by purchasing an off-the-shelf enterprise Kubernetes platform or vendor solution who specializes in deploying Kubernetes</p><p>Unfortunately, deploying Kubernetes this way is typically too constraining (can’t pick your own service mesh for example) or slow (changes need to be made by a third party). So while the solutions are enterprise ready, <strong>they’re not agile or customizable</strong>.</p><p>So that’s left enterprises with one choice; building their own Kubernetes PaaS which includes infrastructure configuration, policy automation, and abstraction for easy self service by app teams.</p><p>Typically this is a 1-3 year process with 5-10 SREs working full time on the problem, costing millions of dollars. Thankfully, Upbound can help you dramatically accelerate this effort using Crossplane and Upbound Cloud.</p><p>In 10 minutes, we can define “EKS with batteries included” in a Crossplane Platform Configuration, a single YAML repository which describes desired AWS networking setup and how your EKS clusters need to be configured with monitoring and service meshes upon creation.</p><p>Then we use our package manager to build the Platform Configuration and upload it to Upbound Registry where we can instantiate it inside of Upbound Cloud.</p><p><strong>With Upbound Cloud, your PaaS comes to life</strong>. You get a single management interface into all the infrastructure defined in your Platform Configuration. You can see the status of the object inside of Crossplane as well as the actual EKS cluster health in AWS.</p><p>The best part is Upbound Cloud lets you take that infrastructure and put it inside of <strong>self service consoles</strong> for teams of your choosing to have access to. So when your app teams login to Upbound Cloud for the first time, they’re welcomed to what looks and feels like their very own personalized cloud console.</p><p>Now whenever one of the cloud service providers releases a new service, you can just update your Crossplane Platform Configuration, and enable access to the service in Upbound Cloud for the teams who need it. The new service automatically adheres to the policies you’ve previously defined. No more having to do a security audit, update policy handbooks, and train your teams how to use the services responsibly without running up the cloud bill.</p><p>If all of this sounds interesting to you, we’d love to talk. We’re hosting free PaaS consultations for organizations looking to learn how to better setup and configure their Kubernetes clusters running in the cloud. You can setup some time with us at info@upbound.io.</p><h2>Learn more</h2><p>We're excited to get your feedback on the <a href="https://upbound.io/register" target="_blank">Upbound Cloud Community Preview</a>, so drop by and say hi — we'd love to hear from you!</p><ul><li><a href="https://upbound.io/" target="_blank">Website</a></li><li><a href="https://twitter.com/upbound_io" target="_blank">Twitter</a></li><li><a href="https://github.com/upbound" target="_blank">GitHub</a></li><li><a href="https://www.linkedin.com/company/upbound-io/" target="_blank">LinkedIn</a></li><li><a href="https://www.youtube.com/channel/UCm_v2HL0pdqtShHD-ZDDTaA" target="_blank">YouTube</a></li><li><a href="mailto:info@upbound.io">Email</a></li></ul>

Webinar: Kubernetes with Managed Services in 10 Minutes

<p>As another Kubecon + CloudNativeCon draws near, we here at Upbound are making our final preparations and getting pumped to travel to Chicago for all the festivities next week! Kubecons really do make for some of the highlights of our year as we get the opportunity to connect with a ton of amazing people from the ecosystem, as well as share all of our exciting recent progress that we’re very proud of. We’re still feeling a bit of the <a href="https://blog.upbound.io/wrapping-up-kubecon-amsterdam">buzz from Kubecon Amsterdam</a>, but we’re ready to top it off again!</p><p>Chicago is a great city to host us next week. In addition to the conference content and experiences, I personally will be aiming for more than just one dinner of Chicago-style deep-dish pizza 🍕. Unfortunately, I can’t work it into my schedule to also catch a show at <a href="https://www.secondcity.com/chicago-shows/" target="_blank">The Second City</a>, but I highly recommend their hilarious brand of comedy to any folks that can make it next week alongside everything else going on! Okay, back to conference content…</p>

<h2>Our Experts Have a Lot to Say</h2><p>There are <a href="https://www.upbound.io/resources/events/kubecon-chicago-2023?utm_campaign=2023_Q4_NALA_CEV_KubeCon-Chicago#session" target="_blank">multiple sessions</a> being delivered by Upbounders next week, all of which will make for very interesting material and experiences. Tuesday will be a bit hectic with multiple sessions going on, but we are always ready to rise to the challenge to share great content with you all! We’re even hoping to see our recent Crossplane project updates on the big stage for the Keynotes, so keep an eye out for that! Other talks we’re involved in that you should not miss include:</p><ul><li><strong>Crossplane Contribfest @ </strong><a href="https://sched.co/1R2mW" target="_blank">Tues Nov 7 11:00 </a>AM - Write code hands on alongside the Crossplane maintainers (more below)</li><li><strong>API Machinery maintainer track</strong> @ <a href="https://sched.co/1R2my" target="_blank">Tues Nov 7 11:55</a> AM- Learn more about the internals of the Kubernetes API with our very own control plane expert Stefan Schimanski</li><li><strong>Crossplane Introduction and Deep-dive</strong> @ <a href="https://sched.co/1R2ne" target="_blank">Tues Nov 7 2:30</a> PM- The maintainers of Crossplane will provide an overview of Crossplane for newcomers and then dive deeply into some of the most impactful new features like Composition Functions</li><li><strong>Streamlining Infrastructure with Crossplane: a Transformation Story</strong> @ <a href="https://sched.co/1R2p8" target="_blank">Tues Nov 7 5:25</a> PM- Hear the details, obstacles and successes of the Crossplane adoption journey of <a href="https://consensys.io/" target="_blank">Consensys</a>, along with my explanations of foundational Crossplane features to up your control plane game</li><li><strong>Introducing The BACK Stack!</strong> @ <a href="https://sched.co/1R2uS" target="_blank">Thurs Nov 9 2</a> PM- David Murphy will be walking everyone through a new architecture and practical guidance for building internal developer platforms that utilizes 4 key cloud native technology projects (of course the ‘C’ stands for Crossplane!)</li></ul><h2>Empowering New Crossplane Coders</h2><p>I’m particularly excited about the Crossplane Contribfest session that we’ll be running next week. We did the first ever Contribfest in Amsterdam earlier this year and we’ve got a few ideas to improve the experience even more. This will be such a fantastic opportunity to connect with the community in a more intimate setting and put our heads together to write new code for Crossplane. We’ll be collaborating on writing <a href="https://docs.crossplane.io/v1.14/concepts/composition-functions/" target="_blank">Composition Functions</a> to teach Crossplane new tricks and accelerate the building of your control planes.</p><p>We only get 90 minutes together, so it will be chock-full of “roll up the sleeves” hands-on learning, so bring your laptop and get ready to code with us!</p><h2>Come by for a Valuable Chat</h2><p>As usual, we’ll be hosting two booths in the Solutions Showcase - one Crossplane focused booth that is part of the <a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/program/project-engagement/#project-pavilion" target="_blank">CNCF’s Project Pavilion</a> and an Upbound booth focusing on our products and solutions that are ready to scale with your production ready Crossplane needs. These booths are seriously where some of our best connections with the ecosystem are formed.</p><p>If you’re a Systems Integrator, Reseller or Software Vendor interested in working with Upbound to deliver Crossplane-based solutions, you can <a href="https://calendly.com/rob_upbound/upbound-partnerships-kubecon" target="_blank">connect with our partner team here</a>.</p><p>I can’t stress enough how valuable all the feedback and insight truly is from everyone stopping by and sharing their experiences with us in person. It’s not an exaggeration to say that all that data impacts our roadmap and helps shape what features we prioritize delivering.</p><p>So we highly encourage you to stop by, share your thoughts, and get your questions answered as well! I hear there’s a really cool new popsicle neon sign that will show you the way to our booths…</p><h2>See you in Chicago</h2><p>We certainly recognize its a privilege to be able to travel to Chicago next week, so for all the Crossplane and Upbound fans around the globe that can’t make it, we not only hope to see you soon too, but we invite you reach out virtually on <a href="http://slack.crossplane.io/" target="_blank">Crossplane Slack</a> and the <a href="https://www.upbound.io/contact" target="_blank">Upbound website</a> - we’d love to hear from you.</p><p>For folks fortunate enough to travel next week, there are a lot of opportunities to connect with us at Kubecon Chicago, and we encourage you to take advantage of as many as you can! Our <a href="https://www.upbound.io/resources/events/kubecon-chicago-2023" target="_blank">Kubecon Chicago page</a> has details to find us, including an exciting happy hour event and directly booking a meeting to sit down with us away from all the chaos. As we said, LOTS of opportunities to see each other!</p><ul><li><a href="https://www.upbound.io/resources/events/kubecon-chicago-2023" target="_blank">Upbound talks, events, and opportunities to connect at Kubecon Chicago</a></li></ul><p>With your help, the Upbound team is ready to make this the best Kubecon + CloudNativeCon yet. We hope to see you next week! 🍕🌬️🏙️</p>

Upbound at Kubecon + CloudNativeCon in the Windy City

<p>HashiCorp recently announced that they are changing the licenses on their open source projects from MPL to BSL including the Terraform project. At Upbound we understand and respect HashiCorp’s decision to change to a BSL license. This licensing change does not impact Upbound’s use of Terraform. <strong>As Upbound is a maintainer of Crossplane and manages the open source project's use of Terraform, this licensing change also does not impact the project's use of Terraform.</strong></p><h2>Background on Crossplane Providers and Terraform</h2><p>Crossplane Providers enable Crossplane to provision infrastructure on an external service. Providers create new Kubernetes APIs and map them to external APIs. Providers are responsible for all aspects of connecting to non-Kubernetes resources. This includes authentication, making external API calls and providing Kubernetes Controller logic for any external resources.</p><p>There are 100s of Crossplane providers today authored by different parties. Some of these rely on code generation approaches to interface with the backed vendor, and some are implemented natively. See <a href="https://marketplace.upbound.io/providers">https://marketplace.upbound.io/providers</a> for a growing list of providers today.</p><p>Upbound maintains a set of Official Providers for Crossplane that are open source (Apache 2.0), and enterprise-ready. Some of these Official Providers rely on Terraform providers underneath the covers via code generation technology called “upjet”. Upjet translates Kubernetes API calls in Crossplane to Terraform provider invocations which in turn invoke backend cloud or infrastructure API.</p><p>Upjet today invokes the Terraform CLI directly and is in compliance with the licensing of Terraform. Over the last few months we have identified a number of performance improvements that can be achieved by circumventing the Terraform CLI completely and invoking the Terraform providers directly. This work is currently underway and we expect new providers to be released over the next few months. Terraform providers will continue to be licensed under MPL2.</p><p>In summary, Upbound’s official providers are not impacted by the license change today.</p><h2>Looking forward</h2><p>Upbound is committed to ensuring the ecosystem around Crossplane is healthy and vibrant. We’ve recently announced our <a href="https://www.upbound.io/why-upbound/partners">partner program</a> and a verification program for providers. We host 100s of providers in the <a href="https://marketplace.upbound.io/providers">Upbound Marketplace</a> and work with the community to publish these.</p><p>As Crossplane grows, we believe the community will benefit from having providers being maintained and owned by the cloud and infrastructure vendors. This ensures the highest level of quality and support for their product offerings. This is why we’ve been working with cloud providers like AWS, Azure and Google Cloud, and infrastructure vendors like Datadog, Aviatrix, and more.</p><p>We are investing in <a href="https://docs.upbound.io/providers/migration/">tooling to make provider migrations</a> automatic and seamless. As we continue to improve our providers, recently with the launch of <a href="https://blog.upbound.io/new-provider-families">provider families</a>, or in the future with new features, native bindings, and more, we will continue to have automated migration tooling and always have a focus on the end-user experience.</p><p>We feel confident this will not impact Crossplane community members or Upbound customers. If you have any questions or concerns about your Upbound solutions, please reach out to your account manager directly or <a href="mailto:customer-success@upbound.io" target="_blank" rel="noopener noreferrer">email us</a>.</p>

HashiCorp's BSL License Change: What this Means for Upbound Customers and the Crossplane Community

<p>Wondering what <a href="https://www.crossplane.io/" target="_blank" rel="noopener noreferrer">Crossplane</a> is all about? <a href="https://blog.upbound.io/what-is-crossplane/">Read my first blog here</a>.</p><p><i>This excerpt was originally published in the </i><a href="https://learning.oreilly.com/" target="_blank"><i>O’Reilly Report: “What is Crossplane?”</i></a><i>.</i></p><h2>All About Control Planes</h2><p>In my opinion, a good control plane should do three things:</p><ul><li>Drive the data plane toward the desired state</li><li>Protect the data plane</li><li>Be reliably available</li></ul><p><strong>In this blog I’ll explore how a control plane built with Crossplane achieves the first goal.</strong> If you know how <a href="https://blog.upbound.io/control-plane-revolution/">Kubernetes’s control plane works</a>, you’ll notice a lot of familiar concepts in this blog. This is because Crossplane builds on the Kubernetes control plane and enjoys many of its benefits.</p><p>If you’d like to learn more about how Crossplane protects the data plane and is readily available, I’ve detailed everything you need to know in my report, <a href="https://www.upbound.io/resources/lp/whitepaper-b/oreilly-what-is-crossplane">“What Is Crossplane?: A Low-Code Framework for Building Cloud Native Control Planes”</a> here.</p><h2>Driving Toward the Desired State</h2><p>A control plane’s core function is to reconcile the actual state of its data plane with a desired state. <strong>A thermostat is a good example:</strong> you might desire your room to be 72 degrees F, but its actual temperature is 65 degrees. The thermostat can reconcile the actual state of your room (its temperature) with your desired state by running the heat until the room reaches 72 degrees F.</p><p>You tell Crossplane what the desired state of your data plane is by making API calls. Crossplane exposes a JavaScript object notation (JSON) representational state transfer (REST) API over hypertext transfer protocol (HTTP). JSON REST APIs are broadly supported by tools and programming languages. This makes it easier to integrate Crossplane with other systems.</p><p><strong>Crossplane APIs look a lot like Kubernetes APIs.</strong> They are compatible with popular tools in the Kubernetes ecosystem like Helm, ArgoCD, Kyverno, and of course Kubectl. All of Crossplane’s APIs are actually Kubernetes Custom Resources. This means that, like all Custom Resources, they’re declarative. You just tell Crossplane the desired state of the data plane — not how to achieve it.</p><p>APIs exist on a spectrum from purely declarative to purely imperative. Imperative APIs require you to specify how to achieve your desired state. Declarative APIs are simpler: would you rather imperatively tell your thermostat to run the heater for 20 minutes each time you notice it’s cold, or declaratively tell it to keep the room at 72 degrees?</p><p><strong>A huge benefit of Crossplane’s declarative APIs is that if you save the body of your API request to disk, you have a configuration file.</strong> This lets you keep your desired state using a revision-control system like Git. Crossplane API objects use JSON, but tools like Kubectl typically convert them to YAML ain’t markup language (YAML) when you save them to a file. A desired-state API object saved as a YAML file is often called a manifest. Example 2-1 shows a Crossplane API resource in YAML form. This will look familiar to you if you’ve ever seen a Kubernetes Deployment or Pod manifest.</p><p>Each resource in Crossplane’s API represents a data plane entity. You can create, read, update, or delete that entity using HTTP verbs (such as POST, GET, PUT, and DELETE). API resources represent things, not actions. Think back to the SimplePostgres example from the <a href="https://blog.upbound.io/what-is-crossplane/">first blog</a>.</p><p>All Crossplane API resources include at least three top-level fields:</p><h3>apiVersion and kind</h3><p>Specifies the type of the resource. This makes it easy to identify the type of the resource without knowing its URL, for example when reading a YAML manifest.</p><h3>metadata</h3><p>Metadata that is supported by all resources, including their name, creation time, and arbitrary labels and annotations.</p><p>Most Crossplane API resources contain only two other top-level fields: spec and status. The spec field specifies the desired state of the resource, while the status field reflects its observed actual state. All API resources have OpenAPI v3 schemas.</p><h4><strong>Example 2-1: A SimplePostgres YAML manifest</strong></h4>

<p>Crossplane offers a consistent, uniform API to express desired state and read actual state. This makes working with Crossplane predictable—all of its APIs look and feel similar. It reduces users’ cognitive burden and helps tools integrate with Crossplane. <strong>In many places, Crossplane APIs are more uniform than Kubernetes APIs.</strong> For example, the same API field in all Crossplane resources reflects whether they are ready to use.</p><p>You might have noticed that a thermostat must measure the temperature of the room to determine whether it needs to run the heat. The thermostat uses a feedback loop to make the right corrections over time. <strong>This is called closed-loop control, and it’s a desirable property of a control plane. Crossplane does this using controllers.</strong> A controller subscribes to the API that it is responsible for and watches for changes to its desired state. Each controller is responsible for a single kind of API resource. A SimplePostgres API is powered by a SimplePostgres controller.</p><p>When the desired state changes, the controller:</p><ol style="list-style-type:decimal;"><li>Reads the desired state from its API</li><li>Observes the actual state of the part of the data plane its API represents</li><li>Determines whether the actual state is different from the desired state</li><li>Corrects the actual state by creating, updating, or deleting something</li></ol><p>Imagine you’ve configured Crossplane to satisfy calls to the SimplePostgres API by creating RDS instances. Each SimplePostgres instance corresponds to an RDS instance.</p><p>When the SimplePostgres API is called to create a new PostgreSQL instance, its controller will:</p><ol style="list-style-type:decimal;"><li>Read the desired state from the SimplePostgres API</li><li>Attempt to observe the actual state of the corresponding RDS instance</li><li>Find that the RDS instance does not exist</li><li>Create the RDS instance, deriving its configuration from the desired state read from the SimplePostgres API</li></ol><p>Crossplane controllers use closed-loop control to ensure that your data plane never drifts from your desired state. Once you tell Crossplane the desired state of your data plane, it will constantly observe the data plane’s actual state and quickly self-heal when that drifts.</p><h2>Want the Full Picture?</h2><p>If you’d like to understand the full picture of how a Crossplane control plane works, my colleague Viktor covers everything in his book, “<a href="https://www.upbound.io/resources/lp/book/crossplane-cloud-native-control-plane">Crossplane: The Cloud Native Control Plane</a>." It goes into depth about Crossplane, how a Crossplane Control Plane works &amp; how to build one, and more.</p>

<p>&nbsp;</p><p><a href="https://www.upbound.io/resources/lp/book/crossplane-cloud-native-control-plane">Feel free to download it here</a> to get the full rundown of Crossplane &amp; Control planes to level up your infrastructure.</p>

How a Crossplane Control Plane Works: Driving Infrastructure Toward Your Desired State

<p><i>This excerpt was originally published in the </i><a href="https://learning.oreilly.com/" target="_blank"><i>O’Reilly Report: “What is Crossplane?”</i></a><i>.</i></p><p><a href="https://www.crossplane.io/" target="_blank">Crossplane</a> is a framework for building cloud native control planes. When the founders of Crossplane at Upbound told me what they were working on in late 2018, shortly before the <a href="https://blog.upbound.io/introducing-crossplane-open-source-multicloud-control-plane/">v0.1 release</a>, I was very excited. I’d spent a lot of my career building cloud control planes from scratch. I loved the idea of an open source framework that baked in many best practices of control plane design and could be extended to control almost anything. I immediately joined the project and have been a maintainer since Crossplane v0.1.</p><p>A cloud platform is a distributed system, and distributed systems are often architecturally separated into two parts: control planes and <a href="https://ia802905.us.archive.org/35/items/memoryprocessorc557lawr/memoryprocessorc557lawr.pdf" target="_blank">data planes</a>. The data plane is the system under control. It’s what provides direct business value. As for the control plane, it works like an air-traffic-control tower. This analogy is illustrated in Figure 1-1. The aircraft around the airport represent the data plane. The aircraft provide the business value—moving things around—but the air-traffic-control tower is crucial to ensure they can do so safely and efficiently.</p>

<p><a href="https://resources.upbound.io/aws-partnership" target="_blank">Amazon Web Services (AWS)</a> is another example of control planes and data planes. AWS’s data planes are the virtual machines, databases, caches, and other services folks use to build products. Each of these services has a control plane that is responsible for starting, stopping, and configuring them. When you use the AWS console or call an AWS Application Programming Interface (API), you’re interacting with one of AWS’s control planes.</p><p>You can also think of a cloud control plane as having a backend that orchestrates the data plane and a frontend that exposes an API. That’s where Crossplane comes in: it takes the Kubernetes control plane and makes both its backend and its frontend highly extensible. You can extend the backend to control any kind of system, and you can extend the frontend APIs to expose whatever concepts make sense for your organization. Figure 1-2 shows an engineer making an API call to the control plane’s frontend API, and the control plane backend orchestrating a cloud provider.</p>

<p>Imagine you’re part of the platform engineering team at a growing technology company—say, a music streaming service. Your team supports 40 engineers, who are split across 7 other teams. These teams deliver your company’s core product: streaming music. Perhaps they also work on social features or recommendation engines. Let's call these folks product engineers. Most of your product-engineering teams want to use PostgreSQL databases to build their features, but they’re not experts in securing and scaling them.</p><p>Your team’s job, as <a href="https://blog.upbound.io/top-ten-things-slowing-down-your-platform-team/">platform engineers</a>, is to make it safer and more efficient for the product engineers to do their regular tasks. This is a form of <a href="https://en.wikipedia.org/wiki/Force_multiplication" target="_blank">force multiplication</a>: the platform engineers multiply the effectiveness of the product engineers.</p><p>How could you make it safer and more efficient for the product engineers to provision PostgreSQL databases?</p><p>You can learn even more about how a Crossplane Control Plane works and how to build one, and more from my colleague Viktor in his latest book, “<a href="https://www.upbound.io/resources/lp/book/crossplane-cloud-native-control-plane">Crossplane: The Cloud Native Control Plane</a>”.&nbsp;</p>

<p>&nbsp;</p><p><a href="https://www.upbound.io/resources/lp/whitepaper-a/oreilly-what-is-crossplane" target="_blank">Download it here</a> to get the full rundown of Crossplane and control planes to level up your infrastructure.</p>

What is Crossplane?

<p>Another KubeCon + CloudNativeCon is now in the books, and<a href="https://www.linkedin.com/feed/update/urn:li:activity:7056990954842443776" target="_blank"> we’re still feeling the energy</a> from the sights and sounds of Amsterdam and another successful cloud native conference. As the excitement starts to wind down, and we successfully found our way out of the labyrinthine halls of the RAI (has there ever been a more confusing KubeCon venue to navigate?), we wanted to take a minute to reflect on what we learned and experienced, especially compared to KubeCons of years past.</p>

<h2>KubeCon Attendees ❤️ Crossplane</h2><p>As the creators of the <a href="https://www.crossplane.io/" target="_blank">Crossplane</a> project, we are always thrilled to see the enthusiasm and engagement from attendees at each KubeCon. There was never a dull moment in the Solutions Showcase, as traffic to the Upbound and Crossplane booths was always flowing. We continuously had people coming up to us and asking something similar to “I keep seeing and hearing Crossplane everywhere - please tell me more about it!”, which we can assure you we’ll never get tired of hearing. While this excitement is nothing new compared to previous KubeCons, we did notice there were more folks approaching us this year that were already well familiar with Crossplane’s vision. We’re confident some of the “what is this Crossplane I keep hearing about” folks from Detroit are now well into their adoption journey with Crossplane and have already become valued members of the community. If you’re still curious to learn what Crossplane does and why it’s valuable, we encourage you to check out this post we recently authored for the <a href="https://thenewstack.io/crossplane-what-most-people-get-wrong-and-how-to-get-it-right/" target="_blank">New Stack</a>.</p>

<h2>It’s so Good to Connect in Person</h2><p>We presented more sessions and talks in Amsterdam than at any previous KubeCon. The most recent addition to our stable of sessions was the Crossplane <a href="https://sched.co/1Hzcf" target="_blank">Contribfest</a>, where developers interested in becoming contributors gathered together for an interactive session with hands-on labs. By the end of the productive session, attendees had learned how to write, build, and test new code for Crossplane as well as write their own <a href="https://docs.crossplane.io/knowledge-base/guides/composition-functions/" target="_blank">Composition Functions</a>. We are looking forward to the developers that joined the Contribfest to continue participating as contributors to the project, and even becoming deeply involved to the point of joining the <a href="https://github.com/crossplane/crossplane/blob/master/GOVERNANCE.md#becoming-a-maintainer" target="_blank">maintainer team</a> as well! A great place to start getting more active in the project is our <a href="https://slack.crossplane.io/" target="_blank">Crossplane Slack</a>, as well as the code base and issue backlog in the <a href="https://github.com/crossplane" target="_blank">Crossplane GitHub</a>.</p><p>We also hosted a very successful <a href="https://sched.co/1HySE" target="_blank">Maintainer Track talk</a> that had more than 1,000 registered attendees. These talks are always really interesting because they combine an introduction to the project along with a deep dive into some of its advanced concepts and architecture, so there’s something for all levels of experience with Crossplane. During this talk, we gave three separate live demos that covered the most exciting and highly demanded new features, and we hope attendees left the session inspired to accomplish new scenarios with their Crossplane powered platforms.</p><p>In addition to the ContribFest and Maintainer Track sessions, we also took advantage of all the opportunities to connect with the community in the CNCF Project Pavilion, with Upbound customers at our booth in the Solutions Showcase, and a select set of top contributors to the Crossplane project for a special appreciation dinner at <a href="https://brouwerijtroost.nl/depijp-amsterdam/" target="_blank">Brouwerij Troost De Pijp</a>. It was such a fantastic week connecting with old friends and building great new relationships!</p><h2>Everyone was Wondering…</h2><p>From the myriad of conversations during the week, we heard some common questions that folks were frequently asking. Let’s review these questions and dive into some answers here.</p><h4>“I’m already using Terraform, but Crossplane looks awesome! How can I start using it?”</h4><p>Lots of Crossplane adopters have (or have had) Terraform in their environment, and it’s totally reasonable to run both within your environment at the same time - it’s not an all or nothing proposition with Crossplane. There are a few strategies for adopting Crossplane in a phased approach over time that are worth reading more about in this resource for <a href="https://resources.upbound.io/all-content/optimizing-terraform-with-crossplane?ref=blog.upbound.io" target="_blank">optimizing Terraform with Crossplane</a>.</p><h4>“All the CRDs that Crossplane installs impact the performance of my cluster, what can I do?”</h4><p>This is priority number one in the Crossplane <a href="https://github.com/crossplane/crossplane#roadmap" target="_blank">roadmap</a> right now. Kubernetes was designed with some key scaling properties in mind, but unfortunately the number of CRDs was not one of them. We’re attacking this issue from two major fronts: We’ve already made a handful of fixes to upstream Kubernetes to start scaling the supported number of CRDs in general, and we are rapidly executing on a plan within Crossplane to <a href="https://github.com/crossplane/crossplane/blob/master/design/design-doc-smaller-providers.md" target="_blank">install only the CRD groups</a> that are important to you. These parallel efforts should greatly improve the control plane performance soon, so we appreciate all the feedback and patience while we thoughtfully roll these out.</p><h4>“What do I use to provision a cluster for Crossplane, is this a chicken and egg problem?”</h4><p>Crossplane has very few requirements to get up and running, so one simple approach here is to create a cheap and ephemeral bootstrapping cluster, for example <a href="https://kind.sigs.k8s.io/" target="_blank">kind</a>, and leverage that to provision your long term Crossplane instance. Viktor Farcic did a great video on this that can be found on the <a href="https://youtu.be/IlaYGgyg06o" target="_blank">Upbound YouTube channel</a>. An even easier approach though would be to offload this problem entirely, including the full spectrum of operational burdens and concerns, to the experts at Upbound by taking advantage of our latest product, Upbound, which hosts <a href="https://www.upbound.io/product/upbound" target="_blank">managed control planes</a>.</p>

<h2>Looking Forward to Chicago</h2><p>The cycle for the next KubeCon + CloudNativeCon in <a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/" target="_blank">Chicago</a> is already ramping up, as the <a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/program/cfp/" target="_blank">Call for Proposals</a> is now officially open. It won’t be long before we’re all together again and we are definitely looking forward to it. What’s not to love about Chicago? With its characteristic <a href="https://en.wikipedia.org/wiki/Chicago-style_pizza" target="_blank">deep-dish pizza</a>, marvelous <a href="https://en.wikipedia.org/wiki/Architecture_of_Chicago" target="_blank">architecture</a>, and hilarious <a href="https://www.secondcity.com/" target="_blank">comedy scene</a>, we know it will be a fabulous week together. Just keep thinking warm thoughts for the week until then!<br>&nbsp;</p><p>We will submit proposals for multiple talks about Crossplane and cloud native control plane concepts, so we hope to have many opportunities to connect with you all again at the booths and Crossplane sessions. We’re always eager to support the community in submitting ideas of their own, so feel free to share your ideas in the <a href="https://slack.crossplane.io/" target="_blank">Crossplane Slack</a> or reach out directly to Jared (<a href="https://crossplane.slack.com/archives/DEFND2TS8" target="_blank">@jbw976</a>) for feedback on any proposals you are thinking about. &nbsp;Finally, we have exciting ideas for how to continue evolving the next Contribfest and keep investing in our amazing Crossplane community, so get ready to write more code together!</p><h2>What’s Next?</h2><p>In the meantime, if you’re curious about Upbound and what we can offer, we invite you to try our latest product offering, Upbound, for free. Enable SRE teams with a more efficient workflow and the building blocks of platform engineering by setting up your first cloud platform powered by Upbound’s managed control planes. <a href="https://upbound.io/register?ref=blog.upbound.io" target="_blank">Sign up for your 30 day free trial here</a>, and we’ll see you in <a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/" target="_blank">Chicago</a>!</p>

Wrapping Up KubeCon + CloudNativeCon Amsterdam & Looking Ahead

<p>Control planes are a hot topic of conversation within the cloud native community. Increasingly, SREs describe the way they build platforms as being <a href="https://blog.upbound.io/__GHOST_URL__/composing-a-platform-by-patching-crossplane-resources/">architected around a control plane</a>. These platforms come in many shapes and sizes. Some are full-blown Internal Developer Platforms used across the entire organization. Other kinds of platforms simply orchestrate core services in a way that ensures high uptime. And still other platforms seek to better integrate with the organization’s cloud native tooling by removing a legacy Infrastructure as Code dependency.</p><p>Upbound has been at the forefront of this control plane revolution since inventing the Crossplane project in 2018. The key insight behind the creation of Crossplane was that the Kubernetes architecture was more broadly applicable than just container orchestration. In the end, Kubernetes pioneered an entire operating model of resource management, one which was highly relevant (if correctly applied) to SREs tasked with building platforms and managing infrastructure.</p><h2>Extending the Kubernetes Operating Model</h2><p>Kubernetes revolutionized cloud computing by applying an “Observe, Analyze, Act” model for resource management to container orchestration use cases. Massive amounts of complexity are required within a Kubernetes cluster to make the idea of this seemingly simple model a reality. At a high level, Kubernetes looks at what the desired state of things needs to be, assesses what’s different in reality, and automatically makes configuration changes which eliminate any differences between desired and actual state.</p><p>Until Kubernetes officially supported Custom Resource Definitions (CRDs) in Kubernetes version 1.7, the Kubernetes operating model only applied to bits inside of the cluster. CRDs allowed for Kubernetes users to extend their cluster’s API with custom types, freeing the Kubernetes operating model to be applied to things other than pods, containers, deployments, and so on.</p><p>When Upbound launched Crossplane in 2018, many Kubernetes users had never heard of a CRD before. Crossplane was one of the first cloud native projects to take the idea of Custom Resource Definitions to their logical conclusion. By relying on a Kubernetes controller and CRD for every cloud resource, Kubernetes users are able to orchestrate resources in the public clouds using the exact same tools, products, and methodologies they used for applications.</p><p>This insight has allowed Crossplane users to rapidly converge both application and infrastructure workflows around the Kubernetes API, and standardize on cloud native tooling which has been built up around it. Crossplane users are some of the most <a href="https://blog.upbound.io/__GHOST_URL__/scaling-kubernetes-to-thousands-of-crds/">advanced Kubernetes users on the planet</a>. A typical user will tell us how they’ve integrated their control planes with cloud native continuous delivery systems like Argo or Flux while simultaneously collecting platform-level metrics with modern services such as Datadog or New Relic.</p><h2>From IaC to Control Plane Architectures</h2><p>The journey towards convergence has partially been motivated by Crossplane users realizing they’ve been using Infrastructure as Code tools such as Terraform as a substitute for a proper control plane. Countless users have told us stories about how their CD pipelines diff the output of `tf plan` commands, and how they’re constantly struggling to manage TFSTATE between clients as they attempt to deploy infrastructure changes using GitOps.</p><p><a href="https://resources.upbound.io/control-plane" target="_blank">Control planes</a> offer an alternative architecture that completely eliminates these issues. Because Crossplane leverages the Kubernetes operating model, desired state and actual state are always kept in sync, eliminating the need to have a plan command or write state to a file entirely. Additionally, day two operations become streamlined with <a href="https://resources.upbound.io/configuration-drift" target="_blank">configuration drift</a> automatically being reconciled by Crossplane whenever an accidental change is detected.</p><p>Lastly, platforms built around a control plane can provide consumers of these platforms with self-service super powers. On top of the CRDs that Crossplane installs into a cluster for infrastructure management, Crossplane enables platform builders to present a custom cloud API to the developers using those platforms. For example, Crossplane is commonly used to power Kubernetes-as-a-Service platforms inside large enterprises. The platform teams running these services are trying to solve the problem of enabling any developer to get a Kubernetes cluster when they need one and making sure those clusters are properly configured.</p><p>Before Crossplane, each cluster would be a one-off deployment for developers, typically with weeks of requirements gathering and custom configuration done by the platform team. With Crossplane, platform teams compose a custom cloud API consisting of two parts. One part defines the infrastructure configuration details - things like VPCs, subnets, and permissions. The second part is a definition for a self-service API designed to be used by the developers using the platform. Developers consume this API in order to deploy a new Kubernetes cluster without having to know anything about the details of the cloud environment or cluster configuration.</p><h2>Democratizing Control Plane Technology with Crossplane</h2><p>Engineers who’ve built hyperscale cloud computing services like AWS, Azure, and GCP have been building control planes for almost 15 years. Each of the services in the hyperscalers are designed around a unified control plane that powers the entire cloud service, while presenting an easy-to-use API to customers. For example, while an AWS S3 end-user may consume a bucket via the popular boto3 SDK, behind the scenes is an incredibly complex service with a control plane orchestrating everything from data replication to supply chain logistics for new hard drives coming online.</p><p>For almost 15 years, how to build and use a control plane has essentially been proprietary knowledge. The consequence is that well-architected services and systems were the exception rather than the norm. Platform engineers simply didn’t have a corpus of work to build on or a community where best practices could be shared and components reused.</p><p>By open sourcing Crossplane and donating it to the CNCF, Upbound democratized control plane technology for the masses. Now anyone who knows how to build a microservice using Kubernetes can deploy a control plane for their organization. There’s also a vibrant and thriving community being built around Crossplane, too. Every day, new platform engineers are joining the <a href="https://slack.crossplane.io/" target="_blank">Crossplane Slack</a> to learn from one another, share best practices, and collaborate on the project.</p><p>The community’s growth and collaboration has increased so dramatically that Upbound launched Upbound Marketplace, an open hub for the Crossplane community to discover and share plugins for Crossplane.</p><h2>Production with Upbound</h2><p><a href="https://www.upbound.io/" target="_blank">Upbound</a> created the Crossplane project to take the cloud native space to a new level. As the inventors of this technology, Upbound is uniquely positioned with a wealth of knowledge and understanding to effectively harness Crossplane’s capabilities into a complete solution that fits your specific needs.</p><p>If you are interested in leveraging the power of control planes without needing to worry about operational concerns or managing the underlying infrastructure, <a href="https://resources.upbound.io/all-content/welcome-to-the-control-plane-revolution" target="_blank">check out my webinar where I talk more in depth about the control plane revolution with my collaege Sumbry.</a></p>

Control Plane Revolution

<h2>Building the Upbound Marketplace to Run Anywhere.</h2><p>When we set out to build the <a href="https://marketplace.upbound.io/" target="_blank">Upbound Marketplace</a>, we knew the functionality it offered would be useful in a variety of contexts. With that in mind, we architected a system that could fit into existing customer environments while still being able to scale to handle large amounts of traffic from the Crossplane community, which <a href="https://crossplane.devstats.cncf.io/d/74/contributions-chart?orgId=1&amp;var-period=d7&amp;var-metric=contributions&amp;var-repogroup_name=All&amp;var-country_name=All&amp;var-company_name=All&amp;var-company=all&amp;from=now-4y&amp;to=now-2d" target="_blank">continues to grow at an impressive rate</a>. Today we are going to take a deeper look at some of our design decisions, explore how they were implemented, and talk about the places we can go from here.</p><h2>Just an OCI Image</h2><p>One of the decisions we made early on in the Crossplane ecosystem was to standardize on OCI images for package distribution. This choice has paid dividends as we continue to leverage the features offered by the <a href="https://github.com/opencontainers/image-spec" target="_blank">image</a> and <a href="https://github.com/opencontainers/distribution-spec" target="_blank">distribution</a> specifications. One of the immediate benefits it offered was access: nearly every Crossplane user had access to a registry, and tooling to interact with them generically.</p><p>The generic aspect is not to be overlooked. When Crossplane packages were first being designed we considered leveraging <a href="https://github.com/opencontainers/image-spec/blob/main/artifact.md" target="_blank">OCI artifacts</a>, which were in their early stages of development, as a solution. However, <a href="https://www.docker.com/blog/announcing-docker-hub-oci-artifacts-support/" target="_blank">until recently</a>, some registries did not support artifacts and some only supported some subset of them. In contrast, every registry implementation we have heard of Crossplane users interacting with has supported Crossplane packages from day one because they are <i>just OCI images</i>.</p><blockquote><p>Thanks to the tremendous efforts of working groups in the OCI organization, much progress has been made around the standardization of artifacts, as well as the more recently introduced <a href="https://github.com/opencontainers/image-spec/pull/934" target="_blank">reference types</a>. We are excited to explore how this work may be leveraged in the Crossplane community and the Upbound Marketplace in the future.</p></blockquote><p>Making this decision wasn’t without a cost. Namely, we needed to be explicit about how Crossplane packages bundled as OCI images would be processed by any conformant Crossplane distributions. Like many open source projects, Crossplane is both an API specification and a reference implementation. Downstream distributions, such as <a href="https://github.com/upbound/universal-crossplane" target="_blank">Upbound Universal Crossplane (UXP)</a>, must handle Crossplane packages in a uniform manner. To enable this, the <a href="https://crossplane.io/docs/v1.10/reference/xpkg.html" target="_blank">xpkg specification</a> was proposed and ratified.</p>

<p>Defining a dedicated standard in the context of OCI images allowed for some creative optimizations and extension points. For example, Crossplane Provider packages include both YAML content, which is processed by the Crossplane package manager, and controller binaries which ultimately get run as a container on a node in the cluster. Typically, the binary and supporting content is much larger than the YAML content, and because Crossplane only needs to consider the latter, a convention has been established to store the YAML content in a dedicated layer, which is annotated in the image manifest. Crossplane first fetches the manifest, and if any layer is annotated with <code>io.crossplane.xpkg: base</code> only that layer is fetched and processed.</p><p>The xpkg specification also defines extension points that can be optionally honored by third-party systems. Extensions can be useful if a given Crossplane distribution or registry wants to add custom functionality to packages. Defining extensions within the bounds of the OCI image specification decouples image storage and distribution from other features that may enrich the package management experience. This model provides the foundation for the Upbound Marketplace. If we take a look at the manifest for <a href="https://marketplace.upbound.io/providers/upbound/provider-aws" target="_blank">Upbound’s Official AWS Provider</a>, we’ll notice not only the <code>base</code> layer, but also a layer annotated as <code>upbound</code>:</p>

<p>This layer is not considered by the Crossplane package manager, but includes data that can enrich the Upbound Marketplace experience. In this case, the layer contains a YAML stream of examples that are processed when packages are pushed. <strong>The secret behind the Upbound Marketplace is that it is not an OCI registry at all (per se), but rather a proxy that sits in front of any OCI registry</strong>. This proxy is internally named <code>shimmer</code> – it is a registry shim that implements the OCI distribution specification but doesn’t handle storing blobs or manifests, it simply passes them along to another registry.</p><h2>Bring Your Own Registry</h2><p>This architecture provides a number of advantages. Perhaps the greatest benefit is that it, in tandem with Crossplane’s package specification, allows for flexibility in what registry is used to actually store and serve any given image. This is useful internally as we can easily migrate from one registry to another, or, even more interestingly, proxying across multiple registries. For example, for packages used internally at Upbound we may opt to proxy the images to the registry offered by the cloud provider where our services are running, while for some customers we may proxy to a registry running in a specific geographic region. Similarly, Upbound customers who operate in an air-gapped or heavily regulated environment could deploy an on-prem version of the Upbound Marketplace in front of their own internal registry, allowing them to experience all the benefits of automatic documentation, discovery, and searchability, without compromising on compliance.</p>

<p>When packages pass through <code>shimmer</code>, in addition to deciding where to direct them, <code>shimmer</code> enqueues the image for parsing and indexing. Much like Crossplane’s package manager, <code>shimmer</code>’s indexer only fetches the layers that it needs. In this case, the base layer is required to serve documentation about the package and the API types it offers, while the upbound layer provides additional content that is only relevant in the context of the Upbound Marketplace. In other words, a package may support extensions for many different consumers, without increasing the overhead for any one consumer to process just the bits that they care about.</p><p>An alternative model to the <code>shimmer</code> proxy approach would be to utilize webhook functionality <a href="https://docs.docker.com/registry/notifications/" target="_blank">supported by some existing OCI distribution implementations</a>. However, doing so would both eliminate guaranteed compatibility across all implementations (i.e. webhooks are not part of the OCI distribution specification), and restrict introducing custom behavior when pulling a package. Up to this point we have primarily been concerned with validating and indexing content from packages that are already formatted as OCI images, but sitting in between the client and some registry means that what happens when a user pulls can vary widely. In fact, an “underlying image” doesn’t even have to exist as long as content is returned to the user in a manner conformant with the distribution specification. This opens up some interesting possibilities, such as just-in-time (JIT) package building from source on pull… but that’s a post for another day.</p><h2>Expanding the Use of Content Addressability</h2><p>Much is made of the benefits of leveraging content addressability, the practice of referencing a set of data by the digest of its content, to <a href="https://youtu.be/Ve_CFyVYo3M" target="_blank">improve security posture</a>. However, two of the other major benefits of such a system are <strong>storage</strong> and <strong>caching</strong>. When data is stored with a content addressable identifier, subsequent uploads of the same data does not result in occupying twice the storage space. Instead the system will simply add another reference to the existing data.</p><blockquote><p>Some systems will only partially deduplicate storage. For example, a multi-tenant service may only deduplicate data within the context of a single tenant.</p></blockquote><p>While avoiding storing the same data twice is all about space, caching is all about speed. You have likely experienced the benefits of caching content addressable data if you have ever attempted to docker pull the same image twice. If you haven’t give it a try right now:</p>

<p>The second pull was drastically faster because we already had all of the layers cached. How did we know that? The client, in this case Docker, fetched the manifest associated with the <code>v0.20.0</code> for <code>xpkg.upbound.io/upbound/provider-aws</code>, and because all of its layers were identified by the hash of their content, the client can evaluate against the locally stored content.</p><blockquote><p>In reality, evaluating whether local content is up to date can be performed even more quickly by caching the manifest and checking whether its digest has changed via a <code>HEAD</code> request.</p></blockquote><p>The behavior of the Upbound Marketplace is not so different from that of an OCI registry. It ingests content (Kubernetes manifests) and serves it to a wide variety of consumers via client applications (browsers). Much like a registry doesn’t want to store unchanged layers every time a new version of an image is pushed, we do not want to store the same manifests used to generate documentation every time a new package is pushed and indexed. To avoid doing so, <code>shimmer</code> stores manifests by digest, and generates a document that somewhat resembles an image manifest for each package version. For example, the resource manifests for the previously referenced <a href="https://marketplace.upbound.io/providers/upbound/provider-aws/v0.20.0" target="_blank">upbound/provider-aws:v0.20.0</a> package can be observed <a href="https://api.upbound.io/v1/packages/upbound/provider-aws/v0.20.0/resources" target="_blank">here</a>, and the content of a single manifest, Certificate, can be viewed <a href="https://api.upbound.io/v1/packages/upbound/provider-aws/v0.12.0/resources/acm.aws.upbound.io/Certificate" target="_blank">here</a>.</p><p>The reduction in storage is also paired with the ability to perform aggressive caching in many cases. Perhaps the best example of this is the written documentation you see when visiting one of the <a href="https://blog.upbound.io/__GHOST_URL__/first-official-providers/">Official Provider</a> pages. While currently an alpha feature, we intend to support providing documentation for any package in the Marketplace, once again leveraging the Crossplane package extension model to do so. The existing documentation is fairly lightweight, but the service is designed to support serving more extensive content, once again by leveraging content addressable storage.</p>

<p>The documentation is especially interesting because the Marketplace is built on <a href="https://nextjs.org/" target="_blank">Next</a>, allowing us to Server-Side Render (SSR) most pages for efficient search engine indexing. However, with potentially large content, such as documentation and its supporting assets, we don’t want to load all of it up front and we want to fetch it as infrequently as possible. To achieve both of those goals, documentation is served from a content addressable URL on a Content Delivery Network (CDN) and fetched client-side. This enables caching the content both at the edge node of the CDN (i.e. a server close to the user) and in the browser itself. Invalidation is straightforward as the documentation index is rendered server-side, meaning that any change in documentation content will be reflected by a new content-addressable URL being supplied to the client. If that URL has been accessed before by the client, it may still be in the browser cache. If it has been accessed by another user in the region it may be cached at the edge node. Making a full round-trip to the origin server becomes an infrequent operation.</p><h2>Where We Are Going</h2><p>So where do we go from here? Ultimately, that is up to you! We are excited about continuing to build on the foundational architecture of the Upbound Marketplace. Features such as automatic identification of breaking API changes from one package version to another, searching by API types, and enhanced package consumption metrics are all enabled by the design decisions outlined in this post. If these features, or any others, are top of mind for you, make sure to <a href="https://accounts.upbound.io/register" target="_blank">create an Upbound account today</a> and <a href="https://upbound.io/contact" target="_blank">drop us a note</a>. If you love writing and reading design documents, working on complex multi-tenant systems, and building highly scalable services used by platform builders all over the world, <a href="https://boards.greenhouse.io/upbound" target="_blank">check out our open roles</a>!</p>

Why Upbound

Upbound Elevates Platforming with Crossplane Through New Builder Tooling and Platform Services

Upbound

Start for Free

Pricing

17 Added benefits of Upbound over OSS Crossplane

Using Upbound to Roll Out Cluster-as-a-Service to Your Developers

Resources

Events

Docs

Blog

Contact Support

Report a Vulnerability

Streamlining Platform Building with Crossplane: Working with with Upbound's New Platform Engineering Tools

Open Source Crossplane or Upbound? 17 Added Benefits of Using Upbound

Crossplane