Universal Packages

Back in July, the Azure Artifacts team announced a new kind of package type we are adding called Universal Packages. We have been working away on this for a while and have finally enabled it for preview.

What is a Universal Package?

Universal Packages are simply a collection of files with a name and a version. They are stored in a feed hosted in Azure Artifacts and provide a convenient place to store artifacts that don't fit neatly into other package types. You can store anything you like in a Universal Package but here are some of the things that we've seen some of our preview customers use them for.

  • Configuration scripts & templates (e.g. ARM templates)
  • Database snapshots for integration testing
  • Machine learning training data & models
  • Developer tools & SDKs
  • 3D models and textures
  • Build outputs

Getting started

Its easy to get started with Universal Packages. Simply go and sign up for Azure Artifacts and create a feed then go and get the Azure CLI and install it. Azure DevOps shows up as an extension in the Azure CLI, so once you've installed it drop into your favorite shell and execute the following command:

$ az extension add --name azure-devops

You will also want to login:

$ az login

This will launch a browser and get you to authenticate to Azure and once that is done you'll be able to use the CLI against in Azure Artifacts feeds attached to that underlying AAD tenant. Now you are all setup you should be able to issue the publish command:

$ az artifacts universal publish `
    --feed ARMTemplates `
    --name arm-templates-sql-cluster `
    --version 1.0.6 `
    --organization https://dev.azure.com/mitchdenny `
    --path .

This will upload a package called arm-templates-sql-cluster to a feed called arm-templates and a version number of 1.0.6. The packages show up alongside your other packages in Azure Artifacts.

Downloading is just as easy – just replace the publish command with download and the Universal Package will be downloaded.

$ az artifacts universal download `
    --feed ARMTemplates `
    --name arm-templates-sql-cluster `
    --version 1.0.6 `
    --organization https://dev.azure.com/mitchdenny `
    --path .

The command line tools we provide work on Windows, macOS and Linux. We have also built a task that you can drop into Azure Pipelines.

Why another kind of package?

One of the common themes that we see in modern programming languages / platforms is the tooling that takes care of sharing code between developers. Node.js has NPM, Python has PyPI, .NET has NuGet and Java has Maven (and there are countless other examples).

If you are programming on top of one of these platforms it is good and proper to take advantage of the build-in code sharing mechanism – if I am producing a reusable JavaScript module, I should look at distributing that via the NPM packaging format and protocol (for example). In creating Universal Packages, our intention is not to stop people using these other protocols where they are the best choice. However, we often find that there are other "things" that you want to consume or publish as part of your DevOps pipeline. Sometimes these other things have trouble finding a home and so developers will either abuse their package format of choice or simply drop these random files on a file server somewhere. This is why you end up with NuGet packages that clock in at 1GB+.

We created Universal Packages to be a better option for storing things that fall into the "other" category. By better we mean that we want to inherit some of the nice characteristics of packaging systems such as immutability and explicit versioning.

Summary

If you are looking for a generic way to store a collection of files that you consume or produce in your DevOps pipeline, then Universal Packages might be an ideal choice. In the future I will be writing some more posts about how to use Universal Packages to solve various DevOps problems. In the meantime check out the resources below!

Resources