A quick intro to Juju Resources

For all the fancy bits of technology in your infrastructure there are still plenty of simple things that are useful and will probably never go away. One of these is blobs. Blobs are useful, they can be workload payloads, binaries for software, or whatever bits you need to deploy and manage.

Juju never had a way of knowing about blobs. Sure, you could plop something on an http server and your charm could snag it. But then we’re not really solving any problems for you, you still need to deal with managing that blob, versioning it, using a server to serve it to clients, etc.

Ideally, these blobs are accounted for, just like anything else in your infrastructure, so it makes sense that as of Juju 2.0 we can model blobs as part of a model; we call it Juju Resources. That way we can track them, cache them, acl them, and so on, just like everything else.

Resources

A new concept has been introduced into Charms called “resources”. Resources are binary blobs that the charm can utilize, and are declared in the metadata for the Charm. All resources declared will have a version stored in the Charm store, however updates to these can be uploaded from an admin’s local machine to the controller.

Change to Metadata

A new clause has been added to metadata.yaml for resources. Resources can be declared as follows:

resources:
  name:
    type: file                         # the only type initially
    filename: filename.tgz
    description: "One line that is useful when operators need to push it."

New User Commands

Three new commands have been introduced:

  1. juju list-resources

    Pretty obvious, this command shows the resources required by and those in use by an existing service or unit in your model.

  2. juju push-resource

    This command uploads a file from your local disk to the juju controller to be used as a resource for a service.

  3. juju charm list-resources

    juju charm is the the juju CLI equivalent of the “charm” command used by charm authors, though only applicable functionality is mirrored.

In addition, resources may be uploaded when deploying or upgrading charms by specifying the resource option to the deploy command. Following the resource option should be name=filepath pair. This option may be repeated more than once to upload more than one resource.

juju deploy foo --resource bar=/some/file.tgz --resource baz=./docs/cfg.xml

or

juju upgrade-charm foo --resource bar=/some/file.tgz --resource baz=./docs/cfg.xml

Where bar and baz are resources named in the metadata for the foo charm.

Conclusion

It’s pretty simple. Put stuff in a place and be able to snag it later. People can use resources for all sorts of things.

  • Payloads for the service you’re deploying.
  • Software. Let’s face it, when you look at the amount of enterprise software in the wild, you’re not going to be able to apt eveything; you can now gate trusted binaries into resources to be used by charms.

Hope you enjoy it!

New wiki bundles for you to play with...

I’ve pushed new sample bundles to the Juju Charm Store. The first is a simple mediawiki with mysql:

For a more scalable approach I’ve also pushed up a version with MariaDB, haproxy, and memcached. This allows you to add more wiki units to horizontally scale:

I’ll be working on a “smoosh everything onto one machine” bundle next, so stay tuned!

Solarized theme in Ubuntu

terminal

Servers at your fingertips...

One of the nice things about system containers is that it makes the cost of creating a new machine essentially zero. With LXD 2.0 around the corner this is now easier than ever. LXD now ships with native simplestreams support, which means we can now list and cache all sorts of interesting OS cloud images.

You can see every image provided by doing a lxc image list images:, but the nice thing is the syntax is easy to remember. You can now just launch all sorts of Linux Server cloud images from all sorts of vendors and projects:

$ lxc launch images:centos/7/amd64 test-centos
Creating test-centos
Retrieving image: 100%
Starting test-centos
$ lxc exec test-centos /bin/bash
[[email protected] ~]# yum update
Loaded plugins: fastestmirror

… and so on. Give em a try:

lxc launch images:debian/sid/amd64
lxc launch images:gentoo/current/amd64
lxc launch images:oracle/6.5/i386

And of course ubuntu is supported:

lxc launch ubuntu:14.04

So if you’re sick of manually snagging ISOs for things and keeping those up to date, then you’ll dig 16.04, just install LXD and you can launch any Linux almost instantly. We’ll keep ‘em cached and updated for you too. I can lxc launch ubuntu:12.04 and do python --version faster than I can look it up on packages.ubuntu.com. That’s pretty slick.

70k Nvidia Downloads in 6 months

It’s been about six months since ricotz and mamarley have brought fresh Nvidia drivers to Ubuntu users. I covered some of this in my SCaLE 14x talk on Ubuntu Gaming. If you missed SCaLE then feel free to check out Liz’s summary.

In that talk I summarized some of the challenges (and victories!) to making our platform be better for gamers. One of the obvious pain points is of course, delivering fresh drivers for people, which can be difficult (especially when it comes to managing regressions). First let’s look at the version breakdown:

346.72: 57
346.96: 292
352.21: 219
352.79: 62
355.06: 3291
355.11: 11483
358.09: 16949
358.16: 22317
361.18: 4076
361.28: 10219

And then by Ubuntu release:

precise: 630
trusty: 30665
vivid: 10908
wily: 21537
xenial: 5225

Trusty wins, no surprise there. Wow, way more people on Xenial than I expected, good to see people are testing our upcoming LTS! And finally, by architecture:

amd64: 65661
armhf: 31
i386: 3273

So when you add them all up, it’s about 70,000 downloads of Nvidia drivers from the PPA in the last six months. This is great, as it means there’s certainly a demand for fresh drivers, but it’s also frightening when you consider the end-user experience of traditional packaging and all the wonderful face punching that entails. But hey, that’s XCOM. Snappy can’t come fast enough!

Want to help? Buy a game and check out ppa:graphics-drivers.

Oh and happy belated Vulkan day!

The Ubuntu Steam Box

It’s been about a year since I started building my own Steam console for my living room. A ton has changed since then. SteamOS has been released, In Home Streaming is out of beta and generally speaking the living room experience has gotten a ton better.

This blog post will be a summary of what’s changed in the past year, in the hopes that it will help someone who might be interested in building their own “next-gen console” for about the same price, and take advantage of nicer hardware and all the things that PC gaming has to offer.

Step 1: Choosing the hardware

  • I consider the NVIDIA GTX 750Ti to be the best thing to happen in hardware for this sort of project. It’s based on their newest Maxwell technology so it runs cool, it does not need a special power supply plug, and it’s pretty small. It’s also between $120-$150 – which means nearly any computer is now capable of becoming a game console. And a competent one at that.

  • I have settled on the Cooler Master 110 case, which is one of the least obnoxious PC case you can find that won’t look too bad in the living room. Unfortunately Valve’s slick-looking case did not kick the case makers into making awesome-looking living room style cases. The closest you can find is the Silverstone RVZ01, which has the right insides, but they ruined the outside with crazy plastic ribs. The Digital Storm Bolt II looks great, but you can’t buy the case seperately. Both cases have CD drives for some reason, boo!

  • Nvidia has a great guide on building a PC within the console-price range if you want to look around. I also recommend checking out r/buildapc, which has tons of Mini-ITX/750Ti builds.

  • Another alternative is the excellent Intel NUC and Gigabyte Brix. These make for great portable machines, but for the upcoming AAA titles for Linux like Metro Redux, Star Citizen, and so on I decided to go with a dedicated graphics card. Gigabyte makes a very interesting model that is the size of a NUC, but with a GTX 760(!). This looks to be ideal, but unfortunately when Linus reviewed it he found heat/throttling issues. When they make a Maxwell based one of these it will likely be awesome.

  • Don’t forget the controller. The Xbox wireless ones will work out of the box. I recommend avoiding the off-brand dongles you see on Amazon, they can be hit or miss.

Step 2: Choosing the software

I’ve been using SteamOS since it came out. The genious about SteamOS is that fundamentally it does only 2 things. It boots, and then runs Steam Big Picture (BPM) mode. This means for a dedicated console, the OS is really not important. I have 2 drives in the box, one with SteamOS, and one with Ubuntu running BPM. After running both I prefer Ubuntu/Steam to SteamOS:

  • Faster boot (Upstart v. SysV)
  • PPAs enable fresh access to new Nvidia drivers and Plex Home Theater
  • Newer kernels and access to HWE kernels over the next 5 years

I tend to alternate between the two, but since I am more familiar with Ubuntu it makes it easier to use for, so the rest of this post will cover how to build a dedicated Steam Box using Ubuntu.

This isn’t to say SteamOS is bad, in fact, setting it up is actually easier than doing the next few steps; remember that the entire point is to not care about the OS underneath, and get you into Steam. So build whatever is most comfortable for you!

Step 3: Installation

These are the steps I am currently doing. It’s not for beginners, you should be comfortable admining an Ubuntu system.

  • Install Ubuntu 14.04.
  • (Optional) - Install openssh-server. I don’t know about you but lugging a keyboard/mouse back and forth to my living room is not my idea of a good time. I prefer to sit on the couch, and ssh into the box from my laptop.
  • Add the xorg-edgers PPA. You don’t need this per se, but let’s go all in!
  • Install the latest Nvidia drivers: As of this writing, nvidia-graphics-drivers-343.

After you’ve installed the drivers and all the security updates you should reboot to get to your nice new clean desktop system. Now it’s time to make it a console:

  • Log in, and install Steam. Log into steam, make sure it works.
  • Add the Marc Deslaurier’s SteamOS packages PPA. These are rebuilt for Ubuntu and he does a great job keeping them up to date.
  • sudo apt-get install steamos-compositor steamos-modeswitch-inhibitor steamos-xpad-dkms
  • Log out, and in the login screen, click on the Ubuntu symbol by the username and select the Steam session. This will get you the dedicated Steam session. Make sure that works. Exit out of that and now let’s make it so we can boot into that new session by default
  • Enable autologin in LightDM after the fact so that when your machine boots it goes right into Steam’s Big Picture mode.

We’re using Valve’s xpad module instead of xboxdrv because that’s what they use in SteamOS and I don’t want to deviate too much. But if you prefer about xboxdrv then follow this guide.

  • Steam updates itself at the client level so there’s no need to worry about that, the final step for a console-like experience is to enable automatic updates. Remember you’re using PPAs, so if you’re not confident that you can fix things, just leave it and do maintenance by hand every once in a while.

Step 4: Home Theater Bling

If you’re going to have a nice living room box, then let’s use it for other things. I have a dedicated server with media that I share out with Plex Media Server, so in this step I’ll install the client side.

Plex Home Theater:

sudo add-apt-repository ppa:plexapp/plexht
sudo add-apt-repository ppa:ppa:pulse-eight/libcec
sudo apt-get update && sudo apt-get install plexhometheater

In Steam you can then click on the + symbol, Add a non-steam game, and then add Plex. Use the gamepad (not the stick) to navigate the UI once you launch it. If you prefer XBMC/Kodi you can install that instead. I found that the controller also works out of the box there, so it’s a nice experience no matter which one you choose.

Step 5: In Home Streaming

This is a killer Steam feature, that allows you to stream your Windows games to your new console. It’s very straight forward, just have both machines on and logged into Steam on the same network, they will autodiscover each other, and your Windows games will show up in your Ubuntu/Steam UI, and you can stream them. Though it works suprisingly well over wireless, you’ll definately want to ensure you’ve got gigabit ethernet if you want to stream games 1080p at 60 frames per second.

Conclusion

And that’s basically it! There’s tons of stuff I’ve glossed over, but these are the basic steps. There’s lots of little things you can do, like remove a bunch of desktop packages you won’t need (so you don’t need to download and update them) and other tips and tricks, I’ll try to keep everyone up to date on how it’s going.

Enjoy your new next-gen gaming console!

TODO:

  • You can change out the plymouth theme to use the one for SteamOS - but I have an SSD in the box and combined with the fast boot it never comes up for me anyway.
  • It’d be cool to make a prototype of Ubuntu Core and then provide Steam in an LXC container on top of that so we don’t have to use a full blown desktop ISO.