Canonical Voices

Posts tagged with 'cloud'

Victor Palau

In order to test k8s you can always deploy a single-node setup locally using minikube, however it is a bit limited if you want to test interactions that require your services to be externally accessible from a mobile or web front-end.

For this reason, I created a basic k8s setup for a Core OS single node in Azure using . Once I did this, I decided to automate its deployment via script.

It requires a Core OS instance running, then connect to it and:

git clone k8
cd k8
./ [myip-address] –> ip associated to eth, you can find it using ifconfig

This will deploy k8 into a single node, it sets up kubectl in the node and deploys skydns add on.

It also includes a busybox node file that can be deployed by:

kubectl create -f files/busybox

This might come useful to debug issues with the set up. To execute commands in busybox run:
kubectl exec busybox — [command]

The script and config files can be access at

If you hit any issues while deploying k8s in a single node a few things worth checking are:

sudo systemctl status etcd
sudo systemctl status flanneld
sudo systemctl status docker

Also it is worth checking what docker containers are running and if necessarily check the logs

docker ps -a
docker logs [container-id]

Read more
Victor Palau

I recently blogged about deploying kubernetes in Azure.  After doing so, I wanted to keep an eye on usage of the instances and pods.

Kubernetes recommends Heapster as a cluster aggregator to monitor usage of nodes and pods. Very handy if you are deploying in Google Compute (GCE) as it has a pre-build dashboard to hook it to.

Heapster runs on each node, collects statistics of the system and pods which pipes to a storage backend of your choice. A very handy part of Heapster is that export user labels as part of metadata, which I believe can be used to create custom reports on services across nodes.


If you are not using GCE or just don’t want to use their dashboard, you can deploy a combo of InfluxDB and Grafana as a DIY solution. While this seems promising the documentation, as usual, is pretty short on details..

Start by using the “detailed” guide to deploy the add on, which basically consists of:

**wait! don’t run this yet until you finished reading article**

git clone
cd heapster
kubectl create -f deploy/kube-config/influxdb/

These steps exposes Grafana and InfluxDB via the api proxy, you can see them in your deployment by doing:

kubectl cluster-info

This didn’t quite work for me, and while rummaging in the yamls, I found out that this is not really the recommended configuration for live deployments anyway…

So here is what I did:

  1. Remove env variables influxdb-grafana-controller.yaml
  2. Expose service as NodePort or LoadBalancer depends of your preference in grafana-service.yaml. E.g. Under spec section add: type: NodePort
  3. Now run >kubectl create -f deploy/kube-config/influxdb/

You can see the expose port for Grafana by running:
kubectl --namespace=kube-system describe service grafana-service

In this deployment, all the services, rc and pods are added under the kube-system namespace, so remember to add the –namespace flag to your kubectl commands.

Now you should be able to access Grafana on any external ip or dns on the port listed under NodePort. But I was not able to see any data.

Login to Grafana as admin (admin:admin by default), select DataSources>influxdb-datasource and test the connection. The connection is set up as http://monitoring-influxdb:8086, this failed for me.

Since InfluxDB and Grafana are both in the same pod, you can use localhost to access the service. So change the url to http://localhost:8086, save and test the connection again. This worked for me and a minute later I was getting realtime data from nodes and pods.

Proxying Grafana

I run an nginx proxy that terminates https  requests for my domain and a created a https://mydomain/monitoring/ end point as part of it.

For some reason, Grafana needs to know the root-url format that is being accessed from to work properly. This is defined in a config file.. while you could change it and rebuild the image, I preferred to override it via an enviroment variable in the influxdb-grafana-controller.yaml kubernetes file. Just add to the Grafana container section:

value: "%(protocol)s://%(domain)s:%(http_port)s/monitoring"

You can do this with any of the Grafana config values, which allows you to reuse the official Grafana docker image straight from the main registry.

Read more
Victor Palau

First of all, I wanted to recommend the following recipe from Digital Ocean on how to rollout your own Docker Registry in Ubuntu 14.04. As with most of their stuff, it is super easy to follow.

I also wanted to share a small improvement on the recipe to include a UI front-end to the registry.

Once you have completed the recipe and have a repository secured and running, you extend your docker-compose file to look like this:

 image: "nginx:1.9"
 - 443:443
 - 8080:8080
 - registry:registry
 - web:web
 - ./nginx/:/etc/nginx/conf.d:ro

 image: hyper/docker-registry-web
 - 8000:8080
 - registry
 REGISTRY_HOST: registry

 image: registry:2
 - ./data:/data

You will also need to include a configuration file for web in the nginx folder.

file: ~/docker-registry/nginx/web.conf

upstream docker-registry-web {
 server web:8080;

server {
 listen 8080;
 server_name [YOUR DOMAIN];

 ssl on;
 ssl_certificate /etc/nginx/conf.d/domain.crt;
 ssl_certificate_key /etc/nginx/conf.d/domain.key;

location / {

# To add basic authentication to v2 use auth_basic setting plus add_header
 auth_basic "registry.localhost";
 auth_basic_user_file /etc/nginx/conf.d/registry.password;

proxy_pass http://docker-registry-web;
 proxy_set_header Host $http_host; # required for docker client's sake
 proxy_set_header X-Real-IP $remote_addr; # pass on real client's IP
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 proxy_set_header X-Forwarded-Proto $scheme;
 proxy_read_timeout 900;

docker-compose up and you should be able to have a ssl secured UI frontend in port 8080 (https://yourdomain:8080/)
If you have any improvement tips I am all ears!

Read more
Victor Palau

Last month I attended DroidCon 2012 and did a talked about using Juju and Ubuntu to deliver Android applications with a Web Service background. The guys at Skillmatters were kind enough to record and edit the video, and now it publicly available.


Read more
Victor Palau

If you thought I had concluded my blog series on demonstrating how Ubuntu is the best environment to write up “connected” or “cloud backend” Android Apps, think again!

So far this is what we covered:

  • Proof that you can access a Juju local environment from the Android Emulator  done!
  • Using a few charms from the charm store plus a custom one, set up a MySQL database that can be exposed through a web service with simple commands/steps - done!
  • Develop a TODO list android app and connect it to the web service, so they talk to each other. – done!

The next step is “How to test that it all works on a production environment”.  If you have tested to death both your Android application and your web service locally, it is time to check if they will still work in real life. How do we do this? With few simple commands, we are going to deploy the same web service into the Amazon Cloud, and  the application in a mobile phone. All managed from the comfort of my Ubuntu Desktop.

Deploying to Amazon Web Services (AWS)

The only pre-requisite here is that you do have an AWS account. Once you are logged into the AWS website, you can find the credentials that you will need to set up your juju environment.  You can find a tutorial on how to set up your Elastic Compute Cloud (ec2) environment –> here.

The required information for Juju is stored in the environment.yaml file in the ~/.juju folder. In the following sample file you can see that two environments have been defined:

  • “local” is the environment that I have been using in my PC to test my web service using LXC containers.
  • “aws” gives Juju the information required to deploy services using my Amazon account.
  • “local” is set as default. This means that if I just run “juju bootstrap” this command applies to the local environment. To bootstrap the AWS environment, I would do “juju bootstrap -e aws”.
default: local
  type: ec2
  control-bucket: juju-faefb490d69a41f0a3616a4808e0766b
  admin-secret: 81a1e7429e6847c4941fda7591246594
  default-series: precise
  juju-origin: ppa
  ssl-hostname-verification: true
  type: local
  control-bucket: juju-a14dfae3830142d9ac23c499395c2785999
  admin-secret: 6608267bbd6b447b8c90934167b2a294999
  default-series: precise
  juju-origin: distro
  data-dir: /home/victorp/myjuju_data

With my environments now configured, it’s time to deploy my services. This first step is to bootstrap my environment:

juju bootstrap -e aws

With the command completed successfully, I can check the status and I will see that the juju control instance is now up and running in Amazon:

juju status -e aws
2012-09-19 11:43:34,248 INFO Connected to environment.
    agent-state: running
    instance-id: i-0e4f7174
    instance-state: running
services: {}
2012-09-19 11:43:35,322 INFO 'status' command finished successfully

Lets continue deploying the services. As I am only doing testing, I want to pay the minimum for it, it will ask juju to set a constrain to only use micro instances. Then I will deploy a mysql and a lamp service:

juju set-constraints instance-type=t1.micro -e aws
juju deploy mysql -e aws
juju deploy --repository ~/mycharm local:lamp -e aws
juju set lamp website-database="android_todo" -e aws
juju set lamp website-bzr="lp:~vtuson/+junk/mytodo_web" -e aws
juju expose lamp -e aws
juju add-relation lamp mysql -e aws

With all my services now running I can go to the Amazon EC2 instance console and see how they have been deployed as micro instances. I can now also enter the public address for my lamp service and see the ToDo list table as expected.

Testing the Android App on a real phone

Running Juju status, I can retrieve the public url for the lamp service and I replace the uri vairable in the TodoSourceData class with “”.  The next step is to plug a HTC Desire set up on debug mode into my laptop’s usb port. The rest is taken care by the Android Eclipse plug-ins. When I click  the run project button, I am presented with a choice of targets:

I just need to press “OK” and my ToDo app is launched in the handset. Opening the menu options and pressing “Sync” fetches the ToDo data from the Amazon services, as expected:

That is all for today! Let me know if you have any suggestions on what else I should cover on this blog series.

Read more
Victor Palau

Time for the next chapter of my blog series about demonstrating how Ubuntu is the best environment to write up “connected” or “cloud backend” Android Apps.  As you might know, the Android SDK allows you to set up a sandboxed environment to develop Mobile apps in your desktop, using Juju  you can do the same for Cloud apps.

To walk you through how to put these great development tools together, I set out to accomplish:

  • Proof that you can access a Juju local environment from the Android Emulator  done!
  • Using a few charms from the charm store plus a custom one, set up a MySQL database that can be exposed through a web service with simple commands/steps  - done!
  • Develop a TODO list android app
  • Connect the android app and the webservice, so they talk to each other.

Today I am going to cover the bottom two bullet points in one go!  For this post, I am going to assume that you know a bit of Android development. If you want a great source of introductory material check Lars Vogel’s website.

I have created a simple ToDo Android application that can store tasks into a local SQLite db and allows you to “Star” important items. The code for my Simple Todo app is hosted in Launchpad. I have written my application for Android 2.3, but you can use a later version.

Reading remote data from the MySQL server is confined to a small class that retrieves a JSON object and translates it into a TodoItem object.  Equally , the server code that prints that content table into a JSON object is extremely simple. Beyond this, you can go crazy and implement a RESTfull API to sync the databases.

At the moment, I am just inserting Server side data into the Local database and making sure I don’t add duplicates.Here is a video that shows how easy is to work and test both environments:

Or Click Here for the video.

The same environment should then work if you are running the Android application on an external phone. But that is another blog post ;)

Read more
Victor Palau

It is time to continue with my blog series on demonstrating how Ubuntu is the best environment to write up “connected” or “cloud backend” Android Apps.  As you might know, the Android SDK allows you to set up a sandboxed environment to develop Mobile apps in your desktop, using Juju  you can do the same for Cloud apps.

To walk you through how to put these great development tools together, I set out to accomplish:

  • Proof that you can access a Juju local environment from the Android Emulator  done!
  • Using a few charms from the charm store plus a custom one, set up a MySQL database that can be exposed through a web service with simple commands/steps
  • Develop a TODO list android app
  • Connect the android app and the webservice, so they talk to each other.

Today is time to set-up our own Cloud environment in a PC.  A good starting point for a web application is a LAMP stack. If you hope for you service to get popular, you might want to split out the database service and the web service into their own instances so they can be scaled easily.

When I set out to do this, I wanted to write an extremely simple PHP page that would expose a data table from a Mysql server.  I looked up the available charms on the store. I found that you had a lot of charms for existing apps, but none that had  the bare bones of a LAMP stack and just allowed you to install your own web pages. However, I did find a charm to deploy Mysql and a very handy tool to manage it (phpmyadmin). Taking this as a starting point I developed a generic LAMP charm.

The first thing the LAMP charm does is to install and configure a new instance with Apache and PHP5 in an LXC container. The new charm will then copy any files under /website  inside the charm folder structure, into /var/www. It also allows you to specify a Bazaar branch. It will clone the branch into the webserver and copy the contents to /var/www. I keep my TODO application for this exercise here.

Using juju, you can set up a relationship with a Mysql service. A Mysql database is created by default at this time. You can change the name of the database as a configuration options. If you provide a file called mysql_config either on the /website folder or in the root of you Bazaar branch, this will be used to configure further the Mysql database.

The charm will store the details of the relationship in /var/webconfig. In there you can also find opendb.php, a basic script that will open the connection to the MySQL database for you.

So lets get started! Clone the Lamp branch to a folder of your choice. I have it under “~/Mycharm/precise/lamp”. (If you need to get started with juju check my previous post here)

juju bootstrap

#step1 - deploy Mysql as a service.
juju deploy mysql

#step2 - deploy phpmyadmin tool
juju deploy phpmyadmin
juju set phpmyadmin password="password"

#this makes phpmyadmin public - exposes port 80/tcp
juju expose phpmyadmin

#step3 - link phpmyadmin and mysql so the can talk to each other
juju add-relation phpmyadmin mysql

#step4 deploy lamp from your local folder
juju deploy --repository ~/mycharm local:lamp

#step5 set up your bazaar branch and publish your website
juju set lamp website-bzr="lp:~vtuson/+junk/mytodo_web"
juju expose lamp

#step6 configure your database and link lamp and mysql services
juju set lamp website-database="android_todo"
juju add-relation lamp mysql

Once Juju has completed all the commands, run “juju status”. Now go to the ip address for the LAMP services and visit the databaseweb.php page (e.g. , you get this rather ugly table:

Id Todo Is it starred
1 hello this is a sample 1
2 yes it works! 0

This table is displaying the contents of a Mysql database. You will get the same data if you visit your Myphpadmin service:

So there you are a full and scalable LAMP stack in your PC!

If you want to update the web service by pulling the latest version of the Bazaar branch, just run the following commands:

juju set lamp bzr-update="yes"
juju set lamp bzr-update=

With my web developer “hat” on, I can now hand this charm over to my Android app developer. They will be able to set up exactly the same environment locally to test their app without using expensive server time.

So “Using a few charms from the charm store plus a custom one, set up a MySQL database that can be exposed through a web service with simple commands/steps” – DONE!

Read more
Victor Palau

Have you ever wondered what is all the fuss about ARM Servers? Yes? good , good.

Have you ever wish you had some crazy Zooming UI presentation that told you all about it? what.. no!? Well though.. because now you have one :)

If you haven’t heard of Prezi, it is a new way to generate more dynamic presentations. I will give you a few tips:

  • When viewing a Prezi, make sure you click on the “Full Screen” for maximum effect (under More..)
  • You can also click on autorun if you fancy the animation to happen on its own
  • You can also use the right and left arrows to move around the animation at your leisure
  • If you want to zoom into something, just double click on it!

Without further ado, I give you ARM Server on  a Prezi:


Read more
Victor Palau

I was aware that data centers around the world were starting to be talked about as an environmental problem, but perhaps the statistic that data centers have the same carbon footprint than the aviation industry (about 2% of the global carbon footprint pie) really put things in perspective for me.

The Open Data Center Alliance ”Carbon Footprint Values” document starts its executive summary with:

According to market research and consulting firm Pike Research, data centers around the world consumed 201.8 terawatt hours (TWh)
in 2010 and energy expenditures reached $23.3 billion. That’s enough electricity to power 19 million average U.S. households. The
good news is that, according to Pike Research, the adoption of cloud computing could lead to a 38% reduction in worldwide data center
energy expenditures by 2020.

The prediction that cloud computing will lead to large savings of energy consumption can be justified by economies of scale. Todays’ enterprise data centers average 20-30% computing power utilisation. The same data center serving Infrastructure As A Service (IaaS) is expected to run at 80-90% occupancy. This plus the opportunity for enterprises to transform a fix cost of ownership into a flexible service subscription will lead to consolidation of data centers.

Economies of scale will also allow large scales data center providers to invest in propose build more sustainable and cheaper to run  buildings. A good examples of this is the server and data center specifications shared by Facebook via the Open Compute Project, or Google’s water powered and cooled at-Sea data centers.

As discussions of hefty fines for London by the European Union are currently taking place, sustainability is becoming less a matter of corporate responsibility and more of legal compliance.

However, Cloud computing is bringing applications to individuals that were only available to enterprises a few years ago.  This will multiply the need for data centers across the globe beyond the current demand. We need to work beyond finding cheaper ways to cool and power servers and start tackling the real problem, servers themselves need to be exponentially more efficient.

Thankfully both Intel and ARM have quick-started the race for low power servers that will eventually make the Cloud really green.

Read more