Running Cron Jobs on Cloud Foundry

Many cloud applications need to run some sort of scheduled tasks for chores like processing statistics or doing internal housekeeping. There are two different strategies to implement scheduled tasks to cloud applications running on Cloud Foundry. You can either build scheduling and tasks into the application itself or schedule and run the task in separate containers

Some application frameworks like Spring include some built-in scheduling support. However, this scheduling support does not include a distributed coordination mechanism. This means that an application horizontally scaled to multiple instances will each run the scheduled task individually. Depending on the nature of the task, this may cause observable side effects like sending emails to your customers to be repeated.

It's thus preferable to have a central entity for scheduling. You could of course use e.g. a Java Spring App that needs approximately 1G of RAM to do that for you, but that would be very wasteful. Instead, we can build a simple cron scheduler that runs on 16 MB of RAM to get reliable task scheduling for just a few cents per month.

The task scheduler can then execute arbitrary scripts or code, for example to:

• invoke an https endpoint on your application to perform the task
• queue a message on RabbitMQ for processing by a worker
• trigger execution of the job in a seperate Cloud Foundry Task Container

meshcloud\'s cf-cron scheduler

Our sample repository demonstrates how to run scheduled tasks on Cloud Foundry with a very small footprint (8 to 16 MB RAM) using a traditional crontab. Traditional cron daemons need to run as root and have opinionated defaults for logging and error notifications. This makes them unsuitable for running in a containerized environment like Cloud Foundry. Instead of a system cron daemon we\'re thus using supercronic to run our cron tab.

How it works

This application is built using the binary buildpack and executes supercronic on the crontab file. The crontabfile specifies all your cron jobs. To add additional jobs, simply add a new line which specifies a schedule and command to the crontab.

Note: By default, supercronic will log all output to stderr so we redirect that to stdout in our cf manifest.

You can also include additional scripts and binaries to execute more complex actions. This example allows you to install apt and debian packages to use in your cronjobs. You can specify these packages in apt.yml and they will be installed during staging by apt-buildpackcourtesy of the magic multi-buildpack.

After cf pushing this sample app to Cloud Foundry, you can see that it happily executes the jobs from the crontab in the log output:

2018-03-05T10:59:00.00+0100 [APP/PROC/WEB/0] OUT time="2018-03-05T09:59:00Z" level=info msg=starting iteration=237 job.command="echo "hello world, every 2 seconds"" job.position=1 job.schedule="*/2 * * * * * *"
2018-03-05T10:59:00.00+0100 [APP/PROC/WEB/0] OUT time="2018-03-05T09:59:00Z" level=info msg="hello world, every 2 seconds" channel=stdout iteration=237 job.command="echo "hello world, every 2 seconds"" job.position=1 job.schedule="*/2 * * * * * *"
2018-03-05T10:59:00.00+0100 [APP/PROC/WEB/0] OUT time="2018-03-05T09:59:00Z" level=info msg="job succeeded" iteration=237 job.command="echo "hello world, every 2 seconds"" job.position=1 job.schedule="*/2 * * * * * *"
2018-03-05T10:59:00.05+0100 [APP/PROC/WEB/0] OUT time="2018-03-05T09:59:00Z" level=info msg="cf version 6.34.1+bbdf81482.2018-01-17" channel=stdout iteration=7 job.command="cf --version" job.position=0 job.schedule="*/1 * * * *"
2018-03-05T10:59:00.05+0100 [APP/PROC/WEB/0] OUT time="2018-03-05T09:59:00Z" level=info msg="job succeeded" iteration=7 job.command="cf --version" job.position=0 job.schedule="*/1 * * * *"

Scheduling Cloud Foundry Tasks

While the cron container here is designed to be small and lightweight, you may want to use it to trigger more resource intensive tasks and processes. When a simple curl to an http endpoint is not enough to kick off such a task on your existing app, Cloud Foundry Tasks are a great solution to run these processes.

This sample repository thus includes instructions to install the cf cli tool which you can use to trigger such a task using a meshcloud Service User.

by Johannes Rudolph

Deploying Meteor Apps on Cloud Foundry

This repository demonstrates different ways to deploy Meteor applications on Cloud Foundry. There are at least three different ways to deploy Meteor JS applications on Cloud Foundry:

• using a specialized meteor-js buildpack
• using the general node-js buildpack
• using your own docker container image

While a specialized meteor buildpack like meteor-buildpack-horse or cf-meteor-buildpack is a useful option, recent versions of meteor support simplified deployment on servers that provide just a node.js runtime environment. For this demonstration repository, we\'re focusing on this approach because the node.js buildpack is more widely used and better maintained.

The application we\'re going to deploy is located in the ./try-meteor folder of this repository. We also assume you have provisioned a Mongo DB service on your space. In Cloud Foundry you can create a dedicated service instance suitable for use with meteor like this:

cf create-service MongoDB M meteor-mongo

Using the node.js Buildpack

Build a meteor app bundle

On your local machine with the meteor cli installed, build a distributable package into the deploy-buildpack folder:

cd try-meteor && meteor build ../deploy-buildpack/. --server-only --architecture os.linux.x86_64

Building generates a try-meteor.tar.gz file with the meteor application bundled as a plain node.js application, with some helfpful instructions in its README.md file:

This is a Meteor application bundle. It has only one external dependency:
Node.js v8.9.3. To run the application:

$ (cd programs/server && npm install)
$ export MONGO_URL='mongodb://user:password@host:port/databasename'
$ export ROOT_URL='http://example.com'
$ export MAIL_URL='smtp://user:password@mailhost:port/'
$ node main.js

Use the PORT environment variable to set the port where the
application will listen. The default is 80, but that will require
root on most systems.

node.js buildpack wrapper

To deploy this tar.gz file on Cloud Foundry with the node.js buildpack we need to:

1. upload and unpack the tar.gz bundle
2. run npm install on the extracted bundle
3. set the correct environment variables using a launcher js script

We can easily achieve that through a custom package.json that uses npm\'s postinstall and start script to execute these actions. You can find the package.json and all required files for the deployment in the ./deploy-buildpack folder.

Note: at the time of writing the bundles generated by meteor 1.6.0.1 machine lack the meteor-deque dependency so we just explicitly add that by hand.

{
  "name": "try-meteor",
  "private": true,
  "scripts": {
    "start": "node launcher.js",
    "postinstall": "tar -xf try-meteor.tar.gz && (cd bundle/programs/server && npm install)"
  },
  "engines": {
    "node": "8.9.3"
  },
  "dependencies": {
    "meteor-deque": "~2.1.0",
    "cfenv": "1.0.4"
  }
}

Have a look at the launcher.js file if you want to change service names etc. The final bit that we need is a Cloud Foundry Manifest file to describe our application:

---
applications:
- name: try-meteor-app
  memory: 512M
  instances: 1
  buildpack: https://github.com/cloudfoundry/nodejs-buildpack
  services:
    - meteor-mongo

We're all set now, a simple cf push and your app should be up and running on the cloud.

Using a Docker Container

The next option is to use a docker-based deployment of the application. This requires that we build our own docker image of the application and publish it to a docker registry.

You can find the code for the docker-based deployment of our sample application in the MARKDOWN_HASHd786272e16707f3074a294d21f8b83a0MARKDOWN<em>HASH</em> folder. The docker image used in this example is based on the node:8-alpine base image. However, before we can build our container we need to build our meteor application and extract it:

cd try-meteor && meteor build ../deploy-docker/. --server-only --architecture os.linux.x86_64
cd ../deploy-docker && tar -xf try-meteor.tar.gz && rm try-meteor.tar.gz

The docker deployment demonstrated in this repository also uses the same launcher.js script introduced above to automatically initialize meteor environment variables from their Cloud Foundry counterparts. With that out of the way, let\'s build and push the docker image

docker build -t meshcloud/meteor-cf-example .
docker push meshcloud/meteor-cf-example:latest

With the container available in a docker registry, we can push it to Cloud Foundry by specifying the docker image in the manifest:

---
applications:
- name: try-meteor-app-docker
  memory: 512M
  instances: 1
  docker:
  image: meshcloud/meteor-cf-example
  services:
  - meteor-mongo

Now all that\'s left to do is a simple cf push and your app should be up and running on Cloud Foundry in no time.

iDevBook01:deploy-docker jr (master *) $ cf push
Pushing from manifest to org meshcloud-demo / space aproject as c9f7d64c-404d-4b29-b719-b2359f6c8157...
Using manifest file /Users/jr/dev/demo/meteor/deploy-docker/manifest.yml
Getting app info...
Updating app with these attributes...
name: try-meteor-app-docker
docker image: meshcloud/meteor-cf-example
command: node launcher.js
disk quota: 1G
health check type: port
instances: 1
memory: 512M
stack: cflinuxfs2
services:
meteor-mongo
routes:
try-meteor-app-docker.cf.eu-de-darz.msh.host

Updating app try-meteor-app-docker...
Mapping routes...

Stopping app...

Waiting for app to start...

name: try-meteor-app-docker
requested state: started
instances: 1/1
usage: 512M x 1 instances
routes: try-meteor-app-docker.cf.eu-de-darz.msh.host
last uploaded: Mon 12 Feb 10:25:29 CET 2018
stack: cflinuxfs2
docker image: meshcloud/meteor-cf-example
start command: node launcher.js

state since cpu memory disk details
#0 running 2018-02-12T10:12:23Z 0.1% 38.2M of 512M 1.3M of 1G

iDevBook01:deploy-docker jr (master *) $

by Johannes Rudolph

Hosting your WordPress Website in Cloud Foundry

If you are looking for a way to host your WordPress website and want to enjoy the freedom of the cloud, here is how you do it! This tutorial is based on Cloud Foundry.

Getting started

At first we have to download and unzip WordPress to prepare it for the Cloud Foundry deployment. In our example, we download the English version from the WordPress homepage. If you need another version, you can find it under this link.

$ wget https://wordpress.org/latest.zip
$ tar xzvf latest.zip
$ cd WordPress

If you are using a Mac and don't have wget installed, you can install it via hombrew. brew install wget

Now we have to start with the customisation for the cloud use. Being in the WordPress directory now, we have to create the ".bp-config" folder where we'll create the "options.json" file that contains the PHP extensions we need to run WordPress.

$ mkdir .bp-config
$ nano .bp-config/options.json
$ cat .bp-config/options.json
{
"PHP_EXTENSIONS": ["mysql", "bz2", "zlib", "curl", "mcrypt", "mbstring", "mysqli", "gd", "zip"]
}

Prepare Database

Having done that, we have to prepare our database. The first thing to do when a Database is needed in Cloud Foundry is to start a service. To see which services are available in your CF Infrastructure type:

$ cf marketplace
service plans description
MongoDB S*, M*, L* MongoDB Instances
MySQL S*, L*, M* MySQL Instances
PostgreSQL S*, M*, L* PostgreSQL Instances
RabbitMQ S*, M*, L* RabbitMQ Instances

For WordPress we need a MySQL database. You have to decide which plan you want to choose and give the database a descriptive name. Now you can create the service.

$ cf create-service MySQL S WordPress

After doing so, we have to link the credentials of our newly created database in our wp-config. One of the key advantages of cloud deployment is that you can configure your software completely dynamically, so you can deploy it anywhere anytime. To do so, we have to read the database connection information dynamically out of the Cloud Foundry environment instead of referencing to a concrete database. The "diff" command shows the difference between the files. You don't have to run it yourself it only shows you what you have to change in order to get your site running ("-"-Lines are old +-"Lines" are new).

$ cp wp-config-sample.php wp-config.php
$ nano wp-config.php
$ diff wp-config-sample.php wp-config.php

--- wp-config-sample.php 2015-12-16 10:58:26.000000000 +0100
+++ wp-config.php 2017-10-13 14:34:31.000000000 +0200
@@ -18,18 +18,27 @@
* @package WordPress
*/

+
+
+$services = json_decode($_ENV['VCAP_SERVICES'], true);
+$service = $services['MySQL'][0]['credentials']; // pick the first MySQL service
+
// ** MySQL settings - You can get this info from your web host ** //
/** The name of the database for WordPress */
-define('DB_NAME', 'database_name_here');
+define('DB_NAME', $service['database']);
+
+

/** MySQL database username */
-define('DB_USER', 'username_here');
+define('DB_USER', $service['username']);
+

/** MySQL database password */
-define('DB_PASSWORD', 'password_here');
+define('DB_PASSWORD', $service['password']);

/** MySQL hostname */
-define('DB_HOST', 'localhost');
+define('DB_HOST', $service['host'] . ':' . $service['port']);
+

/** Database Charset to use in creating database tables. */
define('DB_CHARSET', 'utf8');
@@ -46,14 +55,14 @@
*
* @since 2.6.0
*/
-define('AUTH_KEY', 'put your unique phrase here');
-define('SECURE_AUTH_KEY', 'put your unique phrase here');
-define('LOGGED_IN_KEY', 'put your unique phrase here');
-define('NONCE_KEY', 'put your unique phrase here');
-define('AUTH_SALT', 'put your unique phrase here');
-define('SECURE_AUTH_SALT', 'put your unique phrase here');
-define('LOGGED_IN_SALT', 'put your unique phrase here');
-define('NONCE_SALT', 'put your unique phrase here');
+define('AUTH_KEY', 's1,d-+8?BFD$n`|0r+sk[/O<e7.86={p');
+define('SECURE_AUTH_KEY', '=|DiG2C+Y1FLKFIZv_S#GZUvfz~Z<FkI#qGV=U25r5ofI). iZZ:atO7R6ru@ms]&C/=K5VxImw#du$}PfiO=?Qc&e-1}Jc*yI$;/87r{!}-Qq9');
+define('SECURE_AUTH_SALT', '.qUX{|m?o4j-(Bs2%6|Sf=^wT<y1dJ-+RV&BTfhEqNi`VB,+?i)DPC5r-FcJe9d<');
+define('LOGGED_IN_SALT', '&ah.M&Vij;X;[85Ox,}Z10PT#f ssFp5UjNU<?5LnK+]1ICBq*E{%8H@v(_Y#Jwz');
+define('NONCE_SALT', '2f3kM}yX5gTJ]AD~3P(/P~N#L&y*,A-0ldJ-wTr-h(Jild]IAW{K])^5<0qP `');

We initialised a php variable called "services", got the 'VCAP_SERVICES'-Json out of "$_ENV" and decoded it. Then we grabbed the first MySQL service credentials and saved it to the variable "service". Now we have the credentials in our service variable and can use them to initialise WordPress. Due to security concerns, we removed the salts of our example project. Just follow the description in the config file to create your own.

If you have problems establishing the database connection, you can check the content of the environment variable by using the following command.

cf env $APPNAME

Pushing WordPress to Cloud Foundry

Before we can finally push the app, we have to create the "manifest.yml" file.

---
applications:
- name: wordpressmeshcloud
memory: 256M
services:
- wordpress

Let Cloud Foundry do it's magic now.

cf push

How to Proceed

If you followed the steps, you should now be able to reach your WordPress blog with the browser of your choice. To get the application's URL and it's status you can use the CF CLI again. Simply type:

cf app wordpressmeshcloud

You should be able to see your url now, it's called "routes:" in Cloud Foundry.

For productive use make sure you use a datastore to persist your data. You can do so by using FUSE or a Wordpress-Plugin for Cloud Storage.

by Christina Kraus