Automating deployment for PHP and Javascript apps using a chat bot

Automation

I’ve set up a sweet deployment workflow at work: any developer on the team can order the bot in our company chatroom to deploy a web app (cms, API server, or angular apps) from the git branch they choose to staging or production servers (with permission check for the latter).
Yeah, we’re that cool ;)

And the bot is really friendly

There’s a bunch of techs involved here:

I’ll go over each of these with workarounds/fixes for the main hurdles I encountered, but installation instructions will be à la RTFM. Be warned, some unix-fu and ssh-fu are required.

Git server

You are using git, right ?
Aside from making our life easier, git repos are where the deployment server takes the code from. No ftp’ing into the server and copying files manually (ohgodwhy) or rsync’ing. Less confusion and mixups.

If you don’t want to get a Github account, you can set up your own Git server with Gitolite.

Naming things

Before I continue, I’ll just clear up what I mean for a couple of terms I’ll use in the next parts.

  • The “deployment server” is the server which runs the deployment commands. For ease of setup, I’ve pulled the repos of each project on this server, since each project has the instructions to deploy itself.
  • “Prod and staging servers” are the (web)servers running the code/website.

PHP app deployment

You know all the little things you usually do when deploying a new version of your app? run composer, set the right environment, run migrations, clear cache, etc…
Rocketeer can do all that for you, and more. It will version your deploys, and enable you to rollback if something goes awry. You can configure multiple “stages” (servers in staging or prod environment) and have access to callbacks for each step during the deployment.

I’ve used it successfully with Laravel 4 (the project was originally a Laravel package) and Yii 1.*, more specifically with Clevertech’s YiiBoilerplate where deployment is a bit more complex than for the mint Yii.

A note on security

It’s a good idea for the prod/staging servers to have a “deploy” user, as the deployment server will be accessing them and we don’t want to give root access to an automatic tool.
How this impacts the deployment is that you usually restart some services afterwards and this requires sudo permissions !
Fear not, you can give sudo powers on specific commands to your deploy user. Here’s for ubuntu server (12.04+ as far as I know) : 

1
$ sudo vi /etc/sudoers.d/deployer

If you want to give your deploy user ( “deployer” here) permissions to restart nginx and fpm, add the lines : 

1
2
deployer ALL=(ALL) NOPASSWD:/usr/sbin/service php5-fpm restart  
deployer ALL=(ALL) NOPASSWD:/usr/sbin/service nginx restart

Another thing, use SSH keys to give access to your git repos and servers to your deploy user.
Little quirk for Laravel deployment, you’ll need an RSA key, not DSA.
Use SSH keys!

Javascript app deployment

If you’re building js apps, you probably know about Grunt by now. I use grunt to minify and version css and js assets.

I’ve coupled it with Shipit to take care of deployment.
Because of the build required prior to the deploy and the way Shipit works, I had some trouble having the right files copied to the server. In short, Shipit checks out the repo into a “workspace” (a temporary directory) while the Grunt build was being run in the current directory. Solution : 

1
2
3
4
5
6
7
grunt.registerTask('workspacebuild', 'Build project in workspace', function () {
	grunt.shipit.local('cd ' + grunt.config('shipit.options.workspace')  +' && npm install && grunt build', this.async());
});

grunt.shipit.on('fetched', function () {
	grunt.task.run(['workspacebuild']);
});

Note: The build and deploy will be run from the deployment server, so it’ll need to have Node.

The only bone I have with Shipit for now is that the deploy is quite slow at about ~7mins, and it’s not even the build because that takes about 30s.

Chatroom

If you don’t already have one, getting a company chatroom is the next step. I can point you to Hipchat, basic features are free with unlimited users.
Running the deploys from a chatroom allows everyone to see what is going on, and not squash each other’s deploys or have concurrent deploys (I don’t even know how that would play out).

Bot

The most fun part! Now that your projects have a deployment system, it’s time to get a bot and explain to it how to run those deploys.

I use Hubot (there are others). Hubot runs on Node, which is in my ballpark, and there is an official Hipchat connector. I’ve set it up on the deployment server.

For the deployment command, if you have a Github repo, you’re in luck.

If not, creating your own deploy script is not so hard. Just give Hubot’s scripting guide a read. Here is the major part of the script I wrote (you’ll need Hubot Auth)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
# list of projects and command to run the deploy per project
projects = 
	laravelapp:
		type: "rocketeer"
		path: "/home/deployer/repos/laravelapp"
		command: 'php artisan deploy:deploy --on="#{target}" --branch="#{branch}" --quiet --no-interaction'
	jsapp: 
		type : "shipit"
		path:"/home/deployer/repos/jsapp"
		command: 'grunt shipit:#{target} --branch="#{branch}" deploy &>/dev/null'
		
module.exports = (robot) ->
	run_cmd = (cmd, path, msg, completion) ->
		{spawn, exec}  = require 'child_process'
		exec cmd, {cwd : path}, (err, stdout, stderr) ->
			message = stdout
			if not stdout?
				message +=  "\r\n" + "Error: no stdout something went wrong"
			if err
				message +=  "\r\n" + "Error: " + err
				if stderr
					message +=  "\r\n" + "stderr:  " + stderr
			msg.send message
			msg.send completion
			
	robot.respond /deploy\ (\S+)\@(\S+)\ to\ (\S+)/i, (msg) ->
		# match[1]: project,	# match[2]: branch,	 match[3]: target
		if msg.match[3] == "prod"
			target = "production"
		else 
			target = msg.match[3]

		branch = msg.match[2]
		project = msg.match[1]

		unless robot.auth.hasRole(msg.envelope.user, "deploy-" + target)
			msg.send ('you are not authorized to perform this deploy')
			return

		if projects[project] == undefined
			msg.send ('you sure about that project name?')
			return
		
		msg.send('deploying: {project: ' + project + ', branch: ' + branch + ', target: ' + target + ' }')
		cmd = projects[project].command.replace('#{target}', target).replace('#{branch}', branch)
		run_cmd cmd, projects[project].path, msg, 'deploy done: {project: ' + project + ', branch: ' + branch + ', target: ' + target + ' }'

Usage : @[bot] deploy [project]@[branch] to [env]
Giving your users deploy rights :
@[bot] Some Dude has deploy-[env] role

TL;DR

  • Have a repo for each of you projects
  • Add a deployment tool + deploy instructions per environment for each of your projects; pull onto your deployment server
  • Set up a chat bot, connect it to your company chatroom
  • Write a script to have your bot execute the deployment scripts from inside the project directories.
  • PROFIT

Profit

With this setup, each developer is much more autonomous and we deploy all day long.

BTW, ever heard of ChatOps? I recently discovered the term myself, and have yet to read thoroughly about it.

Have fun with your deploys.