Deployment

PHP is the Future

Catchy title, eh? I have to admit, I used a bit of an inflammatory title to pull you into the rest of the post, but I think you’ll like what I have to say. The title is true, but probably not in the way you think. No, PHP is not going to be the language that comes out on top, not the “one language to rule them all” sort of thing. Instead, I’m proposing that PHP, the flexible and powerful language that it is, has a very large roll to fill in the future of the web.

There’s lots of little things that the language has done along its life to help make the world wide web a better place to live. Most importantly it’s given the world an Open Source alternative to some of the other closed languages that some of the major companies of the world offer (okay, so I’m pretty transparent). PHP packs a lot of power in a little package, and the general developer community has definitely taken notice. This plucky little language that started as a handy tool for Rasmus to keep track of things in his personal pages has evolved into something that major companies around the world are implementing into their core software. Hear that? *Core* software. PHP isn’t some fly-by-night language that’s the latest fad being passed around like so many YouTube videos. PHP is strong, its powerful and it is most definitely here to stay.

But I digress…let’s get back to the point that the title of this post was trying to make. PHP is the future. No really, it is – trust me on this. Of course, there’s a difference between being the only path to the future and one of many means to getting there, but we’ll toss that aside for now. We want to focus on PHP and what it can do to make all three Ws in the WWW a better place to develop.

So, lets talk a little bit about what PHP has to offer to the world at large.

From its humblest of beginnings, one of the core values of the PHP project has been to provide the most power in the best code possible. Sure, it has its quirks and it can be a little rough around the edges, but line of code for code, PHP packs some serious features into a neat little package. There’s even the extension system that makes it super easy to add in new bits of functionality without even having to recompile the main engine. Oh, and did I mention it’s free? Open Source, well supported and very very popular among both the usual Open Source crowd and among other certain large entities (even those that might live in Redmond)

Every time I look the future of the web (linking services, making things seamless, integrating technologies), I can see PHP at every major crossroads. Its the flexibility of the language that does it. Sure, there’s still a bit of a niche that PHP fits into, but once you take a step out of that safe little shell, you really start to realize what the language has to offer and where it fits in the world. PHP has the potential for being the glue that binds the web together. There’ll always be other languages out there – to dismiss them would just be silly – but PHP, with its flexibility and power really has the feature set to help propel Open Source web development into the spotlight and shine for what it is.

The web needs a language that’s quick to adapt, easy to configure, comes in at a low cost and is popular enough to find good, talented developers for (yup, that’s a big point too). If you ask me, PHP fits all the above and more.

I believe that PHP can be the future of the web and be a key player in the web applications to come.

Working with WebTests (Help?)

Well, I wish I could say that this was going to be a guide to getting WebTests (an automated front-end testing tool that sits on top of Ant) but it seems as though I’m having a bit of a Java issue to content with myself.

Here’s what I’ve done so far:

First, I noted happily that Ant was already installed on my MacBook’s OS X install (seems a strange choice to have bundled, but who am I to judge), making for one less thing that I’d have to get up and running
Next I grabbed the latest install of WebTest from the Canoo website and unpacked it to a testing directory. It comes as a nice, neat binary so there’s no messy compiling to worry about.
I wrote up a simple test and PHP page to try it all out:
[php]

[/php]
Finally, I gave it all a whirl: “bin/webtest -buildfile /my/path/mybuild.xml

This is where I hit the first snag…apparently since OS X’s Java install is just a bit different than some of the normal ones, a few of the files aren’t where they need to be. In my case, I was getting an error:

BUILD FAILED
/www/tests/mytest.xml:5: 
java.lang.NoSuchMethodError: 
org.apache.xpath.compiler.FunctionTable.installFunction
(Ljava/lang/String;Ljava/lang/Class;)I

After poking around a bit on the web, I happily came across this that recommends copying a jar file over to the Java extensions directory. A simple “cp” later and the build made it past that point. Unfortunately, not much past it, though – another Java error popped up that seems a bit more difficult to find information on:

BUILD FAILED
/dev_tmp/webtest/webtest.xml:265: 
java.lang.NoClassDefFoundError: 
org/apache/xml/serializer/ExtendedContentHandler

So far I haven’t been able to find much that’d help me solve this one, so if there’s anyone out there that’s seen it before, help would be appreciated. It seems like it actually runs the test just fine (according to the output reports) but it’s not very useful if the build always fails.

MQ+PHP – Linking IBM’s WebSphere MQ to PHP

During a recent project at work I had to get PHP linked with IBM’s WebSPhere MQ software we have running on another internal server. Our goal was to use our existing web service to take the requests from external vendors and push their XML data back into the queue inside our firewall. Thankfully there’s an extension in PECL that does just that.

Here’s the basic steps I took – hopefully it’ll be useful to someone else out there in the same spot I was. This all assumes you’re working on a web server that doesn’t have an MQ server installed already:

Get the extension: Head over to the PECL page for mqseries and download the latest version. Unpack it into a directory on your local server
Get the MQ client libs: You’ll need to go to IBM’s website to download the latest client/libraries for your install (you’ll need an IBM ID to get to the downloads):
- Go to the IBM page for the MQ client listing
- Look for the “WebSphere MQ Clients” link under the “Related products and technologies” section and click on it
- Scroll down to the “Download Package” section and choose from one of the mirror locations
- Select your package from the list (I went with “Linux for System x86” for our setup)
- Click on the download link and fill out some required information (you didn’t think you were getting off that easy, did you?)
- Agree to the terms and conditions and you’ll get a “Download Now” link
- Drop the archive file (tar, tar.gz, etc) into your server and unpack into a temporary directory (mine had an issue unpacking into the local directory, not a subdirectory)
Install the package(s): Once you have the IBM software extracted, you should have a series of packages. You’ll need to install the “MQSeriesSDK” to get the right libraries in place to compile the PHP extension
Build the mqseries extension: Go into the mqseries directory and run “phpize”, “./configure” and “make” to create the .so file. The process should drop it into the default extensions directory.
If needed, move it: Be sure that the shared module for the extension is in the right directory for the PHP install to find it. (You can make a phpinfo() page if you’re not sure where that is.)
Update your php.ini: Add in a line to include the extension in your current setup. Remember, after any changes to the php.ini, you need to restart the web server.

Now for the fun part – if everything’s working and the extension shows up in your phpinfo() as active, give this script a shot and see if you can connect to your MQ server:

[php]
$mq_host_ip =’127.0.0.1′;
$queue_name = ‘HOST.REMOTE.Q’;
$mq_server = ‘WBRK_QM_U49’;
$mqcno = array(
‘Version’ => MQSERIES_MQCNO_VERSION_2,
‘Options’ => MQSERIES_MQCNO_STANDARD_BINDING,
‘MQCD’ => array(
‘ChannelName’ => ‘CLIENT.CHANNEL’,
‘ConnectionName’ => $mq_host_ip,
‘TransportType’ => MQSERIES_MQXPT_TCP
)
);

// Connect to the MQ server
mqseries_connx($mq_server,$mqcno,$conn,$comp_code,$reason);
if ($comp_code !== MQSERIES_MQCC_OK) {
trigger_error(‘Cannot open connection to server: ‘.$mq_server,E_USER_ERROR);
}else{
echo ‘Connection good!’;
}
[/php]

Obviously you’ll need to adjust the settings to fit your server, but at least this gives you a start.

Working towards a better deployment (Part 4)

I’m back with the fourth part of my look at deployment of PHP applications to focus on the actual deployment technologies. I’ve already talked some about version control, build tools and unit testing in the previous parts of the series and this new information should round it out nicely with a simple approach to the push part of the deployment.

As it stands right now, the final step in our deployment is one of the simplest. We have some bash scripts that are aliased to rsync commands for pushing files out to each of our different sites. Since the assumption is that anything that’s in the QA/staging area is good to go, the rsync commands take everything that’s out on production and, if there’s any differences, overwrites the remote version. It’s light, simple and makes it easy to push out an entire site in one go. Things can get a bit tricky of you only want to deploy a portion of the site at a time though. There haven’t been many cases where we needed to do this, but it would have been nice to have a way to break it up a bit more.

I posted two TwtPolls in the past few days asking some of the other developers in the community (some outside of PHP too) about which method they preferred for deploying their sites out to a production server. Here’s the results of the latest poll (hopefully it’ll stick around for a while – I’ll mention the numbers too, just in case).

There were 45 votes total on ten different options…here’s how it came out:

FTP/SFTP	[ 20% (9 votes) ]
SSH/SCP	[ 4% (2 votes) ]
RSYNC	[ 9% (4 votes) ]
SVN (via ssh or export)	[ 49% (22 votes) ]
Capistrano	[ 7% (3 votes) ]
Ant	[ 2% (1 votes) ]
Cut & Paste	[ 2% (1 votes) ]
Other Manual Process	[ 2% (1 votes) ]
Other Automatic Process	[ 2% (1 votes) ]
A mashup of several	[ 2% (1 votes) ]

As you can see rsync, our current choice, only came in third under FTP/SFTP and the overwhelming favorite, Subversion deployment. Some of them choose to make the copy of the code on the production server a checkout of the current version (svn up when it needs refreshing) and some export from their current version and push those files out. Unfortunately, we’re still stuck with CVS for the time being (coming soon, hopefully!) so working with branching, tagging and all should be quite a bit easier.

This final step can either be a manual one (how we do it now) or as a part of another process (how I want to do it). I currently have a final step in my Phing build file that, if all goes well during the build, will call one of our bash scripts to push the code directly from there. This helps to keep things nice and tidy and makes it a simple “single process” tool that sysadmins or QA folks can use to ship out the code that’s been cleared for public consumption.

Now that I’ve gotten to the end of the descriptions on all of this stuff, let’s head back to the beginning and look at some of the actual code, configurations and technologies related to deployment. I’ve seen some good suggestions as to a step I’ve been missing – optimization. Keep an eye out for this first in the “Deployment Tech” series, a continuation of this general look at deployment of PHP applications.

Working towards a better deployment (Part 3)

So, on the to third part of my little series (part one, part two) on making a shiny new deployment system for our sites. What’s that? Yep, I said sites – plural. I’m working all of this up on a “one site” concept, but in reality we have five or six websites that this process will eventually apply to. They’re all pretty similar, thankfully, and all use a lot of the same libraries. This makes the next step of my little process so much easier – the unit testing.

If you’re not unit testing your code, drop what you’re doing (well, finish this article at least) and go learn as much as you can about it. It can help take that feeling of dread away from your code updates and can help make things go a bit smoother in your deployment. Imagine having an automated way to check and ensure that the results of your scripts/libraries are still working exactly how the should. Unit tests can help you verify that things that should pass the tests still do and that your failures still fail in all the right ways.

One of the more popular unit testing softwares out there is PHPUnit (from Sebastian Bergmann). Its a super-easy way to set up a suite of tests that can be run as a part of your deployment process. The tests run on your scripts with methods corresponding to each of the methods in the target class. You use assertions inside the tests to check for things like pass, fail, true, false, if an array value is set, etc. The methods are called and the output is verified – if everything’s good, the test passed. If there’s a problem (something non-standard) the test fails which can cause the entire build process to fail too.

You can test more than just simple single method calls too. Say you have a class that creates a user by logging them in and setting up a session for them. If your tests require this, you can make the user session in your “setup” method and use the “teardown” method once things are finished to log the user out and destroy any of the resources you might have created.

These unit tests have found a home in my build process right after the lint check on each of the new files. Phing allows you to defne a tests directory for it to look in for any unit tests it should apply. The PHPUnitTask takes care of the dirty work with options to fail the build if the tests have an error and gather code coverage information as they’re tested. You can also set up a PHPUnitReport task to get more information out of your build. This makes reports and, with the help of some XSLT templates, transforms the results into something n bit more human readable.

All that’s really left after this (with the exception of a few other options) is the push of the up-to-date, tested code out to the production server…but that’s for the next article to cover.

Oh, and for those that are wondering, I’m going to be getting to the tech behind all of this pretty soon. Having information about the setup is one thing, but having the actual setup behind it all helps even more. I’m going to go back over all of this from start to finish to show you how it all fits together. Just hang in there with me!

Working towards a better deployment (Part 2)

In the previous part of this series, I looked at some of the current methods we use for code development and deployment as well as some of the goals I put out there for myself in the quest to find a good deployment system that would work for us and our code. I’m back again to look at the next part of the process – the build.

Once you have the process defined for the developers to follow, you need to lay the foundations for the administrator to handle the rest of the process. Once we know that the code is good and all is right with the world, the deployment phase can get started. Ideally, the admin shouldn’t have to do much at all – well, besides making sure the right code gets merged into the trunk, that can be a little tricky. A correctly defined process should handle the rest, though – running tests on the code to ensure correct functionality, locating any syntax errors it might have, etc – all automagically.

The tool I’ve chosen to act as a base for all of this is Phing, a build tool based on Apache’s Ant that uses XML config files and PHP classes for its “tasks” (both interfaces with external programs and internal data handling methods). This is a perfect setup for PHP deployment and makes it simple for just about any admin out there that’s even vaugely familiar with PHP to extend. Our Phing setup has four goals:

Grab the latest version of the repository after its been through QA
Run a lint check on it to ensure there’s no syntax errors
Run a series of unit tests on the code
And, if there were no errors, deploy the code out to production

Each of these steps have the “failonerror” value set to true, so the build will fail if even one check is wrong (and, come to find out, unit tests marked incomplete will cause failures).

Despite what the current Phing documentation says, there is a task for CVS integration that lets you run commands and define parameters. Our first step uses this to check out a clean version of the code to a working directory. Since the build is run as the dedicated web user on the machine, the permissions should be correct, but I used an two exec tasks to ensure the owner and group settings were right. I had to do a little hackery here to keep things a little lighter. My next target (action) in the build file is to lint the files to be sure they don’t have syntax errors. Unfortunately, by default, this would check everything in our repository. The files for our external site don’t number too many, but our internal site with several applications and documents inside is quite a bit larger. This resulted in the linting process to take quite a while and make the build process up to five minutes long.

The hack come in on the CvsTask file that’s included with Phing – I modified the code to find only the files that were updated in the latest “cvs update” call and return that list back to the build to loop through with a foreach. This saves a ton of time and only checks the files that need it – just the files that have changed. We’re just looking for syntax errors, after all.

Next up is unit testing…more to come!

Working towards a better deployment (Part 1)

Since we’re in the “slow time” for the business I work for (no one cares about their natural gas bill when it’s warm and the price is right) I’ve taken some of the flexible time I’ve found to work on a pet project of mine – the overall well-being of our development process.

Let me start off by mentioning what our current setup is:

We use CVS as our version control repository not so much by our choice now but by the choice of someone many years back. It’s worked well for what we’ve needed it for, but we’re going to be upgrading to Subversion here pretty quickly.
We have separate development, QA/testing and production environments so that each developer can work in their own space and have a QA to push it to so there’s no testing on the devs.
Code pushes are done with rsync directly from the QA environments out to production.

That’s pretty much it – a simple process that has worked pretty well for the six years I’ve been with my current employer. Unfortunately, times change and so do development standards. When I first started there were only two of us that worked on the code for both the internal and external websites. Now that team has grown to include at least six others. Needless to say, keeping things straight has become a bit more difficult.

So, back to my “free” time – I stepped back and took a look at our process. I tried to find out what could be improved in it and what was completely missing. What we have is a good base to start from, but there were a lot of other things to consider like quality of the code being pushed, checking to be sure there’s no syntax issues, creating a standardized way for the code to flow and what tools might be best for the job.

I’ve told you what we currently have, so let me tell you where my goal is – I want a system that allows the developers to work on both the main trunk and any branches (for projects and the like) of code quickly and easily and one that, once it’s been committed and tested, is out of their hands (right now, we have a system in place that lets developers push bits of code to production – a definite bad idea). Developers would have the option to change database connection information as they see fit but still have defaults for all to use. We have multiple customer databases and the location of the test databases isn’t always the same.

Once the developer is satisfied with the code and feels it bug free (enough), they can commit it and send it on its way for testing. That’s where their job ends – from here the administrator’s role kicks in. They’re the one that has the cvs-foo (or svn-foo) to pull the code out of the branch and push it into the currently checked out code in the QA environment. This code can then be reviewed and tested by whatever group needs to without interfering with any updates the developer might be working on. Once testing is complete and everything’s good to go, that code can be commited back to the trunk, pushed to a staging area and synced out to the production site. I don’t believe that QA and the base for deployment (staging) should be the same – it’s just too confusing when you could have a merging of codesets from branches all over the place. If you’re going to sync, sync from an up-to-date version of the code.

Which leads me to my next point….the build process. (the next part of this series will cover it)