May 24th, 2009 — 8:33am
This past week I was fortunate enough to attend this year’s php|tek PHP conference up in Chicago. It was four days packed with some great PHP-related content and included two major events for the PHP community – a “standrds session” and a meeting of several core developers of the language to hash out some standing issues face-to-face. The week saw a nice blend of both sessions related directly to the language an more periphery topics like MySQL tuning, project management and version control systems.
In the midst of all of this, there was something else I saw that, I have to admit, slightly caught me off guard. See, I’ve been a part of the PHP community for years now and I’ve seen some of the good and the bad along the way. Last week I saw something that gave me hope about the community and the future of the language – I saw that PHP (and its community) is growing up.
Back when I first started out in the community, PHP wasn’t taken quite as seriously as it is now. Sure, there’s those that dismiss it as “one of those languages” that’s not ready for anything more than small sites or little jobs. The truth is, PHP is running lots of major sites out there and running them very well. A shift in perception like this is all well and good but, especially with an Open Source language like this, there needs to be the people there to back it up. Lots of the other languages have corporate backing to keep them going, but PHP is in a strange place. Zend, the company most people associate with PHP, is not a direct supporter of the PHP project. They provide resources (and some of their staff) to help work on the language, but it’s not a direct involvement as a “sponsor”. Instead, PHP relies on the strength of those in the community to support it through those good and bad times and to help it weather any major storms that might come its way.
At php|tek, through all of the sessions and after-hours activities (oh yeah, we like to party), I could still see people stepping up to fill in spots that were needed. You could see it in the leadership of the core development team, in the community support both from countries here in the U.S. and overseas and, most importantly, in the words and actions of long-time members of the community willing to take the steps needed to get the community flowing. I’ve seen these people come from humbler beginnings – some joining the community after me – to become a strong foundation for the rest of the group to build on.
PHP’s future is bright with this solid crew at the helm. To all of you who have made and are making PHP and its community what it is today, I thank you.
Comment » | Community, PHP
May 22nd, 2009 — 11:29am
I’ve put my slides for my framework presentation from this year’s php|tek conference – “No Really, It’s All About You” comparing CakePHP, CodeIgniter, Solar and the Zend Framework – up on Slideshare:
Unfortunately, no one was there to record the resulting “discussion” that came from the questions after – heh.
6 comments » | Community, PHP, Presentation
May 5th, 2009 — 10:11pm
I’m back with the fourth part of my look at deployment of PHP applications to focus on the actual deployment technologies. I’ve already talked some about version control, build tools and unit testing in the previous parts of the series and this new information should round it out nicely with a simple approach to the push part of the deployment.
As it stands right now, the final step in our deployment is one of the simplest. We have some bash scripts that are aliased to rsync commands for pushing files out to each of our different sites. Since the assumption is that anything that’s in the QA/staging area is good to go, the rsync commands take everything that’s out on production and, if there’s any differences, overwrites the remote version. It’s light, simple and makes it easy to push out an entire site in one go. Things can get a bit tricky of you only want to deploy a portion of the site at a time though. There haven’t been many cases where we needed to do this, but it would have been nice to have a way to break it up a bit more.
I posted two TwtPolls in the past few days asking some of the other developers in the community (some outside of PHP too) about which method they preferred for deploying their sites out to a production server. Here’s the results of the latest poll (hopefully it’ll stick around for a while – I’ll mention the numbers too, just in case).
There were 45 votes total on ten different options…here’s how it came out:
||[ 20% (9 votes) ]
||[ 4% (2 votes) ]
||[ 9% (4 votes) ]
|SVN (via ssh or export)
||[ 49% (22 votes) ]
||[ 7% (3 votes) ]
||[ 2% (1 votes) ]
|Cut & Paste
||[ 2% (1 votes) ]
|Other Manual Process
||[ 2% (1 votes) ]
|Other Automatic Process
||[ 2% (1 votes) ]
|A mashup of several
||[ 2% (1 votes) ]
As you can see rsync, our current choice, only came in third under FTP/SFTP and the overwhelming favorite, Subversion deployment. Some of them choose to make the copy of the code on the production server a checkout of the current version (svn up when it needs refreshing) and some export from their current version and push those files out. Unfortunately, we’re still stuck with CVS for the time being (coming soon, hopefully!) so working with branching, tagging and all should be quite a bit easier.
This final step can either be a manual one (how we do it now) or as a part of another process (how I want to do it). I currently have a final step in my Phing build file that, if all goes well during the build, will call one of our bash scripts to push the code directly from there. This helps to keep things nice and tidy and makes it a simple “single process” tool that sysadmins or QA folks can use to ship out the code that’s been cleared for public consumption.
Now that I’ve gotten to the end of the descriptions on all of this stuff, let’s head back to the beginning and look at some of the actual code, configurations and technologies related to deployment. I’ve seen some good suggestions as to a step I’ve been missing – optimization. Keep an eye out for this first in the “Deployment Tech” series, a continuation of this general look at deployment of PHP applications.
9 comments » | Deployment, PHP, rsync
May 2nd, 2009 — 10:50pm
So, on the to third part of my little series (part one, part two) on making a shiny new deployment system for our sites. What’s that? Yep, I said sites – plural. I’m working all of this up on a “one site” concept, but in reality we have five or six websites that this process will eventually apply to. They’re all pretty similar, thankfully, and all use a lot of the same libraries. This makes the next step of my little process so much easier – the unit testing.
If you’re not unit testing your code, drop what you’re doing (well, finish this article at least) and go learn as much as you can about it. It can help take that feeling of dread away from your code updates and can help make things go a bit smoother in your deployment. Imagine having an automated way to check and ensure that the results of your scripts/libraries are still working exactly how the should. Unit tests can help you verify that things that should pass the tests still do and that your failures still fail in all the right ways.
One of the more popular unit testing softwares out there is PHPUnit (from Sebastian Bergmann). Its a super-easy way to set up a suite of tests that can be run as a part of your deployment process. The tests run on your scripts with methods corresponding to each of the methods in the target class. You use assertions inside the tests to check for things like pass, fail, true, false, if an array value is set, etc. The methods are called and the output is verified – if everything’s good, the test passed. If there’s a problem (something non-standard) the test fails which can cause the entire build process to fail too.
You can test more than just simple single method calls too. Say you have a class that creates a user by logging them in and setting up a session for them. If your tests require this, you can make the user session in your “setup” method and use the “teardown” method once things are finished to log the user out and destroy any of the resources you might have created.
These unit tests have found a home in my build process right after the lint check on each of the new files. Phing allows you to defne a tests directory for it to look in for any unit tests it should apply. The PHPUnitTask takes care of the dirty work with options to fail the build if the tests have an error and gather code coverage information as they’re tested. You can also set up a PHPUnitReport task to get more information out of your build. This makes reports and, with the help of some XSLT templates, transforms the results into something n bit more human readable.
All that’s really left after this (with the exception of a few other options) is the push of the up-to-date, tested code out to the production server…but that’s for the next article to cover.
Oh, and for those that are wondering, I’m going to be getting to the tech behind all of this pretty soon. Having information about the setup is one thing, but having the actual setup behind it all helps even more. I’m going to go back over all of this from start to finish to show you how it all fits together. Just hang in there with me!
Comment » | cvs, Deployment, PHP
May 2nd, 2009 — 7:48am
In the previous part of this series, I looked at some of the current methods we use for code development and deployment as well as some of the goals I put out there for myself in the quest to find a good deployment system that would work for us and our code. I’m back again to look at the next part of the process – the build.
Once you have the process defined for the developers to follow, you need to lay the foundations for the administrator to handle the rest of the process. Once we know that the code is good and all is right with the world, the deployment phase can get started. Ideally, the admin shouldn’t have to do much at all – well, besides making sure the right code gets merged into the trunk, that can be a little tricky. A correctly defined process should handle the rest, though – running tests on the code to ensure correct functionality, locating any syntax errors it might have, etc – all automagically.
The tool I’ve chosen to act as a base for all of this is Phing, a build tool based on Apache’s Ant that uses XML config files and PHP classes for its “tasks” (both interfaces with external programs and internal data handling methods). This is a perfect setup for PHP deployment and makes it simple for just about any admin out there that’s even vaugely familiar with PHP to extend. Our Phing setup has four goals:
- Grab the latest version of the repository after its been through QA
- Run a lint check on it to ensure there’s no syntax errors
- Run a series of unit tests on the code
- And, if there were no errors, deploy the code out to production
Each of these steps have the “failonerror” value set to true, so the build will fail if even one check is wrong (and, come to find out, unit tests marked incomplete will cause failures).
Despite what the current Phing documentation says, there is a task for CVS integration that lets you run commands and define parameters. Our first step uses this to check out a clean version of the code to a working directory. Since the build is run as the dedicated web user on the machine, the permissions should be correct, but I used an two exec tasks to ensure the owner and group settings were right. I had to do a little hackery here to keep things a little lighter. My next target (action) in the build file is to lint the files to be sure they don’t have syntax errors. Unfortunately, by default, this would check everything in our repository. The files for our external site don’t number too many, but our internal site with several applications and documents inside is quite a bit larger. This resulted in the linting process to take quite a while and make the build process up to five minutes long.
The hack come in on the CvsTask file that’s included with Phing – I modified the code to find only the files that were updated in the latest “cvs update” call and return that list back to the build to loop through with a foreach. This saves a ton of time and only checks the files that need it – just the files that have changed. We’re just looking for syntax errors, after all.
Next up is unit testing…more to come!
4 comments » | cvs, Deployment, PHP
May 1st, 2009 — 11:39pm
Since we’re in the “slow time” for the business I work for (no one cares about their natural gas bill when it’s warm and the price is right) I’ve taken some of the flexible time I’ve found to work on a pet project of mine – the overall well-being of our development process.
Let me start off by mentioning what our current setup is:
- We use CVS as our version control repository not so much by our choice now but by the choice of someone many years back. It’s worked well for what we’ve needed it for, but we’re going to be upgrading to Subversion here pretty quickly.
- We have separate development, QA/testing and production environments so that each developer can work in their own space and have a QA to push it to so there’s no testing on the devs.
- Code pushes are done with rsync directly from the QA environments out to production.
That’s pretty much it – a simple process that has worked pretty well for the six years I’ve been with my current employer. Unfortunately, times change and so do development standards. When I first started there were only two of us that worked on the code for both the internal and external websites. Now that team has grown to include at least six others. Needless to say, keeping things straight has become a bit more difficult.
So, back to my “free” time – I stepped back and took a look at our process. I tried to find out what could be improved in it and what was completely missing. What we have is a good base to start from, but there were a lot of other things to consider like quality of the code being pushed, checking to be sure there’s no syntax issues, creating a standardized way for the code to flow and what tools might be best for the job.
I’ve told you what we currently have, so let me tell you where my goal is – I want a system that allows the developers to work on both the main trunk and any branches (for projects and the like) of code quickly and easily and one that, once it’s been committed and tested, is out of their hands (right now, we have a system in place that lets developers push bits of code to production – a definite bad idea). Developers would have the option to change database connection information as they see fit but still have defaults for all to use. We have multiple customer databases and the location of the test databases isn’t always the same.
Once the developer is satisfied with the code and feels it bug free (enough), they can commit it and send it on its way for testing. That’s where their job ends – from here the administrator’s role kicks in. They’re the one that has the cvs-foo (or svn-foo) to pull the code out of the branch and push it into the currently checked out code in the QA environment. This code can then be reviewed and tested by whatever group needs to without interfering with any updates the developer might be working on. Once testing is complete and everything’s good to go, that code can be commited back to the trunk, pushed to a staging area and synced out to the production site. I don’t believe that QA and the base for deployment (staging) should be the same – it’s just too confusing when you could have a merging of codesets from branches all over the place. If you’re going to sync, sync from an up-to-date version of the code.
Which leads me to my next point….the build process. (the next part of this series will cover it)
11 comments » | cvs, Deployment, PHP