Innovation’s Not The “Ah-Hah!”

After reading through his “Confessions of a Public Speaker” (as a beginning speaker, I learned some good things from this one – I’d suggest it if you do any kind of speaking) I was anxious to check out some of Scott Berkun’s other books. The topics of some of the others didn’t really appeal to me, but the one that’s caught my attention recently is his “Myths of Innovation” book. I’m maybe a third of the way through it right now, and there’s one thing that keeps resonating in my mind as I go through it. In a previous chapter, he makes the point that innovation, despite what the history books and popular culture would have us assume – it’s less of an “Ah-hah!” and more of a “Finally!”.

See, most of the common stories of innovators out there leave out something that’s very important – the reference frame of their lives. They don’t provide a larger picture of who someone is (like Einstein or Newton) and how all of their work, everything they’ve done in their career led up to the discoveries that they’re known for.

I think this is important to remember as software developers, too. All of us start projects and never finish them, it’s just a fact of life in the world of a coder. We find something that we either think is the “Next Big Idea” or something that we’ll find amazingly useful and latch onto it, giving it our all for a week, maybe a month. Nine times out of ten, though, that project falls by the wayside. Now, don’t get me wrong, there’s some folks out there that do a great job with anything they touch, but for the average developer, it’s all about hacking away at the latest “shiny”.

Sometimes it’s about the technology (“everyone’s learning Backbone.js, why shouldn’t I?”) and other times there’s a bit of pride that kicks in (“I could do this so much better if…”) but there’s always one thing to remember. It doesn’t matter if the project you’re working on goes anywhere. Remember this. Just like some of the great innovators of the past, it takes a lot of dedication and work to get to be the “Ah-hah Guy” that wows the world with something new and amazing. Don’t forget that the code of the Next Great App isn’t just going to fly from your fingertips.

Work hard at your craft and it will pay off. Maybe not in fame and glory, maybe in making real, useful contributions to the culture and technology around you. Don’t stop trying to innovate, don’t focus on the failures and, above all, keep learning and keep doing.

Composer Dependency Woes

I spent the better part of this afternoon trying to figure out why a Composer installation wasn’t working and finally figured out the problem…it wasn’t mine.

First, a little context – I’m currently working on a testing presentation for some folks at work and I wanted to show them how to work with the Behat testing tool to create some handy functional/integration tests for our framework-based apps. I threw together a little framework (yes yes, I know) and got the PHPUnit tests set up and running in no time. When it came to the Behat tests, though, no matter what I did, I was still having a problem:

[php]
PHP Fatal error: Class ‘GoutteClient’ not found in /www/htdocs/testing-examples/app/vendor/behat/mink/src/Behat/Mink/Driver/Goutte/Client.php on line 13
[/php]

No matter how I tried to configure the composer install, it always gave me this message. I tried everything I could think of and, finally, at the suggestion of Rafael Dohms, checked out the github repository for the Goutte client (a href=”http://github.com/fabpot/goutte”>here). As it turns out, in the past day or so, there’s been a large change where Fabien implemented composer support on the repo.

Apparently this was what broke things – thankfully not something obvious I was missing.

So, how did I solve it so I could see the lovely green of passing tests again? Well, if you’re familiar with composer, you know there’s a composer.lock file that’s created after you install. When you run the “composer install” and it fetches from “fabpot/goutte”:”*”, you get this latest version that has the issues. A quick modification of the composer.lock file takes care of that though:

[php]
{
"package": "fabpot/goutte",
"version": "master-dev",
"source-reference": "5ecceb7c28a428fb93f283982cc4f5edfd96630b"
},
[/php]

See that “source-reference” setting? Well, that can either point to the branch or version you want to pull from or it can point to a specific commit. In my case, I just pulled the hash for the commit before all of the changes and dropped it in there. Then it’s just a matter of running a “composer install” to get the code from this commit instead. Don’t run an update though – that will wipe out your manual changes to the lock file and you’ll be back to square one.

Hope this helps someone out there who might be dealing with a similar issue regarding brokenness on an external lib!

Quick and Dirty REST Security (or Hashes For All!)

So in working up a new RESTful service I’ve been tinkering with, I wanted to provide some kind of “authentication” system for it. I started to look into OAuth, but got a bit overwhelmed by everything that was involved with it. Looking for something a bit more lightweight (and simpler to implement a bit more quickly) I came across this older article with a suggestion of a private key/hash combination. I figured that could do the job nicely for a first shot, so I set to implementing it.

On the Server Side

I’m using the FuelPHP framework for this one, but that’s really only giving me a structure to work in and pull the request information from. This would work in most major frameworks (and even outside of one if you you’re a “do it my way” kind of developer). First off, let’s start with the controller side:

[php]
<?php
class Controller_User extends Controller_Rest
{
protected function validateHash()
{
$request = file_get_contents(‘php://input’);
$requestHeaders = apache_request_headers();

if (!isset($requestHeaders[‘X-Auth’]) || !isset($requestHeaders[‘X-Auth-Hash’])) {
$this->response(‘fail!’,401);
} else {
// we have the headers – let’s match!
$user = Model_User::find()->where(‘public_key’,$requestHeaders[‘X-Auth’])->get_one();

if ($user !== null) {
$hash = hash_hmac(‘sha256’,$request,$user->private_key);
return ($hash == $requestHeaders[‘X-Auth-Hash’]) ? true : false;
} else {
return false;
}
}
}

public function post_index()
{
// return the user details here….
}

public function router($resource, array $arguments)
{
if ($this->validateHash() == false) {
$resource = ‘error’;
$arguments = array(‘Not Authorized’,401);
}

parent::router($resource,$arguments);
}
}
?>
[/php]

There’s a lot going on here, so let me walk you through each of the steps:

  1. First off, we’re making a RESTful service, so we’re going to extend the Controller_Rest that Fuel comes with. It has some handy special routing. Our POST request in the example below would try to hit the “post_index” method and have its hashes checked in the process.
  2. Next up is the “validateHash” method – this is where the hard work happens:
    • The request and headers are read into variables for easier use ($request and $requestHeaders).
    • It then checks to be sure that both of our required headers are set (X-Auth and X-Auth-Hash). There’s nothing magical about these header names, so they can be switched out depending on need and naming preference.
    • If they’re there, the next step is to find the user based on the public key that was sent. This value is okay to openly share because, without the private key to correctly hash the data, your requests will fail.
    • The hash_hmac function is then used (with the “sha256” hash type) to regenerate the hash off of the contents of the request and the private key on the found user.
  3. If all goes well, the request continues on and the “post_index” method is used. If it fails, however, the check in the “route” method of the controller makes a switch. It changes the currently requested resource to “/error/index” instead of what the user wants. This seamlessly shows the user a “Not Authorized” error message (401) if the hash checking fails.

A Client Example

Now, to help make it a bit clearer, here’s an example little script showing a curl request using the hashes:

[php]
<?php

$privateKey = ‘caa68fb2160b428bd1e7d78fcf0ce2d5′;
$publicKey = ’01fa456c4e2a2bc13e5c0c4977297fbb’;

$data = ‘{"username":"happyFunBall"}’;
$hash = hash_hmac(‘sha256’,$data,$privateKey);

$headers = array(
‘X-Auth: ‘.$publicKey,
‘X-Auth-Hash: ‘.$hash
);

$ch = curl_init(‘http://mysite.localhost:8080/user&#8217;);

curl_setopt($ch,CURLOPT_HEADER,true);
curl_setopt($ch,CURLOPT_HTTPHEADER,$headers);
curl_setopt($ch,CURLOPT_POSTFIELDS,$data);
curl_setopt($ch,CURLOPT_RETURNTRANSFER,true);

$result = curl_exec($ch);
curl_close($ch);

print_r($result);
echo "nn";
?>
[/php]

You can see that both the public and private keys are specified (but on the PHP side, not visible to the user) and are sent as the “X-Auth*” headers as a part of the request. In this case, we’re POSTing to the “/user/enygma” resource and creating a user. To create the value to put into $hash, we use the same “sha256” hashing method and salt it with the private key. This value – the result of the data+private key hashing – is then passed to the service who uses the controller code from above to validate that it’s correct.

It’s not a true “login” process, but it can help to validate that a piece of information known only to the user (the private key) and never shared directly, is known and matches the requests that the user sends.

You can find the code for this in this gist if you’d like something a bit more readable: https://gist.github.com/2697434

Book Review: “Code Simplicity”

Last night I finished my latest read from O’Reiily, Code Simplicity – The Science of Software Development. I spotted the book the other day when O’Reilly was running a special on a few books and the ebook was cheap so I figured it couldn’t hurt to give it a try. After all, the “science” part in the title made it sound like there might be some hidden truths that could be applied anywhere in development. Unfortunately, most of the book just ended up being more of a rambling journey though things that most software developers that have any years of experience (even the bad ones) would already know.

The author spent a good bit of the book dedicated to definitions and explanations about various practices and ideas in development, as if he thought that maybe the audience reading the book wasn’t savvy on the topic. The first few chapters also included several sections about the book itself – why it was relevant and mentions of a “science” that never seemed to fully resolve. Granted, trying to make a “science” (more a set of laws than just best practices) out of something so varied as software development is a pretty difficult task, but I felt like the author tried a little too hard to make his case for the book and less time actually defining something that could have been interesting.

All this being said, if you don’t worry too much about him trying to propose a “science” to it all, there were some good best practices reminders in here for developers of any language:

  • Don’t rewrite, rework – a reminder that, despite it seeming easier to chuck the whole system and start over with the knowledge you now have, you’d do better in the long run to change things from the inside out, a piece at a time (hint: unit tests make a world of difference here)
  • Know the problem before writing the solution – listen and understand the problem before you start with even one piece of code. If you don’t fully understand the problem, you’ll end up with half-assed software that only does part of what was needed.
  • Think specific, not general – if you immediately jump to the “well, if I use a plugin architecture for this part…” chances are you’ve already added too much complexity. Think small first – make it work, then make it better (I’m a big fan of iterative development)
  • Use more experienced developers as a sounding board – chances are, if someone’s been in the development biz longer than you, they’ve come across your situation before. Sometimes you have to seek out that person on a specific topic, but don’t just forge ahead blindly. At the very least, try to find blog posts or articles that you can use as a guide.
  • Don’t forget that time is important too – most developers (me included) easily forget that time is a factor in their development. No, I’m not talking about the actual time to write the code or the looming deadline to finish it by. I’m talking more about the time you’ll need to do research, try things out or even consult with fellow developers. Time put into something to gain knowledge is an investment too…don’t forget to remember the value of it.

There were other points made throughout the book, some more relevant than others, but I wish the author had spent less time focusing on definitions and more on expanding the sections with some more practical advice. This (relatively short) book probably could have been summed up in a small series of blog posts and been just fine.

Book: Code Simplicity – The Science of Software Development
Publisher: O’Reilly
Author: Max Kanat-Alexander
Pages: 92

Dynamic Toolbar Menus with ExtJS + PHP

In Ext JS 4 there’s some handy things that come bundled with it (there’s lots of stuff actually – it’s a pretty large library). Recently, though, I needed to pull in navigation information from a remote source (JSON output from a PHP script). Thankfully, Ext still made this pretty easy with its Toolbar and Menu components with their listeners. Here’s my example code:

[javascript]
Ext.create(‘Ext.toolbar.Toolbar’, {
floating: false,
id: ‘menuToolbar’,
cls: ‘appMenu’,
height: 30,
items: [], // dynamically built below
listeners: {
beforerender: function() {
var navStore = Ext.create(‘Ext.data.Store’, {
fields: [‘text’],
proxy: {
type: ‘ajax’,
url: ‘/path/to/navigation-output’,
reader: {
type: ‘json’,
root: ‘navigation’
}
},
autoLoad: true,
listeners: {
load: function(store,records,success,operation,opts) {

var toolbar = Ext.getCmp(‘menuToolbar’);

// First the top level items
store.each(function(record) {

var menu = Ext.create(‘Ext.menu.Menu’);
Ext.each(record.raw.menu, function(item){
menu.add({
text: item.text
})
})

toolbar.add({
xtype: ‘button’,
text: record.data.text,
menu: menu
});
});
}
}
});
}
}
});
[/javascript]

Then the PHP to make the output is pretty easy (slightly simplified here):

[php]
<?php
echo json_encode(
‘navigation’ => array(
‘text’ => ‘Option #1’,
‘menu’ => array(
array(‘text’ => ‘Foo’),
array(‘text’ => ‘Bar’),
array(‘text’ => ‘Baz’)
)
)
);
?>
[/php]

Now – a little explaination of what we’re doing here:

  1. In Ext, we create a generic Toolbar object – this is what’s going to contain the “buttons” that act as the top level menu.
  2. There’s a few config options (like an ID, height and a custom class to apply) but the key is in the “listeners” section. This is where Ext looks to for events on the objects. In our case, we’re telling it to, on the “beforerender” event, call this given inline method. This method then makes our store.
  3. For those not familiar with the ideas of “stores”, think of them as Javascript-based database tables (kinda). They pull in data from some source or can be manually populated in memory to prevent you from having to go back and fetch the data every time you need it. in our case, we just make a basic one (Ext.data.Store) that is set up with a proxy to pull from the JSON source (our /path/to/navigation-output). With the “autoLoad” property set to “true” it automatically pulls in the JSON as soon as it’s created.
  4. You’ll see that we’ve, once again, tapped into the “listeners”, this time for the store. Our “load” listener fires when the store data is completely loaded. In here is where we’re going to add our elements into the Toolbar.
  5. As one of the options, we get the current store back and we use it to loop through and get each of the records. Each of these top level records is going to be one of our Toolbar buttons, each with its own menu. You can see as we loop through them, we create an “Ext.menu.Menu” object adding the “text” value to it. This menu is then appended to the button via the “menu” property on the button. The “add” is called again on the button config and it’s dropped into the Toolbar.

The fun thing about this is that it makes a reusable component that you can use across products/sites without having to hard-code the navigation options into the actual code. There’s probably a simpler way to do this, but this one seemed to make the most sense to me.

Behat + FuelPHP = RESTful Testing Happiness

If you’ve been following my recent posts, you know I’ve been working more lately with Behat for resting some REST services. In this post I showed you how to get things set up for some testing. In this post, I’ll show you how to use a custom class that I’ve put together to make a reusable system for testing REST.

For those that want to cut to the chase, I’ve posted some example code to github showing the code for the two different sides of the equation – the Behat testing and the PHP framework side (I went with FuelPHP because I was already familiar with it and it makes RESTful interfaces dead simple). The key is in the FeatureContextRest.php file that uses the Guzzle client to make the HTTP requests over to the framework. It provides some handy contexts that make it easy to create Scenarios.

So, still with me? Good – I’ll show you how to get my sample scripts installed so you can see my example in action. In the repository you’ll find two different directories – “fuel” and “behat”. In each you’ll find some files:

behat

  • features/index.features:
    An example set of Scenarios that use the REST testing contexts to create, find and deleting a “User” from the system
  • features/bootstrap/FeaturesContextRest.php:
    The key to the puzzle, the implementation of some context methods for use in Behat

fuelphp

  • fuelphp/classes:
    These are the example models and controllers you can drop into your Fuel install to have them “magically” work (with any recent version of the framework).

This is assuming you already have Behat installed and working and have set up FuelPHP on a host somwhere. For our example, we’ll just use http://behat-test.localhost:8080&#8221; Here’s what to do:

  1. Start by copying over the FeaturesContextRest.php file to the “features/bootstrap” directory of your Behat installation alongside the default FeaturesContext.php.
  2. Copy over the behat.yml configuration file into your testing root (the same level as the “features” directory) and update the base_url value for your hostname.
  3. Take the “classes” directory and copy over its contents to the location of your FuelPHP installation – if you’re familiar with the structure of the framework, this is simple…it just goes in “app/classes”.
  4. You’ll need a database to get this working, so you’ll need to get the FuelPHP ORM stuff configured and working in your install. Be sure to update your config.php to automatically load the “orm” module.
  5. Make a database for the project and use the init.sql to create the “users” table.
  6. To test and be sure our models/controllers are working right, hit your host in a browser, calling the “User” action like: “http://behat-test.localhost:8080/user&#8221;. This should call what’s in Controller_User::get_index(). It’ll probably just return an empty result though, since there’s nothing for it to pull.
  7. To use the FeaturesContextRest.php, be sure that you include that file into your default FeaturesContext.php file and that your class “extends FeaturesContextRest”.

So, if all has gone well, you have all the pieces in place to get started. Let’s start with a sample Scenario to show you how it all works:

Scenario: Creating a new User
	Given that I want to make a new "User"
	And that its "name" is "Chris"
	When I request "/user/index.json"
	Then the response is JSON
	And the response has a "userId" property
	And the type of the "userId" property is numeric
	Then the response status code should be 200

If you’ve used Mink in the past, so of this will seem familiar. This doesn’t use Mink, however, and I’ve opted to use Guzzle as the client for the backend to replace some of the browser interactions. Really, we don’t need all of the fancy interface interaction Mink gives, so this streamlined interface is more useful.

In this example, we’re creating a user with a property “name” of “Chris” and sending that off to the “/user/index.json” endpoint. A few tests are included like: checking the response code, seeing if the return value has a “userId” property, etc. If all went well and the user was created (this is a POST, so it calls “post_index” in the controller), this test should pass with flying colors….green, hopefully. :)

The key is in the phrases “to make a new” (POST), “to find a” (GET) and “to delete a” (DELETE) to tell the tests which verb to use.

Hopefully this library will be useful to some folks out there – I’m going to be improving it as I work on more tests for some current projects to ensure better quality of the software.

From the Start: Setting up Behat testing

In a previous post I showed you how to use HTTP Auth with the Goutte driver in Behat testing. I want to take a step back and show you how I even got to that point with a simple guide to installing the tool and creating your first tests. In a future post, I’ll show the (super simple) integration you can do with Jenkins to execute the tests and plug in the results.

First off, a definition for those not sure what Behat is:

Behat was inspired by Ruby’s Cucumber project and especially its syntax part (Gherkin). It tries to be like Cucumber with input (Feature files) and output (console formatters), but in core, it has been built from the ground on pure php with Symfony2 components.

And, for those not sure what Cucumber is:

Cucumber lets software development teams describe how software should behave in plain text. The text is written in a business-readable domain-specific language and serves as documentation, automated tests and development-aid – all rolled into one format.

These are both tools that implement something called behavior-driven development, the practice of approaching testing from a more natural angle, allowing tests to be written in “plain English” (or the language of your choosing with Behat) and executed at a domain level rather than a code level (as unit tests would). Behat makes this super simple and it really only takes a few minutes to get it up and running.

For my examples, I’m going to be working on a Linux-based system (CentOS actually) but the same kinds of ideas apply to other platforms too, just modified slightly for things like pathing. So, let’s get started…first off, the installation. Personally, I recommend the Composer method for installation – it just makes it easier to get everything you need without having to think hardly at all.

Here’s a set of commands (linux command line) to get everything set up for a basic project structure:

mkdir behat-testing-ftw; cd behat-testing-ftw;
curl -s http://getcomposer.org/installer | php
vi composer.json

Inside the composer.json file, enter the stuff from the Behat instructions here (the JSON format), then we can run composer on it:

php composer.phar install

At this point you should see a bunch of things being installed in the “Installing dependencies” section (at the time of this post, that includes things like symfony/finder, symfony/dependency-injection and behat/gherkin). Once this is finished, you should have a directory structure similar to:

Chriss-MacBook-Pro:behat-testing-ftw chris$ ls
bin		composer.json	composer.lock	composer.phar	vendor

Behat should be magically installed in the bin/ directory – to test it out, try this:

Chriss-MacBook-Pro:behat-testing-ftw chris$ bin/behat

If all goes well, you should see a big red error talking about “features” not existing. This is good – we’ll fix this now by making the testing base for Behat. Thankfully, it makes this easy too:

Chriss-MacBook-Pro:behat-testing-ftw chris$ bin/behat --init
+d features - place your *.feature files here
+d features/bootstrap - place bootstrap scripts and static files here
+f features/bootstrap/FeatureContext.php - place your feature related code here

Your directory should now look like this:

Chriss-MacBook-Pro:behat-testing-ftw chris$ ls
bin		composer.json	composer.lock	composer.phar	features	vendor

The new features directory is where your Behat tests will live. If you look in there now, there’s not much – just a bootstrap directory with a FeatureContext.php file in it. This file is where you’ll define any custom rules you might need to use outside of the pre-defined ones Behat (and later Mink) come with. If you re-run Behat again, you should see:

Chriss-MacBook-Pro:behat-testing-ftw chris$ bin/behat
No scenarios
No steps
0m0.001s

So, let’s give Behat something to do now – Behat tests are organized in “feature” files. Let’s make a simple one for illustration and call it “sample.features” and put some content in it:

cd features
vi sample.feature

And put the following inside it:

Feature: Testing Behat install
        This is a testing feature to see if everything
        is working correctly

        Scenario: My first test
                When I run "ls"
                Then I should see the file "composer.json"

By default, Behat doesn’t come with any contexts pre-defined (you can see this by executing /bin/behat -dl) so we’ll need to make some for our new feature. Thankfully, Behat makes this simple too (sensing a theme here?) – if you run behat again, you should get something similar to:

eature: Testing Behat install
  This is a testing feature to see if everything
  is working correctly

  Scenario: My first test                      # features/sample.feature:5
    When I run "ls"
    Then I should see the file "composer.json"

1 scenario (1 undefined)
2 steps (2 undefined)
0m0.051s

You can implement step definitions for undefined steps with these snippets:

    /**
     * @When /^I run "([^"]*)"$/
     */
    public function iRun($argument1)
    {
        throw new PendingException();
    }

    /**
     * @Then /^I should see the file "([^"]*)"$/
     */
    public function iShouldSeeTheFile($argument1)
    {
        throw new PendingException();
    }

Se those last few lines? That tells you exactly what you’ll need to drop into the FeaturesContext.php file to make things work. The key to the matching is in the DocBlock comments – there’s regular expression matches in there that Behat tries to use when it’s executing the tests. So, let’s update our FeaturesContext with some new code:

vi features/bootstrap/FeatureContext.php

And insert into the class the following:

[php]
/**
* @When /^I run "([^"]*)"$/
*/
public function iRun($command)
{
exec($command,$result);
$this->output = $result;
}

/**
* @Then /^I should see the file "([^"]*)"$/
*/
public function iShouldSeeTheFile($fileName)
{
if (!in_array($fileName,$this->output)) {
throw new Exception(‘File named ‘.$fileName.’ not found!’);
}
}
[/php]

This code tells Behat to execute the $command (via exec), return the results to the “output” property and then check to see if the $fileName is in the list. As long as you were in the main “behat-testing-ftw” directory when you executed this, all the tests should come up green:

Feature: Testing Behat install
  This is a testing feature to see if everything
  is working correctly

  Scenario: My first test                      # features/sample.feature:5
    When I run "ls"                            # FeatureContext::iRun()
    Then I should see the file "composer.json" # FeatureContext::iShouldSeeTheFile()

1 scenario (1 passed)
2 steps (2 passed)
0m0.016s

To see how it looks when tests fail, try changing the filename in the sample.feature to something like “composer.json-foo” and you’ll see that step turn red.

So, that’s pretty much it – that’s, in a nutshell how to get Behat testing up and running (via the Composer option). There’s a few minor things that I came across when I was trying to execute the Composer and Behat tools, but those usually stemmed from missing PHP functionality/packages not being installed on the server. Those were solved with a quick “yum install” call for the needed functionality.

I hope to continue this series and talk next about getting Mink up and working to add some really useful features to Behat. I’ve been using it for testing a REST service, so that’s one of the end goals.