Friday, 11 June 2010

Conversion of UTF-8 form fields in Multi Part Form

Got to say a big thanks to Abhishek at Running Commentary for the solution to this problem.

While using the Apache Commons File Upload Library on a Spanish Language form, I was having problems with the UTF-8 characters that were being submitted in people's names etc. Everything else was working properly, the rest of the Spanish characters on the page were displaying properly but the input was being translated into garbage when submitted.

The solution ultimately was to change




While extracting the items from the List element that the FileUploadHandler returned after parsing the request object.

Bear in mind, it doesn't matter what you do before calling this, if you don't explicitely tell the FUH what encoding you want to you use, you'll always get standard ASCII back (and subsequently garbage if your form fields contain anything other than standard ASCII...

Hope this helps.

Friday, 23 April 2010

How to ask questions the smart way

Found this absolutely fantastic article on how to ask questions on forums, newsgroups etc. in an intelligent and considered way that is more likely to get you a decent response.

Very good advice...

How to change the default JRE in Mac OS X (Leopard)

The problem was, my Eclipse installation was setup to use JRE 1.6 but when I checked using
mvn --version
it reported that it was using JRE 1.5. The solution's pretty simple in fact. The versions of Java JRE that are installed on your Mac are located in
If you list the contents of the folder in terminal using
ls -al
then you'll see a symbolic link called "CurrentJDK" pointing to a symbolic link called "1.5" pointing to the latest version of 1.5 installed. Delete this link and recreate it pointing it to the symbolic link called "1.6" and you're done. The following code will achieve this.
 cd /System/Library/Frameworks/JavaVM.framwork/Versions/
sudo rm -r CurrentJDK
sudo ln -s 1.6 CurrentJDK
UPDATE Due to obvious confusion caused by my $ sign, I have reworked the code snippets to make them clearer

Learning Java now :)

So, the company where I am working at the moment has made the move into a Java based web environment. Lots of reasons and I'm not going into them here as this is a personal blog (despite the technical nature) and I'm not going to discuss work decisions or anything that may be confidential in nature of course.

But - I have a blog, I'm learning a lot of new stuff and it makes sense to switch the focus of the blog to whatever I'm using/doing/learning rather than a single focus on Symfony as I originally planned.

So watch out for a lot of new posts as I go... most of what I learn will posted here for my future reference

Saturday, 6 February 2010

Quick and easy server control

I use a macports installation of Apache2, PHP5 and MySQl5 for development on my macbooks. This is a great system and really simplifies the installation and setup the utilities.

However, the start/stop commands for each are not added to the PATH by default and it's tough remembering the long paths to the control programs.

But there's a solution:

  1. Open the Terminal
  2. Make sure that you're in your home folder.
  3. Edit (or create) .bash_profile

    $ nano .bash_profile
  4. Add aliases for your commands

    alias apache2ctl="sudo /opt/local/apache/bin/apachectl"
    alias mysqlstart="sudo /opt/local/bin/mysqld_safe5"
    alias mysqlstop="sudo /opt/local/bin/mysqladmin5 -u root -p shutdown"
  5. restart your bash session with

    source ~/.bash_profile
    and you're good to go...
Then you simply issue the following commands from the command line.
apache2ctl start # Start Apache
apache2ctl stop # Stop Apache
apache2ctl graceful # Restart Apache
mysqlstart # Start Mysql
mysqlstop # Stop Mysql

Wednesday, 3 February 2010

Organising Partials in Symfony

A little tip for organising partial templates in Symfony.

If you want to store partials in sub folders, simply name the first level of folders prefixed with an underscore and remove the underscore from the partial name (or make sure include the underscore when you're calling it.

That way, let's say you send out emails for various reasons on your site, so you have lots of templates (partials) in a mailer module to make your emails nice and easy to manage. If you wanted to organise your registration email body and subject partials in sub folders, you could do the following:

Your subject partial would be located at:


and your body partials might be located at:


You would then call these partials from your sendMail action as follows:

    // subject 

    // email body
    $message->setBody($this->getPartial('mailer/registration/body_html'), 'text/html')
        ->addPart($this->getPartial('mailer/registration/body_text'), 'text/plain');

Thanks to my colleague Eric for helping work this one out. A good idea that we'll use a lot now to help organise our templates and partials in all sorts of wonderful ways

Monday, 1 February 2010

Adding embedded forms dynamically using ajax

Over this last weekend, I've been building an 'invite your friends' form. Fairly standard fair, a registered user is presented with the opportunity to send invitations to friends for one reason or another by entering their name & email address. This would have been fairly simple had it not been for the requirement to add new rows to the form using ajax. I know all about embedded forms and use them a lot - they really speed work up - but adding fields or forms to a form using ajax caused me a lot of head-scratching. But, I figured it out using the
material linked to below and some heavy doses of trial and error :)

This post is based on the extremely enlightening work of Nacho Martin - kudos and thanks to him for pointing me towards the light. The reason for me reinventing the wheel is that his post is more about the Symfony 1.2 way of doing things and this caused problems in 1.4 (Such as using a form as an array within the form or action for example...) So onwards and upwards...

The schema is very simple for this particular problem as we're using sfDoctrineGuardPlugin to handle the needs of the user model so we'll concentrate  on the schema of the invitation table for this example.

Our invitation table needs to be able to simply save a name and an email address. We can extend it later if we need to provide recognition tokens, tracking, follow-up emails etc etc. But for now, we just want simple :) so...

// /data/doctrine/schema.yml

  user_id:   { type: integer(4), notnull: true } 
  # note the integer(4) to make the definition equal the sf_guard_table
  email:     { type: string(255), notnull: true, unique: true }
  name:      { type: string(255), notnull: true }
      class:   sfGuardUser
      local:   user_id
      foreign: id
      type: one
      foreignType: many

Nothing complicated there - we've got an id (auto-set by Symfony), a name and email (so we know who to address our email to) and a relationship to the sfGuardUser object defined so that the Inivitation has one User and a User can have lots of Invitations.

Ok, so this is where I started forking from Nacho's contribution. In my situation, I didn't want to create or even update my entire User object even though I did want to associate it with the Invitations created. So I created a new form called MasterInvitationForm

// lib/form/MasterInvitationForm.class.php

class MasterInvitationForm extends sfGuardUserForm
    public function configure()
        $subFormCount = sfConfig::get('app_invitation_number',5);
        for($i = 0; $i < $subFormCount; $++)
        $this->mergePostValidator( new InvitationValidatorSchema());

There's a few things that we can draw from the previous code block. First off, our MasterInvitationForm is actually extending our standard sfGuardUser form. This is so we utilise the magic of Symfony later on when it comes to saving our relationships... read on.

Secondly, I've built this with reusability in mind and have set up our number of invitations that we'll display by calling an option and setting a default. If the option doesn't exist, we'll get the default - but this allows us to use this script again without changing the core code. If that's one thing I've learnt while building apps - it's reuse reuse reuse!

Penultimately, you'll see a call to addInvitationForm() within the for loop. This will be very handy indeed - you'll see why in a moment.

And last but definitely not least, we've merged a new custom validator into Post Validator schema. This will enable the custom schema to run over all the values within the POST submissions. Again - this will be very useful, but that will come even later.

So let's have a look at our addInvitationForm() method next.

// lib/form/MasterInvitationForm.class.php

class MasterInvitationForm extends sfGuardUserForm
    public function addInvitationForm($key)
            $invitationsForm = $this->getEmbeddedForm('invitations');
            $invitationsForm = new sfForm();
        $invitation = new Invitation();
        $form = new InvitationForm($invitation);
        $invitationsForm->embedForm($key, $form);
        $this->embedForm('invitations', $invitationsForm);

The addInvitaionForm accepts one parameter which is used as the name of the form within the widget, validator and error schemas - once we've set that all up.. but let's see what's going on.

We basically check for the existence of the form named invitations within the embedded forms collection and if doesn't exist, we'll create a new form and set up it's naming schema so that we can work on all the values as one later on.

We next create a new Invitation object and assign it the form's object (set as the current user in the action - we'll see that later). Then we create a new InvitationForm (the standard doctrine one) using the Invitation obect we just created.

Embed this InvitationForm object into the form named invitations using the  and lastly, re-embed (or embed for the first time if we've just created it) the invitations form into the MasterInvitationForm object.

We also add a custom configuration to our InvitationForm object. We only want to use the name & email fields within the form and we also want to have our labels within the input not externally (NB: The way I'm about to do this is NOT the best way, but it was quick and dirty for this particular project...)

// lib/form/doctrine/InvitationForm.php

class InvitationForm extends BaseInvitationForm
    public function configure()

As I said, not the best way to prepare the setup - but it worked for this situation.

OK - so now we've talked about the setup and preparing for no labels on screen - let's have a look at the view elements quickly.

I created a new module called invite and added an actions class with a new action. This would be the first view our user would see.

// apps/frontend/modules/invite/actions/actions.class.php

class inviteActions extends sfActions
    public function executeNew(sfWebRequest $request)
        if(!$this->getUser()->isAuthenticated()) $this->redirect('@homepage');
        $this->form = new MasterInvitationForm($this->getUser()->getGuardUser());
    protected function processForm($request);
        $params = $request->getParameter('user);
            $invitation = $this->form->save();
            $this->redirect('@thankyou'); // you can define this yourself - or change it as you need to of course...

The view for this action is as follows:
// apps/frontend/modules/templates/newSuccess.php

<?php use_stylesheets_for_form($form) ?>
<?php use_javascripts_for_form($form) ?>
<?php $errors = $form->getErrorSchema()->getErrors(); ?>

<form action="<?php echo url_for('invite/new'); ?>" method="post">
    <div id="invitation_rows">
        <?php foreach($form->getEmbeddedForm('invitations') as $key=>$subForm): ?>
            <?php if($subForm->isHidden()) continue; ?>
            <?php include_partial('invitation_row',array(
        <?php endforeach; ?>
    <div class="form_submit">
        <input type="submit" name="submit" value="submit" />
        <a id="add_invitation">Add another row</a>

<script type="text/javascript">
    function addInvitation(num)
        var r = $.ajax({
            type: 'GET',
            url: '<?php echo (url_for('invite/ajaxAddInvitationRow'); ?>'+'?key='=$('#invitation_rows div[id^=invitation_]').length,
            async: false
        return r;

There's a partial in there as well so that we can use it with the ajax function we'll see in a minute. This is defined as
// apps/frontend/modules/invite/templates/_invitation_row.php

<?php if(isset($errors['invitations'])) $errors=$errors['invitations']; ?>
<div class="form_row" id="invitation_<?php echo $key; ?>">
<?php foreach($form as $field): ?>
    <?php if($field->isHidden()) continue; ?>
    <?php echo $field->render(); ?>
<?php endforeach; ?>
<?php if(isset($errors[$key]): ?>
    <span class="form_errors" id="invitation_error_<?php echo $key ?>">
        <?php echo $errors[$key]->getMessage(); ?>
<?php endif; ?>

Quite a lot going on - and the sharp eyed will have noticed that we've added a call in the main template to an as-yet non-existent method in the inviteActions class.

Before we get to that though, I'll walk through what's going on

I won't bore you with drawing the forms manually - it's pretty standard, if basic, stuff but then we add a link to the form submit div and hide it in the stylesheet (so users without javascript won't get bothered by the offer of adding rows).

Under the form in the template, we output our javascript. There's probably a better way to do this, and when I find it, I'll update this :). But in the meantime, I did it like this so that I could write the url easily from PHP.

It's basic jQuery ajax so I'm not going to walk through the details of that either, suffice to say that we're throwing the count of the rows back to the new action that I mentioned earlier. Which we'll have a look at now.

// apps/frontend/modules/invite/actions/actions.class.php

class inviteActions extends sfActions
    // ...
    public function executeAjaxAddInvitationRow(sfWebRequest $request)
        $key = intval($request->getParameter('key'));
        $form = new MasterInvitationForm($this->getUser()->getGuardUser());
        return $this->renderPartial('invitation_row', array(

So what we're doing here is basically:
 1) getting out of here if it's not an ajax request
 2) grabbing the key
 3) creating a new MasterInvitationForm using the current user
 4) calling the form's addInvitationForm() method we created earlier
 5) returning that invitation_row, passing the newly created form referenced by $key to it for rendering.

The result is a new row appears in the form when you click the link. Great.


We haven't touched on validation yet. If you remember we added a new InvitationValidatorSchema object to the MasterInvitationForm way back at the beginning when we created that class.

It's about time we had a look at that now to get ever closer to having everything working.

// lib/validator/InvitationValidatorSchema.class.php

class InvitationValidatorSchema extends sfValidatorSchema
    public function configure($options=array(),$messages=array())
        $this->addMessage('invalid', 'Please enter a name and valid email');
        $this->addMessage('required', 'Please enter a name and valid email');
    public function doClean($values)
        $errorSchema = new sfValidatorErrorSchema($this);
        foreach($values['invitations'] as $key => $invitation)
            if((strtolower($invitation['name']) != 'name' && trim($invitation['name'])!='') || (strtolower($invitation['email']) != 'email' && trim($invitation['email'])!=''))
                $namVal = new sfValidatorAnd(array(
                    new sfValidatorString(array(
                    ), array(
                        'required'=>'Please enter a name and valid email'
                    new sfValidatorBlacklist(array(
                    ), array(
                        'forbidden'=>'%value% is not allowed'
                $emailVal = new sfValidatorAnd(array(
                    new sfValidatorEmail(array(
                    ), array(
                        'invalid'=>'%value% is not a valid email address',
                        'required'=>'Please enter a valid email address'
                    new sfValidatorBlacklist(array(
                        'forbidden'=>'%value% is not allowed'
                $errorSchemaLocal = new sfValidatorErrorSchema($this);
                    $invitation['name'] = $namVal->clean($invitation['name']);
                catch(Exception $e)
                    $errorSchemaLocal->addError($e, (string)$key);
                    $invitation['email'] = $emailVal->clean($invitation['email']);
                catch(Exception $e)
                    $errorSchemaLocal->addError($e, (string)$key);
                    $errorSchema->addError($errorSchemaLocal, 'invitations');
            $values['invitations'][$key] = $invitation;
            throw new sfValidatorErrorSchema($this, $errorSchema);
        return $values

So this looks massive, but actually all we're doing is iterating over the anticipated values from the post array - in this case the invitations array - and building some new validators to perform the cleaning on the incoming values.

The basic method of the errorschema can be found on the Symfony site - I'll post a link here when I have time to go get it.

But that's it -except. What happens when we add a row and submit it? The fields may all be validated, but we're getting csrf errors as there are too many fields for the form.

The way to combat this is to override the bind method of our MasterInvitationClass

// lib/form/MasterInvitationForm.class.php

class MasterInvitationForm extends sfGuardUserForm

    // ...

    public function bind($taintedValues=null, $taintedFiles=null)
        foreach($taintedValues['invitations'] as $key => $value)
            if(isset($value['name']) && strtolower($value['name']) == 'name')
                $taintedValues['invitations'][$key]['name'] = '';
            if(isset($value['email']) && strtolower($value['email']) == 'email')
                $taintedValues['invitations'][$key]['email'] = '';
            if($taintedValues['invitations'][$key]['name']!='' || $taintedValues['invitations'][$key]['email']!='')
        parent::bind($taintedValues, $taintedFiles);

If by If then...

We iterate over the invitations array in the taintedValues variable. In each case, if the key doesn't exist in the widgetSchema then we add it using the method we created much earlier.

Next we check to see if we still have labels set in the values. (To be fair, we probably ought to move that over to the bind function of the Invitations form but it's late and I want to finish this before bed :)) If we do, then we set them to an empty string so we can test for the empty strings

Lastly, if either of the values are not empty, we bind them to the invitations form - otherwise, we remove them both. This way, we don't get any extra fields errors or extra, empty records that we don't want.

There are ways to improve this without a doubt. I've left out a lot of stuff concerning basic data management and I've skirted round the proper way to do things here in the interest of speed.

WHEN I get time to come back and make some changes and improve this, I shall. Unitl then, night night - it's bed-time.

Sunday, 31 January 2010

Auto setting a user's group(s) using sfDoctrineGuardPlugin

I tried a number of different ways to achieve adding a user to a group automatically today before stumbling across the 'correct' way as per a forum reply from Fabien himself today.

I was trying all sorts of different methods, in the form, in the action, whereever... and finally it was pretty easy.

So let's assume you have RegisterForm which extends the sfGuardUserForm. The proper way to do it is in the doSave method of that form.

So your code might look something like...

class RegisterForm extends sfGuardUserForm
    protected function doSave($con = null)
        $isNew = $this->isNew(); // set this here, because it will change before you want to test for it...
         // ... any other pre-processing that needs to be done...


            $group = Doctrine::getTable('sfGuardGroup')->findOneByName('customer');

Quite easy when you know how and I guess this would be the way to auto-select other, required relationships too...

Monday, 18 January 2010

Forms & Forms

Two little snippets this morning (been busy already).

Firstly, form widget errors and I18n...

Brilliant stuff - if you add the error string you've defined in your form class into your I18n catalogue, it'll automatically translate. No further work needed!

Secondly - and this a little pointer :) I had updated a choice widget to set a default value. However, I noticed that even though the HTML was showing it as selected, Firefox was still showing the previous 'default' - the first option in my alphabetical list.

Clearing cookies for the site (presumably so a new session was generated) resolved the issue and displayed the expected default item.

Friday, 15 January 2010

Find a user's IP address within an action

Simple one but took some time to find the answer so I'm noting it here.

You could of course use standard PHP but that defeats the point of using a framework in my opinion...

But the quick answer is
$request->getHttpHeader('addr', 'remote'));

Easy :)

Custom Validators and messages

Only a quick one now but there's a couple of points here that are worth noting. Especially as I've just been going round in circles for 30 minutes trying to get my custom message to display when a Doctrine form field was empty.

To be fair, I'm not confident it was due to it being a Doctrine field but anyway...


1. If you use the 'addMessage()' method to add a custom method for an error code that is part of the default messages array within the validator base it WON'T get overwritten!!! If a default message exists it will always take precedence over the new version....

2. It's also worth remembering that the sfValidatorError (that your validator should extend from) class has it's clean() method called and this, in turn, calls the doClean() method. During the clean() method however, the isEmpty() method is called if the 'required' option is set to true. If the isEmpty() method returns true then your doClean() method won't ever be called... The way to get around this is to do one of the following:
a) set the 'required' option to false... not ideal if your value is required.
b) override the isEmpty() method to always return false and allow your doClean code to be called.

Done & Done

Wednesday, 13 January 2010

sfValidatorI18nChoiceLanguage "The following languages do not exist"

Another interesting issue with the symfony I18n system.

Using the language form from sfFormExtra plugin automatically uses the sfValidatorI18nChoiceLanguage object. We've set up our microsite to use the full format cultures made up of a 2 character language code, followed by an underscore and a 2 character code: e.g. en_GB for British english or fr_FR for French. This offers consistency with our other sites that also use this form of localisation selection.

However, while trying to use nl_NL, de_DE & fr_FR, we are told via a 500 server error that those languages don't exist.

Dumping out the languages obtained from sfCultureInfo::getInstance()->getLanguages(); shows that indeed, those 3 language selections don't exist. Only, nl, de, and fr do.

I don't know, as yet, why this is so - and i haven't found a consistent and 'correct' method to update my other languages but I shall update this post when I do find one...

sfValidatorI18nChoiceLanguage does not support the following options: 'culture'.

This is the first of two posts about the sfFormExtra plugin after finding a bug and an interesting 'inconsistency' yesterday.

We had need to use the language form bundled with the FormExtra plugin yesterday to build a microsite that needs to handle 4 languages. However, trying to implement the form resulted in the following error:

sfValidatorI18nChoiceLanguage does not support the following options: 'culture'.

The bug is logged in the Symfony ticket system but is not fixed yet. There is a patch at the link over there, but to be honest, it's just as easy to manually correct it until a new release fixes it.

Basically, change a line in the sfFormLanguage::configure() method from:
       'language' => new sfValidatorI18nChoiceLanguage(array('culture' => $this->user->getCulture(), 'languages' => $this->options['languages'])),
       'language' => new sfValidatorI18nChoiceLanguage(array('languages' => $this->options['languages'])),

Problem solved :)

Thursday, 7 January 2010

sfDoctrineGuardPlugin login with email bug

There is a bug in 4.0.1, but it's been patched here

NB: you may need to subscribe to the symfony project site in order to see the ticket - but if you haven't already, why are you reading about symfony bugs! LOL

How to display source code with line numbers and formatting in blogger

While setting up this blog, I realised I wanted some nicely formatted source code. As I'm blogging about coding, even if it is for my own use mainly, I wanted a solution that made the source code easy to read.

I settled on (after not too long a search) these two ideas which I have combined into one....

OK, so the css solution works like a charm, no need to go into detail here just pop over to the link above for details. Kudos to The Blogger Guide.

However, I had to tweak the line numbers solution. Again, most of the description at that site is valid, but I changed the way the new content is generated, not least using lists rather than tables. This should still  allow for content to be copied and pasted ok too.

Enjoy :)

Anyway, I'm not going through everything here as it's all been covered in the two links above. but I have posted the javascript and css I'm using on this site as a starting point:

pre {     border:1px dashed #aaaaaa;     background:#eeeeee;     white-space:pre;     font-family:monospace;     font-size:9pt;     line-height: 10pt;     height:auto;     width:auto;     padding:5px;     overflow:auto; } ol.pre {     margin: 0 0 0 15px;      padding: 0px;      padding-left: 5px; } .pre li {     margin: 0;     padding: 0;      background: none !important; }
and the JavaScript
function showLineNumbers() {
    * Written by Andreas Papadopoulos       *
    *      *
    *                  *
    *                                       *
    * And heavily refactored by Matt Keeble *
    *   *
    var isIE = navigator.appName.indexOf('Microsoft') != -1;
    var preElems = document.getElementsByTagName('pre');
    if (preElems.length == 0) return;

    for (var i = 0; i < preElems.length; i++) {
        var pre = preElems[i];
        var oldContent = pre.innerHTML;
        oldContent = oldContent.replace(/ /g," ");
        oldContent = oldContent.replace(/\n/g, "<br />");
        var strs = oldContent.split("<br />");
        if (isIE) {
            strs = oldContent.split("<BR />");
        if(oldContent.substring(0,4) == '<br />')
            oldContent = oldContent.substring(4);
        newContent = '<ol class="pre">';
        if(strs.length == 1)
            newContent+= "<li>"+strs+"</li>";
            for(var j=0; j < strs.length; j++) {
                if(strs[j] !== '')    newContent += "<li>"+strs[j]+"</li>"
        newContent+= "</ol>";
        pre.innerHTML = newContent;

sfDoctrineGuardPlugin and custom algorithms

OK - so I have hunted around for a solution to this problem twice now and struggled to find it. So as my memory's crap, I've decided to start this blog in order to capture the solutions, hints & tips I find. (yeah, this is the first post...)

So the problem is, that when using the symfony doctrine:data-load to import new data into my app, the sf_guard_user table was not being updated with the correct algorithm. Whatever happened, the default 'sha1' algorithm was being used.

I finally realised, that my app specific settings for the alogrithm_callable option were not being called during the data-load process, even though they were within the app - this obviously caused problems when trying to log in users constructed during the build/load process from the command line.

There are two solutions: 

The hard way
The reason it's the hard way is because it relies on my memory... the solution is to add the application option to the data-load task. So instead of just calling symfony doctrine:data-load actually call
symfony doctrine:data-load --application=frontend
(replacing frontend with the correct application name of course!)

The easy way
This solution is more robust because I don't have to remember to do anything different from normal.... basically, create app.yml in the project's config directory and add the option there. This way, you don't need to remember any additional options when running data-load and if you do want a different alogrithm to be used for your backend application for example (to ensure your customers can't login to the admin section without worrying about permissions) you can override the option on an app-by-app basis.
    So all is good - should solve the problem! I'm off to refactor my code :)

    [UPDATE:] make sure you run
    ./symfony cc
    to clear the cache before you try and reload the data - otherwise it won't work