Archive for the ‘Web Development’ Category

December 17, 2010
Filed Under (Architecture and Strategy, Open Source, Technology, Web Development) by Ollie Cronk on 17-12-2010

We’ve gone through quite a few security / penetration / web application tests at work (often as part of compliance with HMG SPF / InfoSec standards for UK Government projects) and thought it would be useful to list some of the steps you need to consider (hardening, configuring etc) to ensure your application has a reduced security exposure. I feel that you should view security testing as an opportunity to improve the quality of your work rather than see it as a box ticking exercise (ultimately the testing is about making your application more secure which can only be a good thing). Whilst a lost of our work is based on LAMP (Linux, Apache, MySQL, PHP) many of the concepts below apply regardless of the technology used.

Firewalls and Port Access

Firewalls and access to ports – one of the most obvious – but you need to consider whether the risk profile requires one or 2 levels of hardware firewall, or whether iptables is sufficient. Can you lock down the environment such that you only expose port 80 or 443 to wider internet and create a restricted IP address based white list for administration (eg SSH access)? On many of our Architectures we only expose the load balancer(s) and or proxy layer to the internet, everything else is not available at all to general IP addresses across the internet.

If you do have to have SSH open to all make sure that you install denyhosts (which helps to prevent SSH brute force attacks by adding persistant bad username/password attempts to /etc/hosts.deny – preventing access from the offending IP address)

Cross Site Scripting (XSS) and SQL Injection vectors

Check that your application does something sensible if someone attempts to put javascript into text input boxes. Check that putting in something like:

“><script>alert(‘If you see this in an alert box there is a XSS vector in your application’)</script> into a username box (for example) does. If it brings up an alert dialog you know you have a problem. See the  XSS Wikipedia page for more info.

Similarly for SQL – if you put in rogue SQL key words does it mess with the SQL that is run? Do something non- destructive (particularly if you are spot checking a live web site environment!) A good example I like to use is can I add parameters to a where clause to see data I shouldn’t be able to see.

Personally I prefer 2 levels of checks for SQL Injection and XSS type code in application input: – one at the application input layer (eg sanitising user input asap) and another at the database interface / wrapper layer to ensure nothing nasty can get sent to be stored or messed about with on the database tier.

Server Hardening / Configuring

Ensuring the server is setup and configured properly

Google for and check the hardening guide for the operating system for recommended steps.

Ensure that security updates are being applied on a regular basis.

Ensure that anti-virus software is installed (for the Linux Platform ClamAV is an option)

Review (and peer review if possible) the configuration files for the main services on this box – for LAMP this means a minimum of:

(You can run locate <name of config file> to check where it is located)

  • /etc/ssh/sshd_config
  • php.ini
  • httpd.conf / apache2.conf (depending on how the server is configured) and configuration files for virtual hosts / SSL configuration
  • my.cnf (or other database config)
  • Load Balancer config files (for Pound this is typically /etc/pound.cfg)

These checks are particularly important if you are having a white box review of your system (where you give the SSH login details to a security tester to check the configuration).

Pre test checks

Before you hand over the system to the Internet Security guys run some of the kinds of tools that they will be running yourself to see what is available. As a minimum run an NMAP command against your ip addresses:

nmap -A -vv [IP Address]

And see what ports (and information about the ports) is returned. Also check if NMAP can enumerate what Operating System and Versions of Web Server software is running (can you do anything to remove version numbers or product names?)

These days  I like to use Backtrack (a Linux Distribution design for security testing) for security checks. I am running it as a Virtual Machine from with my Windows 7 machine (http://g0tmi1k.blogspot.com/2010/01/tutorial-video-how-to-install-backtrack.html as a useful video for getting it set up).

I could probably write all day about security but hopefully this gives a feel for the key aspects. Would be interested to hear anyone’s tips or must dos for LAMP security.



August 27, 2010
Filed Under (Architecture and Strategy, Technology, Web Development) by Ollie Cronk on 27-08-2010

Continuing in my series on professional development – see the previous article on documentation here (ok so there has been a bit of a pause and I am stretching things to call this a series – I had intended to post this some time ago!). This post concentrates on the benefits of using an Issue / Task / Bug Tracking Tool…

Keeping track of development tasks and issues in a centralised system helps enormously. Living without task tracking for your issues is a lot like not having having source control for your code. A good task tracking system – such as Fogbugz or Countersoft Gemini helps keep track of what the team needs to do, allows issues to be delegated / reallocated to more appropriate team members and enables multiple lines of support (eg 1st line, 2nd line etc). It also allows transparency on tasks (allowing Jane who requested a new developer to check the issue tracker for progress rather than interupting the technical team) and (particularly for those of us that need to follow ISO9001 type standards) provides an Audit trail if used properly.

Of course its not just about standing up an issue tracking tool – you need to agree on things like

– what defines an High Severity issue over a medium severity one? What Service Level agreements do we have and how does the issue track tie to those (eg does selecting medium mean response within a day as opposed to high which requires a response within 1 hour for arguments sake).

– What is the process from issue inception through to resolution (does a new change request issue go to Bob -or better Bob’s role “Change Manager” – who allocates it to someone to estimate and changes the status to pending estimation).

– What level of documentation are you looking for in the comments associated with a case – is just referencing a source control commit enough (which is ok if your source control commits are verbose) or do you want a short explaination of what was done?

Clearly if done right this can allow your team to scale and stop your developers getting bogged down with admin (make sure there is someone overseeing the issue tracker). It can also make it easier to seperate support work from new big development work (the former you can give to junior colleagues to help them get up to speed with support from more senior ones – preventing senior guys/girls from getting bored with smaller stuff).

One other observation on this is that whilst you do need to be strict in order to implement these tools (eg ensuring that folks always use the tracker rather than continuing to email you all the time) you need to make sure they don’t become a barrier to communication between the technical team and its customers. One thing I like to do when involved in an operational issue is to cc the issue tracker in on an more detailed email explaining an issue – the customer gets a personalised response and the issue tracker captures the commentary (preventing time wasted by copying and pasting).

Would love to hear others thoughts on their use of issue tracking systems and the pros / cons.



March 01, 2010
Filed Under (Open Source, PHP, Technology, Web Development) by Ollie Cronk on 01-03-2010

Here are some notes / interesting products/thoughts that were mentioned (apologies this is more of a set of notes for me than a proper blog post – if I get time I will refine this!)

Started the day on a conference call back to the office so had to miss the keynote which was a shame as it was by quite an eccentric guy who Microsoft have hired (as a UX Architect Evangelist) largely about keeping thing simple and usability from what I gathered of the end of the talk.

Day was very tough as a I had a late night catching up on various things to allow me to free up the Friday – its difficult sitting through talks when really tired!

Met with several former colleagues from my last company (and former colleagues from my current company) so was a bit of a blast from the past at times.

There appear to be a lot of development and interest around NoSQL / document based databases at the moment – definitely something to keep an eye on as it matures as a technology.
http://www.phpconference.co.uk/talks

RDBMS in the social networks age
by Lorenzo Alberton

Database Graph Structures via advanced features of SQL, using SQL-99 and SQL-2003 functionality that certainly MySQL doesn’t have any many other DBs won’t have the 2003 extensions. Obviously using this kind of advanced functionality will have an impact on Database server load.

This talk felt a bit like it was flying in the wind of most new thinking at the moment (although to be fair – this is partly what Lorenzo has now put on his website below) which is to keep your database tier minimally loaded as it’s the part that has most issues with vertical and horzontal scalability (keep most of the CPU load in the web app tier as its easier to add more nodes there).

Slides available at:
http://www.alberton.info/talks

Legacy Code Talk by Ibuildings
doxygen – code documentation for any language not just PHP

ctags.sourceforge.net

BOUML bouml.free.fr (reverse engineering capabilities)

phpcs – Codesniffer (part of PhpUnderControl)

Thoughts for tackling older PHP4 based projects and code bases – get them in Source Control, start to apply Continous Integration type approaches.

Suggestions made around
Full isolation (separate server)
Using wrapper classes
Possible code rewriting routes for legacy code:
Going from random mix of PHP business logic and HTML outputting to neater procedural based code
Procedural to OO
OO to full OO

CouchDB
Early sight of the possible future of web application data persistance and replication. Interesting that CouchDB makes uses of HTTP as the connecting protocol. Might be possible (but probably not desirable apart from specific cases) in the future to create web applications that are JS direct to CouchDB in certain cases?

http://couchdb.apache.org/

Web and mobile application monetisation models / Paypal X

Paypal appear to be launching a new platform / API

  • Adaptive Payments
  • Pay multiplerecipients at once
  • Partnership
  • Chained payments (e.g. commission based payments)

Bit disappointed by this one as it was about PayPal’s API (https://www.x.com) rather than strategies for monetisation which is what the title lead me to believe.

Web Services Best Practise
At the beginning lots of stuff about basic HTTP (eg HTTP headers, Verbs)that ever developer should know about.

Lorna (also from iBuildings) who gave this talk seems to have a bit of a sarcastic talking down to you type tone I found slightly annoying – maybe she gives training to newbies all the time or something. Or maybe I was just tired. She had some interesting things to say about Web Services design particularly towards the end of her talk. The talk was caveated as being a bit of “a rant” and it was exactly that in places – felt like she was having a go at everyone a lot of the time!

Beers at the end sponsored by Facebook were a nice touch though, although I only had time to grab a quick one whilst chatting to Mark Schaschke from iBuildings and a couple of guys from my previous company. Think next year I will sit this one out to allow more developers to attend as think they will get more value out of it.



May 17, 2009
Filed Under (Open Source, Technology, Web Development, Wireless) by Ollie Cronk on 17-05-2009

One of my projects at the moment is to look at our options for building SMS enabled web applications (specifically for us around our Zend Framework based apps). Both for data capture (Inbound) and as an alerting / notification system (Outbound).

Thought I’d pull together some of my thoughts and reference material [not exhaustive or complete yet] in case its of use to anyone else in a similar situation. But first I’d like to thank my good friend Jem who helped identify some different angles on this…

Research Material:

As always the first place to start is Google and Wikipedia – http://en.wikipedia.org/wiki/SMS and http://en.wikipedia.org/wiki/SMS_gateways

LinkedIn Q&A is a great reference – here are a few relevant threads that I came across (you’ll probably need a Linkedin.com account to get to these) there are lots more if you search around with SMS related keywords.

Implementation Options:

There are 2 main options – and as always its the struggle between D.I.Y and DRY (Don’t Repeat Yourself – or my version DRY-OFF – DRY or others [for f sake? I just wanted it to be OFF as it sounded better; anyway I’ll shut up now!])

Roll your own

The Kannel FAQ covers this point quite well – http://www.kannel.org/faq.shtml#1.2

pros – complete control over messaging and ability to iron out any kinks in connectivity etc, potentially cheaper to run / only costs you what you use (rather than having to buy credits)

cons – more complex to setup in the first place, need to buy & setup some hardware somewhere etc

Useful article on Kannel on Ubuntu with PHP5: http://www.chipmunkninja.com/Setting-up-Configuring-and-Using-13@

Outsourced

Pros – ease of getting it up and running if the integration API (eg HTTP, XML/E-mail based) is easy to pick up

Cons  – my concern around these guys is how do you how good they are – will they disappear tomorrow? What gateways are they using, how reliable are their channels etc.

Guide to Gateways (US focused) but has some nice general considerations) http://www.developershome.com/sms/howToChooseSMSGateway.asp This site also has a really nice comparison table – which you could also use as a template for doing your own matrix/scoring comparisions of these services.

We will probably go with a combination of the 2 options – using our own system for the development of services (as we have greater control) and then making use of a partner once the message volumes go above what is finanically viable/scalable in house…

Once the technical bit is out of the way you then need to consider the usability and process flow around the app – eg if users are sending in data, queuing, acknowledging their submissions, correcting mistakes etc…

Hope to post more on this topic if I get the opportunity! If anyone has any insights or good resources on this topic then by all means please comment on this post!

Thanks



December 15, 2008
Filed Under (Open Source, Technology, Web Development) by Ollie Cronk on 15-12-2008

Have just upgraded to the latest and greatest WordPress and was shocked as it has wowed me again in terms of user interface. The pace of improvement of this Open Source product is incredible. Just as I think wouldn’t it be good if… they’ve done it in the next release (and improved stability / fixed bugs).

They’ve introduced a turbo button which makes use of Google Gears (the offline/browser enchancement) and added more time saving shortcuts and a better post writing interface (particularly for re-using existing tags on new posts – this UI for tagging taxonomy might have to be borrowed on some of my projects…) Read their blog post here for more info on features (includes a video).

Below is the new dashboard – compare the below to WordPress 2.5 that I upgraded to back in May 08

fireshot-capture-5-olivere28099s-yard-ollie-cronke28099s-blog-e280ba-add-new-post-e28094-wordpress-blog_cronky_net_wordpress_wp-admin_post-new_php

The new dashboard is below – it now has dragable “modules”

wordpress_wp-admin

Trouble is it now makes me want to upgrade the design of my blog – but I know I have way more important things to do with my time before I faff with that again!



December 14, 2008
Filed Under (Technology, Web Development) by Ollie Cronk on 14-12-2008

Internet Explorer 8 ScreenshotSeems the web browser market is hotting up again, what with the recent launch of Google’s chrome browser (which I suggest you check out – and the cartoon – if you haven’t already – although I have pretty much stuck to Firefox due to some extensions I can’t live without – link). And now Microsoft have IE 8 in public beta: – http://www.microsoft.com/windows/internet-explorer/beta/worldwide-sites.aspx

Will write more once I’ve tried the IE8 beta out but more it would seem that it has (in classic Microsoft style) “borrowed” quite a few features from elsewhere – including some of the stability enhancements that Chrome has introduced…



November 18, 2008
Filed Under (Technology, Web Development) by Ollie Cronk on 18-11-2008

LinkedIn has finally launched its applications area, clearly this has been in germination for some time.

Shame most of the ones I’ve tried are some what buggy – hopefully that will improve over time. Would be interested to know how your mileage with them goes…

Last time I looked the apps platform was a closed beta – will have to check and see if thats changed.

Post on their blog about it here: http://blog.linkedin.com/blog/



November 14, 2008
Filed Under (Open Source, Web Development) by Ollie Cronk on 14-11-2008

Alfresco Logo I came across Alfresco a couple of years ago at the MySQL European Customer conference – at the time they were relatively new and didn’t look that relevant. I’ve kept one eye on them since and have recently had the opportunity at work to look at alternatives to Sharepoint. Alfresco is now looking very, very interesting in this respect.

Alfresco’s features cover:

Its not that often (although admitedly it is now getting more common) you find an Open Source product that is strong on features / usability and is technically strong. The reason I say it is technical strong is due to the way it embraces and leverages Open Standards and Open Source Java components (Such as Hibernate etc – which good PHP frameworks have borrowed heavily from recently).

One of Alfresco’s people writes an interesting blog on the Business/Politics of Open Source: http://news.cnet.com/openroad/

Another Interesting Alfresco blogger is John Newton: http://newton.typepad.com/content/ He has an article on CMIS (Content Management Interoperability Services) which sounds fantastic.

Look forward to learning (and blogging) more about Alfresco if I get the opportunity…



October 24, 2008
Filed Under (Open Source, PHP, Technology, Web Development, Zend Framework) by Ollie Cronk on 24-10-2008

Today we put live the new Intranet at AEA which has been Kat’s major project (should get to spend more time with her now its launched!) with some support from me and impressive efforts from the dev team.

Previously we had a static HTML Intranet which wasn’t really serving the needs of the business as well as it could. The new site is based on the AEA Framework – which I’ve blogged about a bit before. Essentially an extension of the Zend Framework glueing Silverstripe and other 3rd party components together with a single sign on approach (in this case we are also using LDAP to do logins from our current Novell infrastructure). We built on our HTML content crawler tool to hoover in the static HTML into the Content Management System.

Over time we will leverage the framework to create more efficient web 2.0 style business processes and work with PPC to combine it with other off the shelf tools that they specialise in; and hopefully push it globally across the group…

I’ve also had some exciting news this week which I will share when I can.

All exciting stuff!



August 30, 2008
Filed Under (Technology, Ubuntu, Web Development) by Ollie Cronk on 30-08-2008

If your not having any joy when installing addons in Firefox 3 (sorry can’t remember the specific error message) – but nothing seem to fix it – even re-installing) we managed to confirm the fix today at work. Note that involves completely clearing all Firefox related info (so settings, bookmarks etc will be gone)…

* Uninstall Firefox
* Then make sure that the C:\Program Files\Mozilla Firefox is completely gone
* Also clear out the C:\Documents and Settings\<<username>>\Application Data\Mozilla folder (it will probably still contain Firefox and Extensions directories)
* Re-install

Voila – you can install extensions again (like the brilliant FireBug)

This also works for Linux (had a similar issue at home with the Latest version of Ubuntu).