Categories
Architecture and Strategy Open Source Technology Web Development

Preparing a web environment for Security Penetration Testing…

We’ve gone through quite a few security / penetration / web application tests at work (often as part of compliance with HMG SPF / InfoSec standards for UK Government projects) and thought it would be useful to list some of the steps you need to consider (hardening, configuring etc) to ensure your application has a reduced security exposure. I feel that you should view security testing as an opportunity to improve the quality of your work rather than see it as a box ticking exercise (ultimately the testing is about making your application more secure which can only be a good thing). Whilst a lost of our work is based on LAMP (Linux, Apache, MySQL, PHP) many of the concepts below apply regardless of the technology used.

Firewalls and Port Access

Firewalls and access to ports – one of the most obvious – but you need to consider whether the risk profile requires one or 2 levels of hardware firewall, or whether iptables is sufficient. Can you lock down the environment such that you only expose port 80 or 443 to wider internet and create a restricted IP address based white list for administration (eg SSH access)? On many of our Architectures we only expose the load balancer(s) and or proxy layer to the internet, everything else is not available at all to general IP addresses across the internet.

If you do have to have SSH open to all make sure that you install denyhosts (which helps to prevent SSH brute force attacks by adding persistant bad username/password attempts to /etc/hosts.deny – preventing access from the offending IP address)

Cross Site Scripting (XSS) and SQL Injection vectors

Check that your application does something sensible if someone attempts to put javascript into text input boxes. Check that putting in something like:

“><script>alert(‘If you see this in an alert box there is a XSS vector in your application’)</script> into a username box (for example) does. If it brings up an alert dialog you know you have a problem. See the  XSS Wikipedia page for more info.

Similarly for SQL – if you put in rogue SQL key words does it mess with the SQL that is run? Do something non- destructive (particularly if you are spot checking a live web site environment!) A good example I like to use is can I add parameters to a where clause to see data I shouldn’t be able to see.

Personally I prefer 2 levels of checks for SQL Injection and XSS type code in application input: – one at the application input layer (eg sanitising user input asap) and another at the database interface / wrapper layer to ensure nothing nasty can get sent to be stored or messed about with on the database tier.

Server Hardening / Configuring

Ensuring the server is setup and configured properly

Google for and check the hardening guide for the operating system for recommended steps.

Ensure that security updates are being applied on a regular basis.

Ensure that anti-virus software is installed (for the Linux Platform ClamAV is an option)

Review (and peer review if possible) the configuration files for the main services on this box – for LAMP this means a minimum of:

(You can run locate <name of config file> to check where it is located)

  • /etc/ssh/sshd_config
  • php.ini
  • httpd.conf / apache2.conf (depending on how the server is configured) and configuration files for virtual hosts / SSL configuration
  • my.cnf (or other database config)
  • Load Balancer config files (for Pound this is typically /etc/pound.cfg)

These checks are particularly important if you are having a white box review of your system (where you give the SSH login details to a security tester to check the configuration).

Pre test checks

Before you hand over the system to the Internet Security guys run some of the kinds of tools that they will be running yourself to see what is available. As a minimum run an NMAP command against your ip addresses:

nmap -A -vv [IP Address]

And see what ports (and information about the ports) is returned. Also check if NMAP can enumerate what Operating System and Versions of Web Server software is running (can you do anything to remove version numbers or product names?)

These days  I like to use Backtrack (a Linux Distribution design for security testing) for security checks. I am running it as a Virtual Machine from with my Windows 7 machine (http://g0tmi1k.blogspot.com/2010/01/tutorial-video-how-to-install-backtrack.html as a useful video for getting it set up).

I could probably write all day about security but hopefully this gives a feel for the key aspects. Would be interested to hear anyone’s tips or must dos for LAMP security.

Categories
Open Source PHP Technology Web Development

PHP Conference UK 2010 Notes and Thoughts

Here are some notes / interesting products/thoughts that were mentioned (apologies this is more of a set of notes for me than a proper blog post – if I get time I will refine this!)

Started the day on a conference call back to the office so had to miss the keynote which was a shame as it was by quite an eccentric guy who Microsoft have hired (as a UX Architect Evangelist) largely about keeping thing simple and usability from what I gathered of the end of the talk.

Day was very tough as a I had a late night catching up on various things to allow me to free up the Friday – its difficult sitting through talks when really tired!

Met with several former colleagues from my last company (and former colleagues from my current company) so was a bit of a blast from the past at times.

There appear to be a lot of development and interest around NoSQL / document based databases at the moment – definitely something to keep an eye on as it matures as a technology.
http://www.phpconference.co.uk/talks

RDBMS in the social networks age
by Lorenzo Alberton

Database Graph Structures via advanced features of SQL, using SQL-99 and SQL-2003 functionality that certainly MySQL doesn’t have any many other DBs won’t have the 2003 extensions. Obviously using this kind of advanced functionality will have an impact on Database server load.

This talk felt a bit like it was flying in the wind of most new thinking at the moment (although to be fair – this is partly what Lorenzo has now put on his website below) which is to keep your database tier minimally loaded as it’s the part that has most issues with vertical and horzontal scalability (keep most of the CPU load in the web app tier as its easier to add more nodes there).

Slides available at:
http://www.alberton.info/talks

Legacy Code Talk by Ibuildings
doxygen – code documentation for any language not just PHP

ctags.sourceforge.net

BOUML bouml.free.fr (reverse engineering capabilities)

phpcs – Codesniffer (part of PhpUnderControl)

Thoughts for tackling older PHP4 based projects and code bases – get them in Source Control, start to apply Continous Integration type approaches.

Suggestions made around
Full isolation (separate server)
Using wrapper classes
Possible code rewriting routes for legacy code:
Going from random mix of PHP business logic and HTML outputting to neater procedural based code
Procedural to OO
OO to full OO

CouchDB
Early sight of the possible future of web application data persistance and replication. Interesting that CouchDB makes uses of HTTP as the connecting protocol. Might be possible (but probably not desirable apart from specific cases) in the future to create web applications that are JS direct to CouchDB in certain cases?

http://couchdb.apache.org/

Web and mobile application monetisation models / Paypal X

Paypal appear to be launching a new platform / API

  • Adaptive Payments
  • Pay multiplerecipients at once
  • Partnership
  • Chained payments (e.g. commission based payments)

Bit disappointed by this one as it was about PayPal’s API (https://www.x.com) rather than strategies for monetisation which is what the title lead me to believe.

Web Services Best Practise
At the beginning lots of stuff about basic HTTP (eg HTTP headers, Verbs)that ever developer should know about.

Lorna (also from iBuildings) who gave this talk seems to have a bit of a sarcastic talking down to you type tone I found slightly annoying – maybe she gives training to newbies all the time or something. Or maybe I was just tired. She had some interesting things to say about Web Services design particularly towards the end of her talk. The talk was caveated as being a bit of “a rant” and it was exactly that in places – felt like she was having a go at everyone a lot of the time!

Beers at the end sponsored by Facebook were a nice touch though, although I only had time to grab a quick one whilst chatting to Mark Schaschke from iBuildings and a couple of guys from my previous company. Think next year I will sit this one out to allow more developers to attend as think they will get more value out of it.

Categories
Open Source Technology Web Development Wireless

Options for sending & receiving SMS from web applications

One of my projects at the moment is to look at our options for building SMS enabled web applications (specifically for us around our Zend Framework based apps). Both for data capture (Inbound) and as an alerting / notification system (Outbound).

Thought I’d pull together some of my thoughts and reference material [not exhaustive or complete yet] in case its of use to anyone else in a similar situation. But first I’d like to thank my good friend Jem who helped identify some different angles on this…

Research Material:

As always the first place to start is Google and Wikipedia – http://en.wikipedia.org/wiki/SMS and http://en.wikipedia.org/wiki/SMS_gateways

LinkedIn Q&A is a great reference – here are a few relevant threads that I came across (you’ll probably need a Linkedin.com account to get to these) there are lots more if you search around with SMS related keywords.

Implementation Options:

There are 2 main options – and as always its the struggle between D.I.Y and DRY (Don’t Repeat Yourself – or my version DRY-OFF – DRY or others [for f sake? I just wanted it to be OFF as it sounded better; anyway I’ll shut up now!])

Roll your own

The Kannel FAQ covers this point quite well – http://www.kannel.org/faq.shtml#1.2

pros – complete control over messaging and ability to iron out any kinks in connectivity etc, potentially cheaper to run / only costs you what you use (rather than having to buy credits)

cons – more complex to setup in the first place, need to buy & setup some hardware somewhere etc

Useful article on Kannel on Ubuntu with PHP5: http://www.chipmunkninja.com/Setting-up-Configuring-and-Using-13@

Outsourced

Pros – ease of getting it up and running if the integration API (eg HTTP, XML/E-mail based) is easy to pick up

Cons  – my concern around these guys is how do you how good they are – will they disappear tomorrow? What gateways are they using, how reliable are their channels etc.

Guide to Gateways (US focused) but has some nice general considerations) http://www.developershome.com/sms/howToChooseSMSGateway.asp This site also has a really nice comparison table – which you could also use as a template for doing your own matrix/scoring comparisions of these services.

We will probably go with a combination of the 2 options – using our own system for the development of services (as we have greater control) and then making use of a partner once the message volumes go above what is finanically viable/scalable in house…

Once the technical bit is out of the way you then need to consider the usability and process flow around the app – eg if users are sending in data, queuing, acknowledging their submissions, correcting mistakes etc…

Hope to post more on this topic if I get the opportunity! If anyone has any insights or good resources on this topic then by all means please comment on this post!

Thanks

Categories
Environment Technology

Some recent browsing (Green Issues, SOA, Dashboards)

Some sites that I found interesting recently…

I’ve been looking at quite a lot of Green IT/Business type issues recently www.climatechangecorp.com is a good site (even if it does heavily promote its events on the content).

Carbon Calculators article: http://www.climatechangecorp.com/content.asp?ContentID=5119 Its disappointing it mentions AEA only in the context of providing the emissions factors – we actually do a whole lot more including building online carbon/emissions data platforms (and calculators) ourselves.

Great article from Adobe on Web 2.0, SOA etc – gives a great overview: http://www.adobe.com/devnet/articles/web_end.html

Sun Microsystems came into to talk to us about their Java CAPS recently and it looks really interesting. We’ve already started to look at NetBeans IDE more closely (particularly the PHP plugin and its SOA/Business Process (BPEL) diagram/visualisation to BPEL XML code features…

See http://www.sun.com/software/javaenterprisesystem/javacaps/index.jsp

Dashboards, Flex with PHP etc:

One of my current projects is update our (data) visualisation and dashboard piece of our offering – as such we’ve been researching whats out there that we could use to enhance our technology stack:

Flex and PHP http://devzone.zend.com/article/3580-Building-Dashboards-With-PHP-and-Flex

Zend Framework/Adobe Media Remoting – http://corlan.org/2008/11/13/flex-and-php-remoting-with-zend-amf/

Interestingly Microsoft seem to be about to launch a new Dynamics AX product which is focused at Environmental data management – interesting to see the main IT vendors start to move into the space that we’ve been in for some time. This is really good news as it will give more deployment and integration options moving forward (eg for customers already using Dynamics).

Categories
Open Source PHP Technology Web Development Zend Framework

New Zend/AEA Framework based Intranet deployed @ work

Today we put live the new Intranet at AEA which has been Kat’s major project (should get to spend more time with her now its launched!) with some support from me and impressive efforts from the dev team.

Previously we had a static HTML Intranet which wasn’t really serving the needs of the business as well as it could. The new site is based on the AEA Framework – which I’ve blogged about a bit before. Essentially an extension of the Zend Framework glueing Silverstripe and other 3rd party components together with a single sign on approach (in this case we are also using LDAP to do logins from our current Novell infrastructure). We built on our HTML content crawler tool to hoover in the static HTML into the Content Management System.

Over time we will leverage the framework to create more efficient web 2.0 style business processes and work with PPC to combine it with other off the shelf tools that they specialise in; and hopefully push it globally across the group…

I’ve also had some exciting news this week which I will share when I can.

All exciting stuff!

Categories
Open Source PHP Technology Ubuntu

Installing Zend Core PHP on Ubuntu Linux (8.04)

Warning! Techy post – if your not into Web development you’ll want to skip this one…

I’ve been wanting to try out Zend Core (Zend’s standard PHP stack / extension or “Reliable PHP made easy”) for a while but as Ubuntu isn’t officially supported I’ve held off. A problem with some code running on the Ubuntu version of PHP 5 forced my hand – I discovered that its actually not that hard to get ZC up and running on Ubuntu. Thought I’d share in case this is helpful to anyone…

  1. Download, unpack and run the ./install command as per usual
  2. I opted to stick with the version of Apache already installed (as its slightly newer than the Zend bundled one)
  3. My experience of Zend attempting to configure Apache varied – one some boxes it almost worked, others it didn’t. If you find the installer crashing out just untick the configure Apache option and try again.
  4. Once Zend Core has completed you’ll need to tweak things abit…

PHP5 Module

The ZC installer will have probably left your Apache configured with 2 LoadModule php5_module entries (one in /etc/apache2/apache2.conf – thanks to ZC and the other Ubuntu one in /etc/apache2/mods-enabled/php5.load). So you can either do a sudo a2dismod php5 to remove the Ubuntu one or modify the Ubuntu one to the following:

LoadModule php5_module /usr/local/Zend/Core/modules/apache22/libphp5.so

In fact you can be cunning here by commenting out the Ubuntu PHP5 extension and adding the ZC one in this file – allowing you to effectively switch between the 2 different extensions should you need to.

MySQL extension – if you’re using MySQL then you’ll need to watch out for the fact that the bundled MySQL extensions look for the MySQL socket file in /tmp/mysql.sock rather than the Debian / Ubuntu location which is /var/run/mysqld/mysqld.sock

In most cases changing in php.ini:

mysql.default_socket =
to
mysql.default_socket = /var/run/mysqld/mysqld.sock

Does the trick; but alas not in all cases it seems (eg MySQL PDO). Really hacky fix to this (please do let me know if you have something better – other than editing the socket in the /etc/my.cnf file) is to run a scheduled (using crontab) symlink of /var/run/mysqld/mysql.sock to /tmp/mysql.sock

To do this run: sudo crontab -e
and add the following line:

 5 * * * * ln -s /var/run/mysqld/mysqld.sock /tmp/mysql.sock

And for now that works for me. If you want a copy of some of the relevant config files / snippets (including the extra bit that ZC adds to the apache2.conf) then I have zipped them up for ease.

Zend Core itself looks like a good product – being able to access to php.ini options via web based tool is pretty useful, as is knowing you are working on a consistent & tested version of PHP. Its a shame MySQL doesn’t offer a similar free / unsupported version of their MySQL Enterprise in a similar way really…

Categories
Open Source PHP

Upgraded to WordPress 2.5.1…

This blog is now running the latest version of WordPress which offers an overhauled admin area one of the first enhancements you notice is a nice new dashboard with metrics.

WordPress 2.5.1 dashboard screenshot

Widgets management in WordPress 2.5

I’ve discovered the dynamic sidebar (which allows widgets to be added using the visual admin interface rather than editing code in the template files) and switched on the tags widget in this version (I’ve been tagging articles for a while they’ve just never shown up). See the bottom screenshot – what you can’t see is the nice drag and drop re-ordering of the widgets in the sidebar.

Some playing around was needed to get the tags looking reasonable in the sidebar. By default tags are just rendered in text or flat mode. However by changing the line 1362 in widgets.php from

wp_tag_cloud();
to:
wp_tag_cloud($args = array('format' => 'list'));

The tag cloud will then render as a HTML list which looks a bit better (until I can get around to doing some styling and turn it into a true tag cloud – where the tag words get bigger the more popular they are).

Categories
Open Source PHP Ubuntu Web Development Zend Framework

Zend Framework to be bundled with Ubuntu Hardy (8.04)

Interesting & good news – Zend Framework is being bundled as part of Ubuntu Linux: http://devzone.zend.com/article/3232–Zend-Framework-to-be-part-of-Ubuntu

Direct link to article: http://andigutmans.blogspot.com/2008/02/zend-framework-to-be-part-of-ubuntu.html

Also found an interesting presentation on version 1.5 of the Zend Framework.

Categories
Open Source PHP Technology Web Development

Professional Development – Part 1 – Using a Wiki for Documentation

This will be the first in a series* of articles on web applications development – not the specifics about programming, but more tips on the infrastructure and processes that can make life easier, more productive, successful and better aligned with best practises. Its based on my experiences of being in development teams and leading development teams.

I see these articles as being useful to a development team thats growing from a 1 or 2 man operation to a larger team and is perhaps using Open Source development tools such as PHP/PERL/Python and perhaps aren’t in a very processes driven environment…

Documentation

Most developers won’t generally document their work as a matter of course – either they simply forget, overlook it or its just not that exciting for them. So 3 things:

  • Illustrate the value of documentation
  • Build it into the development process
  • Make it exciting (oh okay you can never make documentation that exciting – so lets settle for a bit more interesting!)

Illustrating the value – most developers are already sold on documentation being a generally good idea but others aren’t. Some fear that by documenting they loose control over the project or the work that they primarily work on (in fact the reality is that is the opposite…) or they just really don’t see the point. Highlight the facts that it enables team work, improves quality, makes support and changes easier etc. Also that holding all the knowledge up in your head means that you are stuck in your current role as its not easy to bring others in to do what you do so you can be promoted.

Another great benefit is inducting new team members – it allows you to point new team members at the wiki site to help them get up to speed quickly – and you can also use that process to fill in any gaps in the documentation (and get the new start to include their tips and findings as they learn the ropes…

Build it into the development process – obviously you need to have a development process if you haven’t got one but once you have it just becomes part of the steps:

  • Requirements Gathering – document
  • Spec/design – document
  • Build/development – document

So ensure that documents are required for each step in the process – and make time for that documention. The nice thing is that documentation is a lot easier if done throughout the project life cycle rather than all at the end (then it is just really daunting) – as generally what you plan to do is what gets delivered (and if things deviate from the spec during development you can just adjust it)…

Make it more interesting – its more that just that in reality as its picking a documentation tool that supports the above points and works for the team. For me development documentation seems to works well with an internal/Intranet Wiki (something like MediaWiki for example). The main benefits (over office docs for example) it allows easy collaboration, allows for a geographically disbursed team and is generally nicer than using word processing software. It’s made more interesting by feeling very “Web 2.0” (as much as I hate the phrase!) and has some great tracking features – like the recently edited articles page. Once the team have seen the advantages and you bore you colleagues to death with “the W word” then you’ll find that you have a healthy wiki site and documentation, documentation, documentation (with any luck with minimal pain!)

Categorising the Wiki – here are some ideas on some categories that work:

  • Projects – Pages under here relate to discrete projects the team is working on
  • Infrastructure – documentation on the dev, staging and production server environments (eg Server configs, IP addresses, SLA details)
  • Process – Reference guide to the development and other IT processes so the team can refer to them and also make contributions and feedback on what is working.
  • Meeting Notes – Agendas and minutes for meetings – if you get this right then these allow for team communications when needed (rather than at forced meeting dates) and are links to other pages on the Wiki where the full details of a proposal or a project is available.

Blogging

Another option (which has some other positive side effects including marketing) is blogging about development projects – and this is something I am considering for my current team. The Wiki will be for the more technical and internal documentation aspects and the blog for what the project actually does. The added benefit of the blog is that it can do some link / SEO stuff for your projects and raise the profile of the development work that your team is doing to a wider audience. This is particularly useful if you develop an Intranet system for your company – where you are adding new features or enhancements over time.

To make MediaWiki easier to use (for those who aren’t familar with the syntax used on Wikipedia) we enabled a WYSIWYG editor – FCKEditor. There is a whole range of great Wiki software out there if you don’t like the look of MediaWiki – just do a Google or have a look at wikimatrix.

The next articles will more than likely be about:

  • Project Management tools (eg FogBugz) –
  • Development Meetings and Communication
  • Using a Framework (eg Zend Framework)

… watch this space!

*I don’t know how many there will be yet but if they are received well then heck there might be as many as 3 or 4!