Categories
Architecture and Strategy Technology

Fix bugs before adding features applies to projects not just code…

The 12 rules of the Joel Test served me well when I was a developer and development manager. In particular the principle of fixing bugs before writing new code. I will be honest though and say that at times to be pragmatic this had to be fix bugs whilst developing new features! But the bugs could not be over looked – after all this technical debt would bite at some point. Generally at the middle of the night on a weekend…

It strikes me that when looking at a typical company project portfolio a similar mistake is being made at a project level. Projects to deliver new capabilities or features are being initiated on a somewhat unstable foundation. You can think of the “bugs” at this level as a highly manual (or prone to failure) process as well as technology related issues. I am not saying here that IT teams shouldn’t innovate or add new features or be business aligned – of course they should. But we all need to grow up a bit and realise that the short cuts that get made to deliver projects on time and budget need to be addressed if we want to do sustainable  business in the digital age.

Sustainable business means not just delivering new products or features quickly; but to be able to continue to do so. To be able to continuously innovate quickly and to respond to market forces. Also building robust, scalable, flexible, secure (etc) solutions that lead to satisfied (hopefully delighted) customers – who aren’t constantly irritated by failings in your services. Of course there is a balance to be struck  and in reality it comes down to having a good strategy (e.g. building flexible, well integrated, adaptable platforms rather than churning out a mess of point solutions). There is also a degree of risk taking here too – being bold and deciding what the capabilities are that you believe the business will need in order to respond to the market – rather than having your hand forced into being entirely reactive.

Often I feel like Enterprise Architecture gets challenged (“what value does it add?”) or plainly just isn’t understood (“sorry what was your job title again”, “what does that mean exactly”). And as EAs we often don’t do ourselves any favours by not making it simple for stakeholders to understand our value proposition. For me this area is a fundamental value and benefit of Enterprise Architecture. In that you have a team who is focused on pulling the Enterprise (or if just constrained to thinking technology the IT estate) towards a sustainable path for future success. In a world where projects are often king its important to have a team thinking about the longer term effects and how to pragmatically address failings.

Categories
Architecture and Strategy Random Thoughts Technology

The importance of being courageous in IT; balancing between strategy and responsive delivery

Disclaimer – this doesn’t really describe a single organisation that I have worked at – it’s a collective summary of my experience of working in IT (and that of present and former colleagues) working in medium and large sized organisations. Also the core message probably applies to many other business areas and not just IT in  the value of thinking strategically (and the value of Enterprise Architecture).

Many of you reading this working in an organisation over a few hundred people will recognise that IT is often not able deliver effectively. Either in its ability to provide what the business needs today or its ability to be adapted quickly to the demands of the markets it operates in. Often IT systems are fragmented, silo’d and un-able to share data with each other. This leads to horrible/bizarre manual processes (such as manual re-keying of information) to allow business units to work effectively with each other, cross-functionally. It often seems too much of a bold move to take step back and plan or focus on internal IT improvements when there is so much demand for business driven change that needs to be done yesterday.

The key thing that needs to happen to most organisations IT landscape is that it needs to be simplified. The horrible evolved mess needs to be analysed and worked through to understand how to make it simpler. Some technical teams may criticise architects for wanting to make the IT landscape “look prettier”. However I believe that simplicity = ease of understanding, ease of use, faster to change and crucially lower cost to operate. All good things surely? Sometimes a team mentality might be to keep things as complex, messy or misunderstood as possible – so that they are “indispensable”. But that also means they can’t really be promoted. In technical terms – just like you can have very bad messy programming code – the same applies at the IT landscape level across all the different systems and teams.

I believe a lot of the problems are down to the fact that IT systems tend to evolve rather than being properly planned. Of course there is going to be a degree of emergence when organisations are big and complex and not everything can be planned for; but to me if feels a little like many organisations are in a hole and keep digging themselves deeper. By this I mean that due to the lack of roadmapping and thinking more end to end about what data, systems, processes and skills are needed it results in more and more tactical workarounds to keep delivering. Each time a new solution is added it just makes things more complex and harder to change in the future.

Its easier to be reactive and been seen to deliver, deliver, deliver than think strategically alongside delivery. Also thinking strategically is hard work. It takes time to understand the bigger picture, abstract problems, create models and think about where things should go and how they should work. Not only that but its also hard to think about how to transition from the mess you are in today to your target state once you have come up with it.

I fear this is one of the reasons IT professionals can become reactive – simply responding to the next request from the business to deliver something as quickly as possible. And of course delivering for the business isn’t a bad thing –  just if its done in a way which doesn’t think about the future state of the organisation or the architecture where problems creep (or flood!) in over time.

IT personnel can promoted to recognise their loyalty (and because of the detailed understanding of the mess that has evolved, and they may even be a one man dependency) rather than their ability to take the next step up (and think more strategically). Sometimes this means that they still have to do elements of their previous roles and don’t actually have time to do their new roles properly. So all this compounds the problem – as they often created the problems in the first place they may not radically change approach – if they even recognise some of the problems they need to be brave to admit they made mistakes in the past that need to be put right. That is if they even have the time to think about them – their may simply be fighting the next fire.

“We’ll fix that in the next phase” – How often are promises made to unpick tactical work arounds and technical debt later on but then never happens.

“This is just how it works around here – we don’t have time to improve our processes and systems as we are too busy delivering”

“Our funding is based on a 12 month period – all work needs to deliver by the end of the year – we cannot have projects that go over multiple financial years its just not how the planning cycle works”.

“We don’t ever decommission anything – we just add new systems but as we don’t know if the old ones are still used for something business critical we leave them alone.”

IT costs then simply build up over time to a point where almost all the budget is spent on running stuff that the business is already reliant on and there is then less and less time or money to work strategically. Leading to a vicious cycle.

What is the answer? Well of course there isn’t a magic bullet but I do think some maturing is needed – becoming more confident in pushing back on certain things in order that a better long term path can be taken. Becoming confident in challenging not only the business but technology management. Making sure that business sponsors prioritise and not just claim that everything is top priority and needs to be done now. But also thinking about the full lifecycle of a solution – not just implementing it rolling it out and then letting it rust. Very few people seem to consider how long systems will be used for – 5 years? 10 years? When should you consider to retire an application? Talking about retirement of  a system you are just rolling out seems to be taboo.

Personally I believe you have to try and make time to consider the possibilities of new technology or process approach on your organisation or department – not because you want the technology on your CV but because you can see clear business value – that you can articulate to others. Sell your ideas, if you have to use some of your own time to create roadmaps – they don’t have to be long and complex they can be 1 or 2 page diagrams (showing as is and to be; along with supporting business justification).

Explain the risks of taking a reactive approach – one man dependencies are a massive operational risk for example. Not considering how a solution will scale to meet demand is a reputational risk waiting to happen – run through what if scenarios with your stakeholders to get them to understand why things need to change and/or why investment in internal improvement is crucial. The improvement to IT employee engagement can be a key selling point too – particularly if you have a churn issue in your IT team – ask yourself why people aren’t happy and engaged.

And of course its a balance between getting something out the door quickly which might open up a market opportunity, being engaged with the business and longer term simplicity. You can fall into a trap of being very academic by following architectural frameworks to the letter and getting very theoretical (although a dose of that – i.e. 1-2 determined, principled, purist architects to pull things in a different direction can be healthy for very immature organisations).

One thing I would say is don’t give up on trying to improve – even if its just incremental improvements – maybe to the data models to begin with, introducing a principle, improving documentation, making something more portable or secure (as its generally the non functional requirements like security and scalability that suffer). Think about what the biggest impact will be to the organisation (and in fact what will free up technology team time so you can pick off the next challenge?)

You should reach a tipping point where you can start to deliver things more consistently and with a high level of quality – and then it will then click with everyone else and people will wonder why they didn’t plan more and consider things over a longer time frame before!

Hopefully some food for thought anyway…

 

Categories
Architecture and Strategy Open Source Technology

Is “big bang” IT dying? Replaced with iterative approaches and best of breed Open Source?

This is an article I have been stewing on for a while and having recently changed from a consultancy largely working on public sector IT projects back to a private sector IT department its given me several different view points.

I also recently attended the excellent Zapthink SOA and Cloud course in Amsterdam – so I am now a Licensed/Certified Zapthink Architect!

Zapthink course in Amsterdam
Zapthink course (creating a SOA implementation roadmap), my colleague Martin is on the left. FB have changed their access to photos outside FB so this no longer works 🙁

Time for a change?

In the continued difficult financial climate will organisations continue to have the appetite and budget to  invest in large scale greenfield COTS (Commercial Off the Shelf) IT projects and licensing? e.g. Large scale commercial enterprise systems such as ERP? And what’s the next success for Open Source Software (OSS)?

Is the future a more incrementally / agile delivered open source, best of breed systems? Rather than big monolithic, generic packaged software that does everything ok but doesn’t excel at much if anything. And worst of all, often requiring the business to change its processes to fit the software. Of course the lines between commercial software and Open Source are becoming more blurry – with “Commercial Open Source” (in other words commercially backed and supported).

I am thinking here of solutions that are developed on Open Standards / common platforms (eg J2EE) using common / standards based middleware and the XML family of technologies  to connect them together. Of course there is a risk that if you pick and choose lots of niche software that serves its job well then you can end up with a big mess of spaghetti integration and duplication. But that is where effective Architecture, standards and Governance comes in; to keep things on the right track and aligned with business priorities.

Certainly the agile (iterative) methodology seems to be taking hold in larger companies, although waterfall still seems to be favoured in government – due to the perception that it will result in a fixed cost.  Unfortunately too often it doesn’t deliver successful results as its too rigid, ends up costing far more through cunning use by the vendor of change control and depending on the project the initial build can be as little as 10% of the total costs in any case.

What about the cloud? Isn’t that supposed to reduce costs…

I think many in the IT industry (well some vendors anyway) right now would argue that the answer to this is delivery via the cloud using a pay as you need it service based model (to get away from having to make the big upfront investment in hardware and licensing). I guess this is an option but I think most large businesses (who have the budgets for the larger IT projects) are looking at the cloud quite sceptically, waiting for it to mature beyond e-Commerce and online type applications and add the required security and reliability that is needed. Keeping things in their own data centre and exploiting virtualisation to optimise costs at the Infrastructure layer. Cloud as your Disaster Recovery (DR) / Data Archiving environment looks like one of the most compelling use case so far.

I am seeing some suggestions that organisations would like to adopt this approach in some areas (eg Integration). In fact one of the places I worked in the past built its own home grown ERP / eLearning platform on Open Source. In my current role we are looking at Open Source alternatives – particularly for Integration and Infrastructure glue.

Its interesting to see how the adoption of Open Source has matured – from just the Linux OS used for servers, Linux + Apache for static web moving towards LAMP and other Apache projects such as Tomcat etc even more so with “Web 2.0”. Data Integration / ETL is a big area for OSS – eg Talend, ActiveMQ, Glassfish. J2EE is a big success story too.

And of course now with Android OSS has finally come into contact with the casual end user (rather than the techies like me that run Linux on the desktop). This was brought home to me the other day when a completely non IT friend showed me his Motorola Xoom and was extolling its usability etc.

Interesting times. Wonder where OSS will infiltrate next? I guess the answer is probably wherever it can disrupt the marketplace in a engaging way for the consumer, or with a commercial model that is compelling to business/IT decision makers.

Categories
Architecture and Strategy

Useful resources for Enterprise Architecture

Since attending the Seminar by Gartner on Enterprise Architecture last year I have been focussing on formalising my IT Architecture skills (well when time allows!!). TOGAF (The Open Group Architecture Framework) 9 appears to be the way to go. You can think of it much the same way as PRINCE2 is to Project Management – its something that provides the core principles but it needs to be tailored to the organisational specifics.

Came across “TOGAF 9 in pictures” available on http://www.orbussoftware.com/downloads which is a really effective way of getting to grips with the core concepts.

Also (on the same site) found a stencil for ArchiMate; Archimate is a means of standardising the way that Enterprise Architecture is defined at a high level. Chapter 2 of the specification makes good reading – has a nice summary on why EA: http://www.opengroup.org/archimate/doc/ts_archimate/index.html

From my research IT Architecture (in particular Enterprise Architecture) still seems to be something that different people and organisations view differently – in particular role definitions / responsibilities seem to vary massively. I also fear that often the goal behind EA initiatives aren’t clear enough and some organisations just want to “tick the EA box” rather than get true value from it.

Categories
Architecture and Strategy Technology Web Development

Thoughts on and benefits of task trackers and seperating support from development

Continuing in my series on professional development – see the previous article on documentation here (ok so there has been a bit of a pause and I am stretching things to call this a series – I had intended to post this some time ago!). This post concentrates on the benefits of using an Issue / Task / Bug Tracking Tool…

Keeping track of development tasks and issues in a centralised system helps enormously. Living without task tracking for your issues is a lot like not having having source control for your code. A good task tracking system – such as Fogbugz or Countersoft Gemini helps keep track of what the team needs to do, allows issues to be delegated / reallocated to more appropriate team members and enables multiple lines of support (eg 1st line, 2nd line etc). It also allows transparency on tasks (allowing Jane who requested a new developer to check the issue tracker for progress rather than interupting the technical team) and (particularly for those of us that need to follow ISO9001 type standards) provides an Audit trail if used properly.

Of course its not just about standing up an issue tracking tool – you need to agree on things like

– what defines an High Severity issue over a medium severity one? What Service Level agreements do we have and how does the issue track tie to those (eg does selecting medium mean response within a day as opposed to high which requires a response within 1 hour for arguments sake).

– What is the process from issue inception through to resolution (does a new change request issue go to Bob -or better Bob’s role “Change Manager” – who allocates it to someone to estimate and changes the status to pending estimation).

– What level of documentation are you looking for in the comments associated with a case – is just referencing a source control commit enough (which is ok if your source control commits are verbose) or do you want a short explaination of what was done?

Clearly if done right this can allow your team to scale and stop your developers getting bogged down with admin (make sure there is someone overseeing the issue tracker). It can also make it easier to seperate support work from new big development work (the former you can give to junior colleagues to help them get up to speed with support from more senior ones – preventing senior guys/girls from getting bored with smaller stuff).

One other observation on this is that whilst you do need to be strict in order to implement these tools (eg ensuring that folks always use the tracker rather than continuing to email you all the time) you need to make sure they don’t become a barrier to communication between the technical team and its customers. One thing I like to do when involved in an operational issue is to cc the issue tracker in on an more detailed email explaining an issue – the customer gets a personalised response and the issue tracker captures the commentary (preventing time wasted by copying and pasting).

Would love to hear others thoughts on their use of issue tracking systems and the pros / cons.

Categories
Architecture and Strategy Random Thoughts

MDM, EA, KM – Too much IT terminology & Acronyms?!

Those of us in the IT profession (or Information Management as one colleague recently suggested as an alternative*) don’t do ourselves many favours when it comes to using complex terminology and also expecting business people to understand and embrace IT best practises…

Whilst adopting concepts/practises such as Enterprise Architecture (EA), Data Governance, Information Management (IM), Knowledge Management (KM) are all well and good, the sheer number of buzz phrases and concepts must be bewildering for most  non techies. I will admit that sometimes I struggle with the difference for example with Master Data Management vs Master Reference Data without resorting to Google or Wikipedia.

Of course some will argue that is what the Architect or Analyst roles are all about – to match business requirements to IT solutions. But if we ever want colleagues or clients or stakeholders to truly embrace the concepts of Knowledge Management or Data Management / Governance we need to break down these barriers.

Its all too easy to get DM/IM/KM confused if its not the way you think. Generally / at a high level its accepted that Data can be converted into useful Information and that humans (eg employees) walk around with a lot of Knowledge that often needs to be managed (and shared) more effectively. But often we don’t take the time to even explain these concepts – we just jump into enterprise IT lingo and expect others to know what we are on about (or why it makes business sense). Sometimes colleagues can get confused by products such as Sharepoint and what they do – as they can think they are the solution to Knowledge Management – when actually they are just the product or underlying tool that can enable Knowledge Management – its embracing the core concepts of KM that is key.

If we are not careful we will go start to regress back to the bad old days of IT where the IT guy was locked in the cupboard as no one understood him…. Ok maybe that’s going too far but you know what I mean!

Or maybe I am being unfair? After all every different business area I have worked in seems to have its own Acronyms (finance is a nightmare with IPOs, CFA, Swaps, Derivatives etc etc etc) – is it now accepted practise to just Google terms you don’t understand and be proactive about learning these things? Unfortunately in my experience some people aren’t prepared to do that (unless its in their area of expertise) – and you just switch them off or loose them before you can sell them the juicy or beneficial part of the story.

* Information Management was selected as to not confuse people with “plain old Information Technology” – the physical desktop PCs, laptops etc and kit that every business needs. Information Management it was argued is different as it is the leveraging of IT capability (where IM people are part of the core business team) to improve the way Information is managed (or processes are operated) and used as an Asset rather than something just delegated to IT to “sort out”.

Categories
Open Source PHP Technology Web Development

Professional Development – Part 1 – Using a Wiki for Documentation

This will be the first in a series* of articles on web applications development – not the specifics about programming, but more tips on the infrastructure and processes that can make life easier, more productive, successful and better aligned with best practises. Its based on my experiences of being in development teams and leading development teams.

I see these articles as being useful to a development team thats growing from a 1 or 2 man operation to a larger team and is perhaps using Open Source development tools such as PHP/PERL/Python and perhaps aren’t in a very processes driven environment…

Documentation

Most developers won’t generally document their work as a matter of course – either they simply forget, overlook it or its just not that exciting for them. So 3 things:

  • Illustrate the value of documentation
  • Build it into the development process
  • Make it exciting (oh okay you can never make documentation that exciting – so lets settle for a bit more interesting!)

Illustrating the value – most developers are already sold on documentation being a generally good idea but others aren’t. Some fear that by documenting they loose control over the project or the work that they primarily work on (in fact the reality is that is the opposite…) or they just really don’t see the point. Highlight the facts that it enables team work, improves quality, makes support and changes easier etc. Also that holding all the knowledge up in your head means that you are stuck in your current role as its not easy to bring others in to do what you do so you can be promoted.

Another great benefit is inducting new team members – it allows you to point new team members at the wiki site to help them get up to speed quickly – and you can also use that process to fill in any gaps in the documentation (and get the new start to include their tips and findings as they learn the ropes…

Build it into the development process – obviously you need to have a development process if you haven’t got one but once you have it just becomes part of the steps:

  • Requirements Gathering – document
  • Spec/design – document
  • Build/development – document

So ensure that documents are required for each step in the process – and make time for that documention. The nice thing is that documentation is a lot easier if done throughout the project life cycle rather than all at the end (then it is just really daunting) – as generally what you plan to do is what gets delivered (and if things deviate from the spec during development you can just adjust it)…

Make it more interesting – its more that just that in reality as its picking a documentation tool that supports the above points and works for the team. For me development documentation seems to works well with an internal/Intranet Wiki (something like MediaWiki for example). The main benefits (over office docs for example) it allows easy collaboration, allows for a geographically disbursed team and is generally nicer than using word processing software. It’s made more interesting by feeling very “Web 2.0” (as much as I hate the phrase!) and has some great tracking features – like the recently edited articles page. Once the team have seen the advantages and you bore you colleagues to death with “the W word” then you’ll find that you have a healthy wiki site and documentation, documentation, documentation (with any luck with minimal pain!)

Categorising the Wiki – here are some ideas on some categories that work:

  • Projects – Pages under here relate to discrete projects the team is working on
  • Infrastructure – documentation on the dev, staging and production server environments (eg Server configs, IP addresses, SLA details)
  • Process – Reference guide to the development and other IT processes so the team can refer to them and also make contributions and feedback on what is working.
  • Meeting Notes – Agendas and minutes for meetings – if you get this right then these allow for team communications when needed (rather than at forced meeting dates) and are links to other pages on the Wiki where the full details of a proposal or a project is available.

Blogging

Another option (which has some other positive side effects including marketing) is blogging about development projects – and this is something I am considering for my current team. The Wiki will be for the more technical and internal documentation aspects and the blog for what the project actually does. The added benefit of the blog is that it can do some link / SEO stuff for your projects and raise the profile of the development work that your team is doing to a wider audience. This is particularly useful if you develop an Intranet system for your company – where you are adding new features or enhancements over time.

To make MediaWiki easier to use (for those who aren’t familar with the syntax used on Wikipedia) we enabled a WYSIWYG editor – FCKEditor. There is a whole range of great Wiki software out there if you don’t like the look of MediaWiki – just do a Google or have a look at wikimatrix.

The next articles will more than likely be about:

  • Project Management tools (eg FogBugz) –
  • Development Meetings and Communication
  • Using a Framework (eg Zend Framework)

… watch this space!

*I don’t know how many there will be yet but if they are received well then heck there might be as many as 3 or 4!