Supermarket Software

My wife bought a custard scroll for my Saturday breakfast. I was really looking forward to eating it, but when morning came & I sat down to devour its bready-custardy goodness with my coffee, I was sorely disappointed. It had come from the supermarket, not a bakery. It had the semblance of a custard scroll, & was packaged attractively, but it was all wrong. It reminded me of the last time I bought an umbrella from a supermarket - it let me down in the first rainfall.

I fear that we too often write software just like that. We might be getting better at building & deploying, but the software quality is falling down.
One of the side-effects of agile & lean is that fast-paced often gets interpreted as cutting corners. That's simply not the intent.
Another side-effect is that if you only build what you need to satisfy the requirements, then you'd better make sure that you articulate your requirements thoroughly - & then align them with what gets created, through demo & review.

I don't think there's anyone in the supermarket who is taste-testing scrolls or opening umbrellas. They're in the business of selling stuff, not providing a quality service.
This is why we shouldn't be treating software development in that way - software is a service to your business or else your customers.
It shouldn't have designed obsolescence unless that is a specific requirement; & yet too many people build that "feature" in but saying "it will never need to do ..." without actually thinking about the use of the software. In fact, software developers don't really know what the software might be used for, & have to hold themselves back from thinking of code re-use.
Product managers are the experts, using their knowledge of the market, their audience testing, etc.

Software shouldn't go onto the supermarket shelf (be deployed) unless the product manager has tasted it & is happy with the flavour & texture & utility of the software product. I'm not just talking about BAs & testers performing another series of processes that follow the requirements, but that higher level of understanding of what the software is meant to do, relative to its consumer base.

Sometimes, we fall back on the waterfall approach & say "well, that's what you asked for, & it's too late to change", or "that's all we have the budget to deliver". In effect, you're just selling those umbrellas that only open once. It's definitely not agile.
It hurts the company's reputation & bottom line, if you're talking about a product, & it hurts the business unit's ability to deliver anything in the future, even if it's an internal release of services.

A CEO from my past once used to say "we don't have to build the Rolls Royce".
He was right - we need to build the Beetle - the product that everyone was happy to use, was reliable, & targeted at its audience. Behind the scenes, German manufacturing's precision & quality was legendary - & very successful.

We, as software developers (& I include the whole development process here) have the responsibility to work like a bakery - taking care of each masterpiece we create - rather than like a supermarket pumping out software-like things in neat packages because it's cheap to do so.
A good bakery never goes out of business. A supermarket often has to heavily discount its unsold product.

Story Cost & Story Value

I read a complaint recently along the lines of "why do we still fill in timesheets?" There was a lot of opinion from the conservative "that's how project accounting works" to the radical "we don't need them". Somewhere in the middle, timesheets are relevant, useful, & painless.
These three things are not the same, & people too often try to fulfill only one aspect.

Timesheets are useful when they reflect the reporting needs. However, if you're not reporting on the actual activity, then there is a disconnect. Either, you have to change what you are reporting, or you have to accept that the data you are reporting with is not real. That's a disconnect - & a sad fact.

Timesheets are painless when they can be done "automatically" as part of an existing process - preferably without the team member doing anything. This could be as simple as monitoring their keystrokes (big brother = no bother), but it is better if the team are clocking all of their time against tasks in an issue management system - & every activity must be in the system.

This allows you to perform reporting against what is really happening, as opposed what is supposed to be happening. This is relevance.

There are plenty of tools out there to do this type of tracking, & none of them have the word "timesheet" in them.

They do, however, all track time in minutes & hours. In modern agile processes, activities are not estimated in time, but in story points, & they are often prioritised by business value. Therefore, there is a cost - estimated effort - against each piece of functionality that provides business value. In a lean environment, if there is no business value, just don't do the task.
We use this estimated cost in planning. We also use measures like velocity to determine scheduling: the number of story points - & therefore value - that the team as a whole can produce within a sprint (a time period).

We seem to have most of the elements in place to estimate a financial cost for each unit of business value that is delivered as long as we can determine a tangible correlation between delivered functionality & ROI - in terms of market share, increased sales, the value of the IP, or share price.

How can we define that true value of each feature?

Look at the approach taken to determine story points - it comes from experience. The team - the experts who are doing the work & have been doing it - agree upon the most likely cost, using tools like planning poker & the team velocity. They rely on the fact that, over time, estimations even out & have lower variance through increased skill & practice. Most importantly, they don't estimate in man-days. Still, tracking in hours, & having two-week sprints, "works".

The equivalent must be true of business value points - looking back over what impact a feature has on the value of past releases of product, plus the appetite in the market (based on end-user interviews, A/B testing, prototyping, etc), a relative score (not dollars) could be developed that would vastly improve the input to planning product development & this would also contribute to the accountability of the process in a much more tangible way than timesheets that track activities.

Pass the (Open) Source

I have been slightly freaked out recently by the extremely positive feedback I've been getting around a project I've been working on that is based heavily on an open source solution.

I've been down the open source path many times before - almost exclusively as a consumer or indirect contributor (offering suggestions to the project owners) - but not as a contributor with a major enhancement, say. It's just never happened. Most open source "does enough" for what I want to do as is.

At first I was intrigued that my manager showed such excitement that I could take something effectively free & get it to a point where it was actually useful for the project. As I said, I've used enough projects before to expect a non-perfect solution, but usually something viable comes up.

Then, she started getting excited with the idea of me sharing my experiences with other teams to encourage them to think in terms of not just using open source, but doing so with the intention of contributing back to those projects from the outset.

Again, not that radical from my perspective ... until she explained that the company - a financial services giant in the market - never thinks of open source as an option, let alone thinks of being a part of the community of dedicated IT professionals who spend their spare time, or else contribute from their company's development time, to make stuff for other people to just find & use as they see fit.

Admittedly, there are two very different software worlds out there - vendor world, & open source world (plus some blurry bits in the middle). Those two worlds only differ in their attitude towards licensing, not in their attitude towards quality.

In fact, there is a massive industry out there effectively being the consulting services, bespoke development or customisation, or support arms of open source projects. These people are an important part of that open source community because they get direct customer feedback on what makes better product (as opposed what sells better). They drive the development in the same way that vendor product is developed from marketing strategy & product leadership.

Yet many service sectors that rely on IT professionals to provide infrastructure (in financial services, telecoms, utilities) "feel more secure" out-housing the mitigation of their technology risk to the very people providing the technology, that is, through licensing & maintenance agreements.

Once upon a time, banks in particular used to have massive in-house development creating bespoke solutions that worked exactly the way they needed. That's out of favour. What's in favour is getting in the same consultants as the competition did to build something sufficiently different .

I know that my company is not one-hundred-per-cent happy with that arrangement. Very few companies are. I can name very few enterprise solutions that "just work" to the point that you can believe the marketing hype.

So, now's the chance to try something different. Start small & look to see if there is a way to introduce open source solutions into a project - cost it out in terms of support. If that works for you, try enhancing an open source project & contributing to the open source community to become an influencer of the development of such solutions, rather than just a consumer.

The "cost" of transforming thinking may be less than you think. The benefits may be more than simply financial. The end goal is to be able to provide a better service to your customers by owning more of the knowledge around the service & being able to make your service better under your own control.

Mind Your Language

I don't like admitting this, but I feel I've cheated on a recruitment test.

A while back, I was interviewed for a role with a company that worked predominantly with NodeJS. I told them I was a novice, & they said that was OK, because they needed someone who could do some project management work, & I had that experience.
Then they asked me to do their NodeJS test. They said I shouldn't spend more than, say, four hours on it.

I learnt enough NodeJS, & a few other bits of infrastructure I'd never heard of, & produced a competent solution for their test - all in under four hours. I even specifically left out the enhancements that I'd thought of for testability & stability, on the basis that I didn't want to press my luck.
To me, NodeJs is still just javascript. It's no silver bullet.

Two things about the situation have worried me since:

• why test someone's competence in a technology that they can learn from scratch in a few hours?
• why insist on recruiting for that skill if you can train someone in a few hours?

These two questions are at the heart of my broader concerns for the software development industry in particular in the areas of trainig & hiring.

I am a Java dev. I can do lots of other things, but Java's been my core language for fifteen years. I've learnt the new features as they've come on board. I've been exposed to a lot of libraries, frameworks, & open source solutions over the years that have allowed me to develop things a lot faster than I used to, because I can leverage the experience of others' in the community.

I am aware that you can probably do the same things in C#/.Net for the same reasons, but have no interest in doing the same things in a different way if they achieve similar results.

What worries me is when someone specifically tries to do one thing in a different way because it's "too hard" to learn how to do it in the current way. If you look at NodeJS as nothing more than a few short-cuts in (server-side) Javascript, then it's really a technology that saves time & effort for a few mundane tasks. Excellent. Move on.

If you think of NodeJs as a revolution because it makes it easy to do those mundane tasks, then you've missed the point of software development in general. NodeJS is a triumph of software development, not a panacea for it. The people who developed it are craftsmen who knew their language (Javascript) & used a lot of experience & skills to solve real world problems in their community.

Only the marketing folk believes that NodeJS solves the problem of world hunger.

A good javascript dev now breathes a sigh of relief when they have to do that boilerplate for web services. A bad javascript dev (or a NodeJS dev qualifying after four hours) thinks that they are God's gift to the software industry because they can replicate some googled examples.

Whatever programming language you develop in, if you learnt it in a matter of days - or even weeks - then you cannot possibly understand why it's useful, or in what context it should be used. Even after you know the limitations, you need to learn how best to use the programming language - through patterns, best practices, software architecture, design, testing, etc.

This is not something you can learn on your own - ever. It is certainly not learnt in a short amount of time. It comes from a lot of commercial experience (where other people are your customer) & teamwork (where other professionals give you feedback).

If you can acquire "skills" that can be learnt sufficiently to pass a test within hours, then why aren't employers taking some responsibility for that training - offering guidance, mentoring, etc - to ensure that the skills proliferate within an organisation, & to encourage learning in the workplace?

If companies appreciate fine craftsmanship in their software deliverables, then why don't they encourage an environment that fosters improving the skills of the developers to make them better at what they were hired to do?

Although all developers should find out about new technologies & play around & learn constantly, they should also learn more about their craft & the technologies that they already claim to be familiar with, with the hope that they will get better, help others to become better, & contribute to the betterment of their organisation.

Death to Project Managers

Can you imagine a world where a killer virus transmitted through Gantt charts wiped out the population of project managers? How would anything get done?

The short answer is - "by doing things differently" ... & "no".

But bear with me for a few paragraphs. Taking a leaf out of an old Edward de Bono book, simply imagine that project management as we know it is impossible & we have to find another way to get things done that does not involve the skills & techniques that we currently use.
If we had to reinvent the process, would it come out the same?

We don't have to reinvent. In fact, I've worked for several companies that don't have project managers - or they have one who doesn't get involved in projects directly.
Most variations on the Agile development methodology have no room for project management as such, because teams are self-directed.

You can see project management as a top-down approach to achieving a goal. It's quite traditional. Agile is always bottom-up. There's an assumption that the professionals doing the work know more than management about how to deliver the outcome - as long as they are empowered to make decisions & effect them, & have access to the information they need.

Admittedly, agile works best in small teams, but there have been many examples of teams of teams (layers of self-organising) where a team representative is the conduit to a higher level of abstraction. Certainly, if that higher level is concerned with business strategy, for example, then its membership is made of team leaders - those who can ensure the smooth working of the development team.

In this structure, project managers are not appointed to a project. Teams are assigned a business outcome. In some cases, teams own a product or a service, & their output is a part of the overall enterprise's offering - a direct contribution to the business' viability.

The alternative to project management is product management - not someone to ensure that deadlines are met, or to negotiate budgets; but someone to ensure that quality is defined relative to the business needs (usually associated with what an end-user expects), & can understand & articulate the requirements to the team & to management (to justify investment of time & materials).

Project managers have budgets. Product managers seek investment.

Product managers are invested in the product - the outcome - as is the team. Their passion is what drives the business' value proposition to their customers & investors in turn.

We can survive without project managers. For how long can a business survive without product managers?

Arts & Crafts

If you've ever read (or tried to read) Bob Martin's "Clean Code" book, then you will realise that there's a lot more to writing good code than using the IDE's built-in syntax checker - no matter what language you're writing software in. If you're using vi or emacs, then you're on your own.

If you've never written software commercially - say, a recent graduate - you might not get the point at all.
If you've been writing software for a few years, you should get frustrated with the book, looking for that "secret" that will make you the master of your profession.
If you've been writing software for a long time, you flip through nodding your head & say to yourself "sounds about right - what's the big deal?"

That perspective on the book equates to the attitude of those different stages in a career to what should be the core of a software developer's skillset.
It might have many names - call it artistry or craftsmanship, or just experience.
The most important part of it is that you don't acquire it from a book - Bob's book tries to capture it in the same way that an artist captures a scene or a good software developer captures a requirement. Bob's book is not a how-to for the uninitiated, but a biography of an artist.
He's as much a good author as he is a good software developer.

That book - & there are very few like it - has lost popularity because it was written by a master technician for other masters. Any apprentice should have the background to understand what is being said, but doesn't have the experience to understand why it needs to be said, or what to do with the knowledge.

In a way, that's a sad thing to say about our industry.
I've used terms there (apprentice, master) that make perfect sense in a trade context, but no longer make sense even in an artistic context, let alone a software development profession.

We still have tradesmen working in some companies who are masters - they oversee apprentices, they have done their time in several establishments honing their craft, & are now considered at their peak & responsible for their deliverables.

We have lost that concept of the professional artist as an equivalent set of stages - the master sculptor with his journeyman assistants, surrounded by apprentices doing the preparation work, cleaning up & looking after tools.

We have too quickly lost the concept in software development.
I have a vague recollection of graduates being mentored by senior devs, working closely with mid-level devs to learn the skills & techniques. Pair programming was good for this.
Those masters of software development who came up with the agile methodologies have been cast aside in favour of doing things quickly or cheaply, with no time to spend on training apprentices.
The focus has been on tools that could be learnt without supervision & on the fly.

If, as an industry, we fail to train apprentices, continually employing only experienced developers, then at some point we must realise that those developers were never apprentices.
They never learned anything at the direction of a master, or under the eye of a journeyman.
No matter how many years of experience they have accumulated, or books read, or courses attended, if a developer has never worked in a team where their work is constantly under review, or they have never contributed to the review of their peers, then they are effectively self-taught.

As creative & talented as some artists are, unless someone teaches them how to use their tools - brush-stroke technique, sharpening a chisel, mixing glazes - what they create will always lack the fundamentals, the professionalism, that we should expect.
We, as an industry, need to hold our expectations for professionals up high, as a light by which we see the true artistry or craftsmanship of those who would be, or claim to be, masters of software development.