Stories Before Bedtime

I find myself, yet again, forgetting my own advice & doing things in a manner that will give me unending nightmares in the long run.
Agile methodologies always talk in terms of stories. This works for software development when you incorporate non-technical people into a project. You allow them to answer the question "What is it meant to do?" - whether that's a single feature or the whole product - & their answer drives how it does it, how the deliverable is tested, & how success is measured (acceptance criteria).

Why do we forget these things as we go deeper into the development cycle?

Stories are good, in general, for all phases of the product life cycle, & indeed for all phases of business around product. If you can't clearly & succinctly state what the end goal is, or what the deliverable looks like, then failure is the most likely outcome.
That's reality.

Stories start with the big picture, but getting down to the nitty gritty, every component, every package, every class needs its own story. I get a little frustrated that I can't 'document' a package within the code, but that's why we have things like documentation in the first place.
Pictures are meaningless unless they have some words around them that say why things are represented as having certain relationships. Generating pictures from the code doesn't give you anything beyond what's in the code. If the code doesn't have a story in itself, then there is no story 'created' by generating pictures.

Each class should explain succinctly, then with more detail, what the purpose of that class is. That's why we have headers - not just for copyright or company policy - but to tell people what's going on. Tell your peers. Tell your audience - which is anyone who opens that class file.
Similarly, each method needs to have it explained clearly what the purpose is. It's not just the public methods, because that's all that the default javadoc setting generates - it's for every schmuck who follows you. It's for you in six months time when the droob you'd handed the code over to leaves the company suddenly.

Explain what the purpose is. Explain what the outcomes are. If you're not writing tests until afterwards (heaven forbid), then those tests will be meaningless unless they correlate with what the methods do, & therefore what needs to be tested.
Everything needs to be explained as if to newbie. Assume nothing. Start with the simplest possible statement & elaborate. Make references to other classes, packages, components, products - anything that might give the reader a clue as to what your thinking was when you used a HashMap instead of a TreeMap even if you know that the key is Comparable. There could be a valid reason, & it could be one of those things on which the operation of key features depends. If you don't tell the story, then no-one will know.

Stories should give you peace of mind. They should help you to sleep at night, knowing that you've done the best job that you can. Stories should come first, but they need to be in place before you can say that you've finished a project & you can put the whole deliverable to bed.

Failure Driven Development

I'm back in the dev chair at the moment - as opposed the chair that directs devs. Time & again, I reminded of how test-driven development is a wonderful philosophy becaue of what it gives you in terms of direction & scope, & also the artefacts left behind in terms of regression testing. I could not sing the praises enough, & have tried in vain to get devs to comprehend the sheer delight of discovering that example code, proven software, & a general plan are just 'there' for everyone if you do things the right way. Obviously, you have to embrace the vision, because simply following the formula is not enough - you gain no insight, you gain no value.

The reason why test driven development should work for a conscientious dev is that a test is written to fail, & it is the challenge set for the dev to make it not fail. Devs like challenges - I know I do. Failure is not a bad thing. We have to learn from our mistakes, & extreme programming, where test first is a pillar, says that courage is needed to accept 'mistakes', share them, deal with them, & move on.

If a failing test is a 'mistake' - the software doesn't do what is needed, even if it is a new requirement - then you definitely need to discuss it, not bury it. You need to communicate as a team why the software doesn't already fulfill that requirement, & how it will be changed to do so. This makes perfect sense. This is the design phase of any new work. It is the same when the software 'should' have been able to perform the function, but somehow doesn't - a bug. There is no difference in the approach in those two cases. Neither should be hidden, both should be discussed before attempting the corrective work.

If we think calmly about the design phase in this way, then the next stage is thinking calmly about the coding phase in the same way. When approaching the problem of "I need to add this new piece of functionality", most devs follow the white rabbit down the hole with a series of speculative phrases like "to do that, then this needs to happen, which means ..." until they disappear up their own cavity.

Pretty soon, you've got half the code in transit trying to make it all work up & down the line of changes. If, instead, you take a failure-driven approach to development, then the first thing you do is the new function's interface - the requirement is to display a widget, so I'll assume it's there & just display it. To display a widget, I call on the widget's display method, because that's how widgets work. Oh. No widget, no display method. My current UI doesn't work, but it would if I had a widget with a display method.

Stop right there. Make the UI call the widget's display method. It will work once the widget has been developed - even in stub. For that matter, it will work right now with a mock object. You can test the UI. That's just like test-driven development. You can complete the UI as you define the widget. When the UI 'works', you move on to the widget.

Learn to love your IDE. I've been a big fan of Eclipse for years & feel sorry about the demise of NetBeans (sorry, been too long since I used IntelliJ). The important thing is that these IDEs are getting powerful enough to help you develop code (not software) because the people who develop the IDE are the same kind of people who use them.

If you sit at your UI class & type in "widget.display()", then the IDE will tell you that the class doesn't exist - & will help you create it - & that the method doesn't exist - & will help you add it - without leaving the UI class unnecessarily. Those compile failures are beautiful things that allow you to code up quickly. Don't be afraid of creating compile errors just to see where things go wrong. Don't try to keep all of the impacts of your changes in your head. Don't start at the lowest level & forget how the UI was going to use the widget in the first place.

Embrace change. Be driven by 'failure' to complete the functionality intended, not just write the code you expect to write. Test. Test first. Love the tests. Make them fail, so that they become better & more useful.
Never comment out a test. Never make test code that only checked what you were developing at one point in time - either that code is useful enough to be a part of regression, or else it should never have been written because it wasted time best spent writing a real test.

Failure is good. It makes us grow & be better at what we do. It helps us to learn. It also makes it easier next time to avoid the same pitfalls. Recognise the signs of impending failure & either deal with them directly (mitigate), or take the hit & deal with the failure afterwards. In the end, it doesn't matter, as long as you don't bury failures.

If this post has missed its mark, then I will have learned more than if I had never written it.