ERP Customization 80/20 Rule

Infoworld:

Although open source implementations invite all sorts of customization, a clear lesson learned from the ERP wars of the ‘90s appears to have sunk in: When it comes to commercial software, avoid hacking the system when possible or you’ll end up with maintenance costs equal to or greater than those incurred by apps developed in-house.

 After my experience as an ERP manager in the 90s, I concluded that if you customize more than 20%, you would have been better off building.

Eating your own dog food, in Corporate in-house software development

Many contemporarly writers on software development, including successful sofware entrepreneurs, such as Joel Spolsky, Dave Winer and Paul Graham, emphasize the importance of "eating your own dogfood". This phrase is used a lot in contemporary software development, maybe a bit too much, but what it alludes to is using your product. Not just casually, but intensively. The eat-your-own-dogfood theory makes a lot of sense to me, in theory and in terms of the really good software I have ever used, and also the really awful software (PeopleSoft, for instance).

It is easy to see how it can work for consumer software, or tools used by software developers. It is more challenging to make it happen with in-house corporate systems. I mean, how many developers have ever used a claims-payment system, or a reservation booking system, for example? In my experience, large corporations fail terribly in this regard--and oustourcing and offshoring isn't making it any better.

The problem can be attacked. There are several techniques that I have seen to go a long way. One is to bring people with operational experience into IT. Unfortunately, the trend to offshoring it cutting off the opportunities for entry-level transition into IT. I know of Computer Science undergrads, with average credentials, who have left the field, because they could not get their foot in the door, for a development position. So it is even more unlikely that someone will be able to transition from a business operations job to a technical IT job; the best they can hope for is to make it as a Business Analyst.

Another, even less-used alternative is to have IT people spend some time "apprenticing" in functional areas. Although this has a lot to recommend it, this was never popular, and is even less likely to be tried now. So the last alternative is creating a scaled-down version of this situation. Have the IT people spend a significant chunk of time, 1-2 cumulative work weeks, sitting with real users, and learning their processes. Then, continue that partnership, where those real users--at the do-er level, not at the management level--continue to be closely involved in and consulted throughout the software-development process. Whenever I have done this, the payoff has been huge.

Agile: Fail-Fast vs Perfect

One of the benefits of Agile software development is, quite simply, that it allows you to fail fast. If you assume that most first efforts are going to be failures, this becomes a very attractive feature. Build something, get very concrete feedback that shows you why your creation is a failure, and then incorporate that feedback into version 2.

Fast-fail saves immense resources. Because in software development, perfecting, polishing and productionizing takes a TON of work. With Agile, you don't go on to make those investments in a version that is going to be thrown away. You only undertake that very heavy lifting when you have something that sticks to the wall.

An incidental benefit to this arrangement is that the results the 3Ps will be higher quality, and probably more efficiently achieved, because you are documenting something concrete and complete, as to something that is an evolving work-in-process.

Pseudo-Cartestian Products

A Cartestian Product is what you get when you (accidentally) don't specify a where clause in the join between two tables in an RDBMS. It is the product of all rows in T1 times all rows in T2. You only get them if you completely forget to join two tables. It happens, but rarely. What is much more common is a flawed join, where you get a product between a subset of rows in each table, because you aren't using enough columns in the join.

We need a term for "imperfect join which results in a large number of rows, some of which are 'synthesized' and carry data which does not actually exist in the base tables".

Re-Factoring

Dave Winer: As I build layers of software, the simpler I make each layer, the higher I can build. If you don't design to hide complexity behind interfaces, you get overwhelmed by the complexity sooner, and your project can't do as much. Early in my career I often scrapped multiple levels and went down to the roots and rebuilt. Once I understood what the higher levels looked like, it exposed deficiencies in the lower levels. Reworking the lower levels allowed me to build higher.

This almost never happens with in-house corporate software development. Where I have seen large companies go wrong, repeatedly, is that they fail to find a way to start a large project by biting off a small chunk, learning from that, and using that hard-won knowledge to revise their approach. Instead, one way or another, sometimes in total ignorance, sometimes despite better intentions, they wind up making a huge, multi-year bet on a big-bang delivery. And in my experience, that typically leads to either total failure, in the form of outright cancellation, after several years and millions of dollars expended, or substantial failure, in the form of a stunted delivery accompanied by a pro-forma declaration of victory, after several years and millions of dollars expended[1].

Here are some reasons this happens:

1. The proof-of-concept is too small. This is probably the least common reason, but in this case, leadership is smart enough to realize that they need to test the waters. However, the test devised is much too small to provide the kind of reality check needed to provide insight into the much larger project.

2. Gross under-estimation of project scope. This is a common one. If you start out believing the project is medium-sized rather than jumbo, then you won't be thinking you need a significant trial-run. Two comments here. One, I personally feel like I have seen this happen enough to recognize the warning signs, so I would question whether this mistake could be avoided 80% of the time, if companies found a way to tap the experience of their senior people--including being open to bad news. Second comment--even if you accept that this will sometimes happen, despite best intentions, it is a problem that can be recognized before it is too late. The first time that 9-month project is extended to 15 months is a huge warning sign to take a big time-out, and ask--okay, what can we deliver in the original 9 months scope? Then we take stock and re-plan the project. And at this point, we face the facts--its not 9 months, its not 15 months, its going to be at least 24 months.

3. Insistence on business value. Several times I have seen companies that do want to break the delivery down into phases, but they are very insistent that even the very first phase must deliver business value (delvier ROI). This is a huge mistake. It just doesn't usually work this way, any more than the foundation to a house delivers significant value, without the rest of the house sitting on top of it. The tragic result of this approach is that the first pass winds up being way too big and high risk to start with, and more often than not gets expanded from there, as other stakeholders become involved in the project elaboration, and add more requirements.

Here is the right way to look at it. The only value that the pilot has to deliver is to give you the information you need to assess how big the full project really is (still guessing, but less wildly), whether it is worth doing, how much resources it might take to get done, and what different, creative approaches might be employed in planning the project. If you spend 10-30% of your estimated total budget on this phase, and the only result is that you conclude that the project should be scrapped, that is a HUGE win. Because in that case, you clearly were way off in your pre-project estimates, so the ultimate project budget would have been 3-5X that initial estimate--and you still probably wouldn't have gotten what you hoped for. Think of it like buying insurance--a relatively small and manageable outlay to provide a huge risk-mitigation.

4. Unwillingness to take the time to do it right. This is where the corporate reward system, as well as plain-old human impatience, take their toll. Experience tells us that major projects take years. Often, management is not willing to spend a year on a pilot, just to find out whether a larger multi-year project is feasible. Instead, they would rather pretend that the whole thing can be done in less than 2 years.

__________
NOTES

[1] Incidentally, the partial failure may be more expensive in the long run than the complete failure, because this is the kind of thing that leads to the old system remaining in place for certain functions that were cut from the new system. This contributes to the problem of an increasingly complex, rigid and expensive-to-maintain systems environment, which is also something that seems to burden many/most large companies.

Design Before Writing Requirements?

Not a bad article, I agree with a lot of it, this was the best part:

By nature domain understanding does not lead to specification. Understanding suggests design. A very important part of design is limits. When design is allowed to bubble up from understanding it’s easy to put the limit of the design into places that are logical for the domain. In contrast to design, requirement definition is limitless. The requirements gathering process typically goes on until someone says “I think we’ve got enough now”. That arbitrary stopping point becomes a design limit and hobbles further system development.
...
The solution is to write the requirements after the design is created so that the scope will be defined.

Risk Issue Problem

In software development, people tend to use the following definitions:

Issue: something that will cause complications or setbacks for the project.

Risk: something that has the potential to cause complications or setbacks for the project, if it occurs; and there is no plan to actively prevent it.

Under these definitions, an Issue is more severe than a Risk.

To me, that is backward:

Risk: something that has the potential to cause complications or setbacks for the project, if it occurs; and there is no ability to completely be sure of preventing it.

Issue: a complicating detail to attend to. Something intrinsically manageable, however, so that as long as it is addressed as planned, there will be no significant negative consequences to the project.

Problem: a complicating detail which is not routine, and presents a significant challenge to overcome. This seems to be what most people want to call an "issue".

It seems that the general lingusitic trend of replacing the word "problem" with "issue" has crept into software development.

Observation on Test-Driven Development

With TDD, I think probably half the benefit is the process benefit of wrapping your mind around the problem and thinking through the test cases. As opposed to the tangible benefit of having the test cases in hand, and the test automation developed (which I think also is significant).